Skip to content

perf: Optimization of recording function on mobile devices #2719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shaohuzhang1 merged 1 commit into from
Mar 28, 2025
Merged

perf: Optimization of recording function on mobile devices #2719

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions ui/src/components/ai-chat/component/chat-input-operate/index.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@
<div class="operate flex align-center">
<template v-if="props.applicationDetails.stt_model_enable">
<span v-if="mode === 'mobile'">
<el-button text @click="isMicrophone = !isMicrophone">
<el-button text @click="switchMicrophone(!isMicrophone)">
<!-- 键盘 -->
<AppIcon v-if="isMicrophone" iconName="app-keyboard"></AppIcon>
<el-icon v-else>
Expand Down Expand Up @@ -444,15 +444,19 @@ const isDisabledChat = computed(
)
// 是否显示移动端语音按钮
const isMicrophone = ref(false)
watch(isMicrophone, (value: boolean) => {
if (value) {
const switchMicrophone = (status: boolean) => {
if (status) {
// 如果显示就申请麦克风权限
recorderManage.open()
recorderManage.open(() => {
isMicrophone.value = true
})
} else {
// 关闭麦克风
recorderManage.close()
isMicrophone.value = false
}
})
}

const TouchEnd = (bool: Boolean) => {
if (bool) {
stopRecording()
Expand All @@ -471,7 +475,7 @@ class RecorderManage {
constructor(uploadRecording: (blob: Blob, duration: number) => void) {
this.uploadRecording = uploadRecording
}
open() {
open(callback?: () => void) {
const recorder = new Recorder({
type: 'mp3',
bitRate: 128,
Expand All @@ -480,6 +484,9 @@ class RecorderManage {
if (!this.recorder) {
recorder.open(() => {
this.recorder = recorder
if (callback) {
callback()
}
}, this.errorCallBack)
}
}
Expand Down Expand Up @@ -556,7 +563,7 @@ const uploadRecording = async (audioBlob: Blob) => {
try {
// 非自动发送切换输入框
if (!props.applicationDetails.stt_autosend) {
isMicrophone.value = false
switchMicrophone(false)
}
recorderStatus.value = 'TRANSCRIBING'
const formData = new FormData()
Expand All @@ -572,7 +579,7 @@ const uploadRecording = async (audioBlob: Blob) => {
autoSendMessage()
})
} else {
isMicrophone.value = false
switchMicrophone(false)
}
})
.catch((error) => {
Copy link
Copy Markdown
Contributor Author

@shaohuzhang1 shaohuzhang1 Mar 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provided code seems to be correctly implementing a mobile audio recording feature using El-Cli components. However, here are some recommendations for improvement:

1. Error Handling

Ensure that all error handling is comprehensive. The current implementation doesn't have extensive error handling logic.

2. Callback Function

In the open method of RecorderManage, consider adding more detailed logging or notifications when callbacks are used.

3. Cleanup and Rescanning Permissions

If permissions change after initialization, add mechanisms such as rescans during permission denial scenarios to ensure functionality remains consistent.

4. Refactoring

Consider refactoring repetitive code patterns like the microphone toggle button into a reusable component if applicable.

Suggestions

Improved Audio Recording Management:

class RecorderManage {
  private recorder?: Recordable
  uploadRecording: (blob: Blob, duration: number) =>void;

  constructor(uploadRecording: (blob: Blob, duration: number) => void) {
    this.uploadRecording = uploadRecording;
  }

  open(callback?: () => void): Promise<void> {
    return new Promise((resolve, reject) => {
      const config: RecorderConfig = {
        type: 'mp3',
        bitRate: 128,
      };

      Recorder.create(config).then(recorder => {
        this.recorder = recorder;
        recorder.start();
        resolve();

        if (callback) {
          callback();
        }
      }).catch(error => {
        console.error("Failed to start recording:", error);
        reject(error);
      });

      setTimeout(() => {
        recorder.stop().then(blob => {
          let startTime = Date.now() - timestamp;
          startTime /= 1000; // Convert milliseconds to seconds
          uploadRecording(blob, startTime);
          this.recorder!.stop(); // Explicitly stop again to release resources
          this.recorder = undefined;
          resolve();
        }).catch(error => {
          console.error("Failed to process recorded file:", error);
          reject(error);
        });
      }, recTime * SECOND);
    }); // Close Promise
  }

  close(): void {
    this.recorder?.clearDataForTest();
    this.recorder?.destroy();
    this.recorder = undefined;
  }

  destroy(): void {
    this.close();
  }

  private errorHandler(errType: string | null, errStack: StackFrame[]): void {
    throw new Error(`${errType}: ${JSON.stringify(stack)}\n`);
  }

  private errorCallBack(e: unknown): void {
    if (!(e instanceof DOMException)) {
      throw e;
    }
    console.warn("Error during recording:", { e });
  }
}

Enhanced Code Clarity:

  • Ensure comments explain key parts of the code.
  • Consider breaking down complex logic into smaller functions for better readability.

This revised version addresses some issues with error management and provides additional flexibility in managing the recording lifecycle. Adjustments may need to be made based on specific requirements and application context.

Expand Down