Skip to content

fix: Fix some phones unable to play audio #2728

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shaohuzhang1 merged 1 commit into from
Mar 28, 2025
Merged

fix: Fix some phones unable to play audio #2728

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 24 additions & 26 deletions ui/src/components/ai-chat/component/operation-button/ChatOperationButton.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,15 +94,6 @@
</el-button>
</el-tooltip>
</span>

<!-- 先渲染，不然不能播放 -->
<audio
ref="audioPlayer"
v-for="item in audioList"
:key="item"
controls
hidden="hidden"
></audio>
<div ref="audioCiontainer"></div>
</div>
</div>
Expand Down Expand Up @@ -293,7 +284,7 @@ class AudioManage {
index = this.textList.length - 1
if (this.ttsType === 'TTS') {
const audioElement: HTMLAudioElement = document.createElement('audio')
audioElement.controls = true
audioElement.controls = false
audioElement.hidden = true
/**
* 播放结束事件
Expand Down Expand Up @@ -436,7 +427,21 @@ class AudioManage {
}

const audioElement = this.audioList[index]
if (audioElement instanceof SpeechSynthesisUtterance) {

if (audioElement instanceof HTMLAudioElement) {
// 标签朗读
try {
this.statusList[index] = AudioStatus.PLAY_INT
const play = audioElement.play()
if (play instanceof Promise) {
play.catch((e) => {
this.statusList[index] = AudioStatus.READY
})
}
} catch (e: any) {
this.statusList[index] = AudioStatus.ERROR
}
} else {
if (window.speechSynthesis.paused) {
window.speechSynthesis.resume()
} else {
Expand All @@ -446,14 +451,6 @@ class AudioManage {
speechSynthesis.speak(audioElement)
this.statusList[index] = AudioStatus.PLAY_INT
}
} else {
// 标签朗读
try {
audioElement.play()
this.statusList[index] = AudioStatus.PLAY_INT
} catch (e) {
this.statusList[index] = AudioStatus.ERROR
}
}
}
pause(self?: boolean) {
Expand All @@ -462,7 +459,14 @@ class AudioManage {
return
}
const audioElement = this.audioList[index]
if (audioElement instanceof SpeechSynthesisUtterance) {

if (audioElement instanceof HTMLAudioElement) {
if (this.statusList[index] === AudioStatus.PLAY_INT) {
// 标签朗读
this.statusList[index] = AudioStatus.READY
audioElement.pause()
}
} else {
this.statusList[index] = AudioStatus.READY
if (self) {
window.speechSynthesis.pause()
Expand All @@ -474,12 +478,6 @@ class AudioManage {
} else {
window.speechSynthesis.cancel()
}
} else {
if (this.statusList[index] === AudioStatus.PLAY_INT) {
// 标签朗读
this.statusList[index] = AudioStatus.READY
audioElement.pause()
}
}
}
getTextList(text: string, is_end: boolean) {
Copy link
Copy Markdown
Contributor Author

@shaohuzhang1 shaohuzhang1 Mar 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several issues and suggestions with the provided code:

  1. Multiple Instances of Audio Elements: The line <audio ref="audioPlayer" v-for="item in audioList" ...> generates multiple audio elements on each render, which is inefficient and could lead to performance problems.

  2. Conditional Rendering: Ensure that only one instance of div ref="audioCiontainer" is used unless you need dynamic rendering based on some condition.

  3. Speech Synthesis vs HTML Audio Element: The logic is inconsistent between handling SpeechSynthesisUtterance and HTMLAudioElement instances. This leads to redundant checks and can be streamlined.

  4. Error Handling: There's no error handling when creating a new HTMLAudioElement, which should use document.createElement('audio') instead of assigning it directly to a ref using Vue.js syntax.

  5. State Management: The statusList management seems scattered around different methods without consistent usage guidelines. Consider consolidating state management for better readability and maintainability.

  6. Method Names and Signatures: Method names like pause and logic inside them might not work as expected because they don't handle certain states correctly (e.g., PLAY_INT and other enum values).

  7. Code Style: There are minor style improvements that could enhance readability and consistency throughout the file.

Here are cleaned-up version recommendations:

// Remove the repeated div element and single instance control
<div ref="audioPanel">
  <!-- El button -->
</div>

<!-- Single container for all audio elements -->
<div ref="audioContainer"></div>

<script lang="ts">
import { defineComponent } from 'vue';
class AudioManage {
  textList: string[] = [];
  ttsType: 'TTS' | 'TAG_READING' = ''; // Add type annotation here
  statusList: number[] = []; // Define statuses

  async loadAudioItems() {
    await createHtmlElements(this.textList.map(item =>
      this.createAudioElement(item)
    ));
  }

  private createAudioElement(text: string): HTMLElement {
    const audio = document.createElement('audio');
    audio.controls = true;
    audio.hidden = true;
    const utterance = new SpeechSynthesisUtterance(text);
    // Handle playback events if needed
    return audio;
  }
}

function createHtmlElements(elements: HTMLElement[]): Array<HTMLDivElement> {
  const divContainer = document.createElement('div');
  const parentEl = document.getElementById('parentElementId'); // Replace with actual ID
  elements.forEach((element) => {
    divContainer.appendChild(element);
  });
  parentEl.appendChild(divContainer);
}
</script>

Please adapt the above changes according to your project structure and requirements.

Expand Down
5 changes: 4 additions & 1 deletion ui/src/components/ai-chat/index.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -523,7 +523,10 @@ onMounted(() => {
let userFormData = JSON.parse(localStorage.getItem(`${accessToken}userForm`) || '{}')
form_data.value = userFormData
}
window.speechSynthesis.cancel()
if (window.speechSynthesis) {
window.speechSynthesis.cancel()
}

window.sendMessage = sendMessage
bus.on('on:transcribing', (status: boolean) => {
transcribing.value = status
Copy link
Copy Markdown
Contributor Author

@shaohuzhang1 shaohuzhang1 Mar 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code snippet provided is almost correct, with only one minor improvement. The speechSynthesis object might not be defined in some environments when the script runs, so adding a null check before calling its method helps prevent errors.

Optimization Suggestion:

Add an early return after checking if window.speechSynthesis exists to avoid unnecessary code execution.

Here's the updated code:

@@ -523,7 +523,10 @@ onMounted(() => {
     let userFormData = JSON.parse(localStorage.getItem(`${accessToken}userForm`) || '{}')
     form_data.value = userFormData
   }
+  if (window.speechSynthesis) {
     window.speechSynthesis.cancel() // Ensure this happens safely
-  }
+    window.sendMessage = sendMessage;
+    bus.on('on:transcribing', (status: boolean) => {
+      transcribing.value = status;
+    });
   }

This change ensures that the rest of the function can proceed without relying implicitly on the presence of window.speechSynthesis.

Expand Down