feat: Add OpenAI Custom Voice support for stable Audio API#2744
Draft
snopoke wants to merge 8 commits into
Draft
feat: Add OpenAI Custom Voice support for stable Audio API#2744snopoke wants to merge 8 commits into
snopoke wants to merge 8 commits into
Conversation
Phase 1 of OpenAI custom voice migration: - Add OpenAICustomVoice to SyntheticVoice.SERVICES and TEAM_SCOPED_SERVICES - Add config JSONField to SyntheticVoice for storing voice_id and consent_id - Add helper methods get_openai_voice_id() and get_openai_consent_id() - Add openai_custom_voice to VoiceProviderType enum - Increase service field max_length from 17 to 20 chars - Update model tests for new team-scoped service
Phase 2 of OpenAI custom voice migration: - Add OpenAICustomVoiceClient with consent and voice management APIs - Add VoiceConsent and CustomVoice dataclasses for API responses - Add OpenAICustomVoiceSpeechService using stable /v1/audio/speech API - Support voice ID-based synthesis with optional instructions - Add comprehensive tests for API client and speech service
Phase 3 of OpenAI custom voice implementation: - Add OpenAICustomVoiceFileFormset with file extension/size validation - Add OpenAICustomVoiceConfigForm extending OpenAIConfigForm - Add get_custom_voice_client method to VoiceProvider model - Add tests for VoiceProvider client method Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Phase 4 & 5 of OpenAI custom voice implementation: - Add views for listing, creating, and deleting custom voices - Add views for managing voice consents - Add URL patterns for custom voice management endpoints - Create templates for voice and consent management UI Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Contributor
📝 WalkthroughWalkthroughThis PR implements comprehensive OpenAI Custom Voice API support across the platform. It adds a new Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Fix all issues with AI agents
In `@apps/service_providers/views.py`:
- Around line 490-503: The current flow calls provider.get_custom_voice_client()
and client.delete_voice(openai_voice_id) but always proceeds to call
voice.delete() and messages.success even when the remote delete failed; change
the logic in the view so that after calling client.delete_voice(openai_voice_id)
you only delete the local DB record (voice.delete()) and show messages.success
when the remote deletion either succeeds or returns a not-found/404; if the
remote call raises any other exception, do not delete the local voice, log the
error via log.error/warning, and surface an error message to the user (use
messages.error) so they know remote cleanup failed; if openai_voice_id is falsy,
continue to delete locally as before. Ensure you reference
provider.get_custom_voice_client, client.delete_voice, voice.delete, and
messages.success/messages.error when locating the code to modify.
- Around line 417-441: The code calls provider.get_custom_voice_client() and
client.create_voice(...) before persisting SyntheticVoice with
SyntheticVoice.objects.create(...), risking orphaned remote voices if the DB
insert fails; modify the flow to first check for existing SyntheticVoice
duplicates (by voice_name/consent_id/service/voice_provider) and only call
client.create_voice when none exists, or if you must create remotely first, wrap
SyntheticVoice.objects.create in a try/except that on exception calls
client.delete_voice(voice.id) (or the provider's equivalent) to clean up the
remote voice and then re-raise the exception; ensure you reference
provider.get_custom_voice_client, client.create_voice, client.delete_voice, and
SyntheticVoice.objects.create when making the changes.
In `@templates/service_providers/custom_voice/list_voices.html`:
- Line 5: The wrapper div using the Tailwind Typography class "prose" (in
templates/service_providers/custom_voice/list_voices.html) lacks dark-mode
inversion; update the class attribute on the <div class="prose max-w-none"> to
include dark:prose-invert (or the DaisyUI equivalent) so typography inverts
correctly in dark theme and ensures all text remains high-contrast across
themes.
🧹 Nitpick comments (4)
apps/service_providers/forms.py (1)
115-123: Annotate class-level file limits asClassVar
accepted_file_typesandmax_file_size_mbare class-level configs and trigger Ruff RUF012. Mark them asClassVar(or use tuples) to avoid accidental mutation.♻️ Suggested fix
+from typing import ClassVar @@ -class OpenAICustomVoiceFileFormset(BaseFileFormSet): +class OpenAICustomVoiceFileFormset(BaseFileFormSet): @@ - accepted_file_types = ["mp3", "wav", "ogg", "aac", "flac", "webm", "mp4", "mpeg"] - max_file_size_mb = 10 + accepted_file_types: ClassVar[tuple[str, ...]] = ( + "mp3", + "wav", + "ogg", + "aac", + "flac", + "webm", + "mp4", + "mpeg", + ) + max_file_size_mb: ClassVar[int] = 10apps/experiments/models.py (1)
476-484: MakeTEAM_SCOPED_SERVICESimmutableA mutable class list can be changed at runtime and triggers Ruff RUF012. Prefer a tuple +
ClassVar.♻️ Suggested fix
-from typing import Self +from typing import ClassVar, Self @@ - TEAM_SCOPED_SERVICES = [OpenAIVoiceEngine, OpenAICustomVoice] + TEAM_SCOPED_SERVICES: ClassVar[tuple[str, ...]] = (OpenAIVoiceEngine, OpenAICustomVoice)apps/service_providers/views.py (1)
261-286: Silence unusedteam_slugparameters (Ruff ARG001)These views don’t use
team_slug; rename to_team_slug(or adddel team_slug) to satisfy lint without changing the URL signature.♻️ Suggested fix
-def list_custom_voice_consents(request, team_slug: str, pk: int): +def list_custom_voice_consents(request, _team_slug: str, pk: int): @@ -def list_custom_voices(request, team_slug: str, pk: int): +def list_custom_voices(request, _team_slug: str, pk: int):Also applies to: 355-390
apps/service_providers/openai_custom_voice.py (1)
58-74: MarkCONSENT_PHRASESasClassVarThe class-level dict is mutable and triggers Ruff RUF012. Annotate it as
ClassVar(or make it immutable) to prevent accidental mutation.♻️ Suggested fix
+from typing import ClassVar @@ - CONSENT_PHRASES = { + CONSENT_PHRASES: ClassVar[dict[str, str]] = { "en": ( "I am the owner of this voice and I consent to OpenAI using this voice to create a synthetic voice model." ),
Comment on lines
+417
to
+441
| try: | ||
| client = provider.get_custom_voice_client() | ||
| voice = client.create_voice( | ||
| name=voice_name, | ||
| consent_id=consent_id, | ||
| audio_sample_file=audio_sample.file, | ||
| filename=audio_sample.name, | ||
| ) | ||
|
|
||
| # Create SyntheticVoice record | ||
| SyntheticVoice.objects.create( | ||
| name=voice_name, | ||
| neural=True, | ||
| language="", | ||
| language_code="", | ||
| gender="", | ||
| service=SyntheticVoice.OpenAICustomVoice, | ||
| voice_provider=provider, | ||
| config={ | ||
| "voice_id": voice.id, | ||
| "consent_id": consent_id, | ||
| "created_at": voice.created_at, | ||
| "model": "gpt-4o-mini-tts", | ||
| }, | ||
| ) |
Contributor
There was a problem hiding this comment.
Avoid orphaned OpenAI voices if DB insert fails
The remote voice is created before the local SyntheticVoice. If the DB insert fails (unique constraint/validation), the remote voice remains and counts toward the org limit. Consider checking for duplicates before the API call or deleting the remote voice on failure.
🛡️ Suggested cleanup on DB failure
-from django.db import transaction
+from django.db import IntegrityError, transaction
@@
- voice = client.create_voice(
+ voice = client.create_voice(
name=voice_name,
consent_id=consent_id,
audio_sample_file=audio_sample.file,
filename=audio_sample.name,
)
- # Create SyntheticVoice record
- SyntheticVoice.objects.create(
- name=voice_name,
- neural=True,
- language="",
- language_code="",
- gender="",
- service=SyntheticVoice.OpenAICustomVoice,
- voice_provider=provider,
- config={
- "voice_id": voice.id,
- "consent_id": consent_id,
- "created_at": voice.created_at,
- "model": "gpt-4o-mini-tts",
- },
- )
+ try:
+ SyntheticVoice.objects.create(
+ name=voice_name,
+ neural=True,
+ language="",
+ language_code="",
+ gender="",
+ service=SyntheticVoice.OpenAICustomVoice,
+ voice_provider=provider,
+ config={
+ "voice_id": voice.id,
+ "consent_id": consent_id,
+ "created_at": voice.created_at,
+ "model": "gpt-4o-mini-tts",
+ },
+ )
+ except IntegrityError:
+ client.delete_voice(voice.id)
+ raise📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| try: | |
| client = provider.get_custom_voice_client() | |
| voice = client.create_voice( | |
| name=voice_name, | |
| consent_id=consent_id, | |
| audio_sample_file=audio_sample.file, | |
| filename=audio_sample.name, | |
| ) | |
| # Create SyntheticVoice record | |
| SyntheticVoice.objects.create( | |
| name=voice_name, | |
| neural=True, | |
| language="", | |
| language_code="", | |
| gender="", | |
| service=SyntheticVoice.OpenAICustomVoice, | |
| voice_provider=provider, | |
| config={ | |
| "voice_id": voice.id, | |
| "consent_id": consent_id, | |
| "created_at": voice.created_at, | |
| "model": "gpt-4o-mini-tts", | |
| }, | |
| ) | |
| try: | |
| client = provider.get_custom_voice_client() | |
| voice = client.create_voice( | |
| name=voice_name, | |
| consent_id=consent_id, | |
| audio_sample_file=audio_sample.file, | |
| filename=audio_sample.name, | |
| ) | |
| # Create SyntheticVoice record | |
| try: | |
| SyntheticVoice.objects.create( | |
| name=voice_name, | |
| neural=True, | |
| language="", | |
| language_code="", | |
| gender="", | |
| service=SyntheticVoice.OpenAICustomVoice, | |
| voice_provider=provider, | |
| config={ | |
| "voice_id": voice.id, | |
| "consent_id": consent_id, | |
| "created_at": voice.created_at, | |
| "model": "gpt-4o-mini-tts", | |
| }, | |
| ) | |
| except IntegrityError: | |
| client.delete_voice(voice.id) | |
| raise |
🤖 Prompt for AI Agents
In `@apps/service_providers/views.py` around lines 417 - 441, The code calls
provider.get_custom_voice_client() and client.create_voice(...) before
persisting SyntheticVoice with SyntheticVoice.objects.create(...), risking
orphaned remote voices if the DB insert fails; modify the flow to first check
for existing SyntheticVoice duplicates (by
voice_name/consent_id/service/voice_provider) and only call client.create_voice
when none exists, or if you must create remotely first, wrap
SyntheticVoice.objects.create in a try/except that on exception calls
client.delete_voice(voice.id) (or the provider's equivalent) to clean up the
remote voice and then re-raise the exception; ensure you reference
provider.get_custom_voice_client, client.create_voice, client.delete_voice, and
SyntheticVoice.objects.create when making the changes.
Comment on lines
+490
to
+503
| try: | ||
| # Delete from OpenAI first if we have a voice_id | ||
| if openai_voice_id: | ||
| client = provider.get_custom_voice_client() | ||
| try: | ||
| client.delete_voice(openai_voice_id) | ||
| except Exception as e: | ||
| log.warning(f"Could not delete voice from OpenAI: {e}") | ||
| # Continue to delete from DB even if OpenAI deletion fails | ||
|
|
||
| # Delete from database | ||
| voice.delete() | ||
| messages.success(request, f"Voice '{voice_name}' deleted successfully") | ||
| except Exception as e: |
Contributor
There was a problem hiding this comment.
Don’t report success when remote deletion fails
If the OpenAI delete call errors, the code still deletes the DB record and shows success. This can leave an orphaned remote voice and mislead users. Consider only deleting locally on remote success (or 404) and surface a warning otherwise.
🛠️ Suggested flow
- if openai_voice_id:
- client = provider.get_custom_voice_client()
- try:
- client.delete_voice(openai_voice_id)
- except Exception as e:
- log.warning(f"Could not delete voice from OpenAI: {e}")
- # Continue to delete from DB even if OpenAI deletion fails
-
- # Delete from database
- voice.delete()
- messages.success(request, f"Voice '{voice_name}' deleted successfully")
+ deleted_remote = True
+ if openai_voice_id:
+ client = provider.get_custom_voice_client()
+ try:
+ deleted_remote = client.delete_voice(openai_voice_id)
+ except Exception as e:
+ log.warning(f"Could not delete voice from OpenAI: {e}")
+ deleted_remote = False
+
+ if deleted_remote:
+ voice.delete()
+ messages.success(request, f"Voice '{voice_name}' deleted successfully")
+ else:
+ messages.error(
+ request,
+ "OpenAI deletion failed; the voice was not removed locally so you can retry.",
+ )🧰 Tools
🪛 Ruff (0.14.14)
[warning] 496-496: Do not catch blind exception: Exception
(BLE001)
🤖 Prompt for AI Agents
In `@apps/service_providers/views.py` around lines 490 - 503, The current flow
calls provider.get_custom_voice_client() and
client.delete_voice(openai_voice_id) but always proceeds to call voice.delete()
and messages.success even when the remote delete failed; change the logic in the
view so that after calling client.delete_voice(openai_voice_id) you only delete
the local DB record (voice.delete()) and show messages.success when the remote
deletion either succeeds or returns a not-found/404; if the remote call raises
any other exception, do not delete the local voice, log the error via
log.error/warning, and surface an error message to the user (use messages.error)
so they know remote cleanup failed; if openai_voice_id is falsy, continue to
delete locally as before. Ensure you reference provider.get_custom_voice_client,
client.delete_voice, voice.delete, and messages.success/messages.error when
locating the code to modify.
| {% load i18n %} | ||
|
|
||
| {% block app %} | ||
| <div class="prose max-w-none"> |
Contributor
There was a problem hiding this comment.
Add dark‑mode typography for the prose container
Tailwind Typography’s prose class doesn’t invert in dark themes, so text can become low‑contrast. Add dark:prose-invert (or a DaisyUI equivalent) on the wrapper.
As per coding guidelines, Ensure all UI components support both light and dark modes for theme compatibility.
✅ Suggested fix
-<div class="prose max-w-none">
+<div class="prose max-w-none dark:prose-invert">📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| <div class="prose max-w-none"> | |
| <div class="prose max-w-none dark:prose-invert"> |
🤖 Prompt for AI Agents
In `@templates/service_providers/custom_voice/list_voices.html` at line 5, The
wrapper div using the Tailwind Typography class "prose" (in
templates/service_providers/custom_voice/list_voices.html) lacks dark-mode
inversion; update the class attribute on the <div class="prose max-w-none"> to
include dark:prose-invert (or the DaisyUI equivalent) so typography inverts
correctly in dark theme and ensures all text remains high-contrast across
themes.
❌ 1 Tests Failed:
View the top 1 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
Extract BaseOpenAISpeechService to eliminate duplicated _client, _transcribe_audio, and config fields across three OpenAI speech services. Refactor views to use Django Form classes instead of manual POST extraction, adding server-side file validation and language whitelist checking. Sanitize API error messages shown to users, add CustomVoiceConfig TypedDict for typed config access, and add view tests covering create, delete (including two-phase OpenAI+DB), and error paths. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add a "Manage Voices" action button (microphone icon) to the voice provider table rows, visible only for OpenAI Custom Voice providers. This links directly to the custom voice management page, making the feature discoverable from the team settings UI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates OpenAI audio integration from the deprecated beta Voice Engine API (
/v1/audio/synthesize) to the current stable OpenAI Audio API (/v1/audio/speech) with custom voice support.OpenAICustomVoiceservice type alongside existingOpenAIVoiceEngineconfigJSONFieldKey Changes
Models & Migration:
OpenAICustomVoiceservice type inSyntheticVoiceconfigJSONField for storing voice metadataget_openai_voice_id(),get_openai_consent_id()API Client:
OpenAICustomVoiceClientfor consent/voice CRUD operationsVoiceConsentandCustomVoiceresponsesSpeech Service:
OpenAICustomVoiceSpeechServiceusing stable/v1/audio/speechAPIinstructionsparameter for gpt-4o-mini-tts modelViews & UI:
Test plan
Notes
OpenAIVoiceEngineremains excluded by default (feature flag)🤖 Generated with Claude Code