Skip to content

[Suggestion] OpenAI Agents SDK with Deepgram STT and TTS as tool providers (Python) #288

Description

@deepgram-robot

What to build

A working example integrating Deepgram's STT and TTS APIs as tool providers within the OpenAI Agents SDK. The agent should be able to transcribe audio files/streams and generate speech from text as part of its tool-calling workflow — for example, an agent that listens to a voice message, transcribes it, reasons about the content, and responds with synthesized speech.

Why this matters

The OpenAI Agents SDK is becoming a standard for building tool-using AI agents, and developers need voice capabilities in their agent pipelines. There's no existing example showing Deepgram as a tool provider in this framework. By providing a reference implementation, developers building OpenAI Agents SDK projects can add audio transcription and speech synthesis without evaluating alternatives — Deepgram becomes the default voice layer.

Suggested scope

  • Language: Python
  • Framework: OpenAI Agents SDK (latest)
  • Deepgram APIs: Pre-recorded STT (Nova-3), TTS (Aura), optionally streaming STT
  • Tools: transcribe_audio tool (file/URL → transcript), speak_text tool (text → audio file)
  • Demo scenario: Voice message assistant that transcribes incoming audio, extracts key information, and responds with spoken audio
  • Complexity: Intermediate — demonstrates tool definition, registration, and agent orchestration

Acceptance criteria

  • Runnable with minimal setup (clone, add API keys, run)
  • README explains the pattern clearly
  • Uses current Deepgram SDK version and OpenAI Agents SDK
  • Tools are properly typed with Pydantic models
  • Includes sample audio for testing the transcription tool
  • Shows both STT and TTS integration in a single agent workflow

Raised by the DX intelligence system.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions