Skip to content

Add support for audio frame processor#4145

Merged
lukasIO merged 20 commits into
mainfrom
lukas/processor-support
Dec 19, 2025
Merged

Add support for audio frame processor#4145
lukasIO merged 20 commits into
mainfrom
lukas/processor-support

Conversation

@lukasIO

@lukasIO lukasIO commented Dec 2, 2025

Copy link
Copy Markdown
Contributor

Comment thread livekit-agents/livekit/agents/voice/room_io/_input.py
and self._room._token is not None
and self._room._server_url is not None
):
self._processor._update_credentials(token=self._room._token, url=self._room._server_url)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think the livekit-rtc package should be responsible for directly updating the credentials?

@lukasIO lukasIO Dec 6, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree, in theory that would be nice.

I wasn't sure what the best way would be for handling the possibility of the input stream changing here further down in the rtc SDK. More than the credential update this would be about the stream info update.

Any thoughts on this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a look at this from the rtc SDK side.

One challenge I see with shifting responsibility for this to the rtc SDK is that there's no control from within the rtc SDK over when/how users construct audio streams, do you have an idea how to achieve a tighter integration between stream and the room instance?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, yes, it's probably fine for now.
Maybe in the future, rtc.Room should keep track of every rtc.AudioStream, I don't see an easy way to do it today tho

@xianshijing-lk xianshijing-lk left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm in general, one small things to get your attention

Comment thread livekit-agents/livekit/agents/voice/room_io/_input.py

@theomonnom theomonnom left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! but how are users supposed to use this API?
RoomInput isn't exposed to public, only RoomIO

@lukasIO

lukasIO commented Dec 11, 2025

Copy link
Copy Markdown
Contributor Author

lgtm! but how are users supposed to use this API?

forgot to add the type to the AudioInputOptions 🤦
I've now added it also for Ben's new NoiseCancellationSelector

@lukasIO lukasIO force-pushed the lukas/processor-support branch from 5fa77a4 to f2657ec Compare December 11, 2025 10:57
@lukasIO

lukasIO commented Dec 19, 2025

Copy link
Copy Markdown
Contributor Author

ruff checks are failing on main as well due to the durable stuff, going to merge this

@lukasIO lukasIO merged commit 0bfa690 into main Dec 19, 2025
17 of 18 checks passed
@lukasIO lukasIO deleted the lukas/processor-support branch December 19, 2025 10:43
wasaybaig201 pushed a commit to wasaybaig201/agents that referenced this pull request Dec 23, 2025
…#4145)

Co-authored-by: Jacob Gelman <3182119+ladvoc@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants