Accessibility: Multilingual Live Captions

[stefnotch]

Is your feature request related to a problem? Please describe.

As a person with a hearing disability, having subtitles is extremely valuable to me. I speak both German and English, so I need both languages to be supported.

Websites like YouTube consistently offer auto generated subtitles of acceptable quality. However, when I am in Zoom or Teams meetings, there is usually no such option. I believe Zoom actually has it as a feature, but none of my university professors ever managed to enable that.
The same problem applies to Discord calls, where there just is no built-in subtitles option.

Another situation where I struggle is when watching recordings provided to me by my university. They almost always lack subtitles. I have talked to the accessibility offices about it, but their technical means are limited. In the end, I often have to spend an hour manually downloading the recordings and generating captions via Whisper.

This is a terrible experience and some situations, like the Discord calls, simply have no solution.

Describe the solution you'd like

Windows has a very usable live captions feature built in. It supports multiple languages, unlike the https://github.com/abb128/LiveCaptions project.

I would love to see a live captions application with support for multiple languages, including English and German, in Linux Mint! Currently Linux is not an accessible experience for me at all.

[ghost]

On flathub thier is a live captions program called "live captions" , and I belive it also does german. You can install it from the software manager.

For english it works pretty much perfectly, and done 100% localy on your computer, as for how well it handles languages other then english I have no idea as I only speak english.

And yes live captions should become a standard feature, because even if you have perfect hearing being able to watch something in a loud envirment without fancy headphones, or adding to the noise is a great benifet of it's own.

[stefnotch]

I'm afraid that I haven't been able to figure out how to make it work with German. One of the first lines in the readme says the following:

Only the English language is supported currently. Other languages may produce gibberish or a bad phonetic translation.
-- https://github.com/abb128/LiveCaptions

The feature request for the German language is abb128/LiveCaptions#86

[ghost]

Well that not great, well atleast it will cover half of your needs right? Since it is the only working solution I have ever heard of in the linux space the choices are rather sparce.

[stefnotch]

Yes, it does. But that still means frequently having to switch back to Windows. Hence the multilingual live captions request.

[ghost]

I'm not refuting your request just saying what you want 1, barely exists, and 2, what does exist usally can bearly handle their target language let alone anything else.

Making a live caption tool requires voice samples, testing, and someone to check it's work, and someone else to actually code stuff.

Guess how many people want to do all of that, plus deal with something as complex as language and all of it's confusing complexities for free?

Add to that mess the fact someone needs to actually care about improving it and it's a mirricle live captions in english is even an option.

My point is if you want this feature to ever exist prepare do some work yourself otherwise you will beholding your breath for a long time.

But I will root for you getting your wishes someday.