Add a speech Lua library for screen reader and brail output#4762
Add a speech Lua library for screen reader and brail output#4762RealAmethyst wants to merge 2 commits into
Conversation
This ads text to speech and brail support to lua scripting. This is cross platform on Windows, MacOS and Linux
This lets screen readers and other speech libraries interfase with the emulator for lua scripting support.
|
(There is a .NET bindings package for this library, but it's .NET Core only.) I think, despite your intentions, that this will crash on Linux and possibly Windows < Win10. It will have to go through one of our helpers like |
YoshiRulz
added
the
re: Lua API/scripting
YoshiRulz
left a comment
YoshiRulz
left a comment
There was a problem hiding this comment.
I'm also wondering how to deal with attribution, linking messages to the script that produced them. With console.writeline, we've just accepted that the script author decides whether to include an identifying message, but we are now including the filename when logging to stdout. Maybe the first time a script tries to announce something, we could prepend "script <scriptname> has begun using TTS"?
| [LuaMethod("say", "Speaks the given text through the active screen reader (or SAPI fallback). When interrupt is true (the default), any current speech is cancelled first.")] | ||
| public void Say(string text, bool interrupt = true) | ||
| { | ||
| if (!PrismSpeech.Speak(text ?? string.Empty, interrupt, out var error)) Log($"speech.say failed: {error}"); |
There was a problem hiding this comment.
Is it reasonable to announce an empty (or all-whitespace) string?
There was a problem hiding this comment.
I mean there is no reason why speech should announce empty stuff.
There was a problem hiding this comment.
I occasionally use print to see some information and it ends up being actually empty, usually due to a bug in my script. In this case I can still see an empty line once the next print happens. It'd be much more frustrating if the print call did nothing, and I imagine it will also be very frustrating if a speech call that accidentally contains a blank string is ignored.
There was a problem hiding this comment.
I actually didn't think about that, and yeah you're right, that would be frustrating, so having it still speak it is the best.
There was a problem hiding this comment.
In this case I can still see an empty line once the next print happens.
Explain how that applies to TTS.
There was a problem hiding this comment.
I mean, I would be curious how TTS handles the empty string, but SuuperW's "debug empty log messages" use-case seems relevant whether you can see the screen or not.
4 participants
Adds a universal
speechLua library so that lua scripts can utilize screen readers and brail directly, opening games up to accessibility scripting for the blind. Methods: speech.say(text, interrupt),speech.output(text, interrupt) (speak + braille), speech.braille(text), speech.stop().
Backed by prism (https://github.com/ethindp/prism, MPL-2.0), which routes to the active screen reader
(NVDA, JAWS, Narrator/OneCore, ...) with a SAPI fallback, via P/Invoke to its C API. The native prism.dll
(win-x64) is added under Assets/dll. Initialisation is lazy and failures are non-fatal
(calls become no-ops), so a missing DLL can't crash EmuHawk