You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+1-14Lines changed: 1 addition & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,19 +18,6 @@
18
18
19
19
---
20
20
21
-
> [!IMPORTANT]
22
-
> Full response time (from your speech to TTS generated voice) can sometimes take a minute or more right now.
23
-
> This isn't a bug – it's the current reality of running powerful AI models locally and for free:
24
-
> - Good STT (like Whisper) needs time to be accurate.
25
-
> - Even small LLM (via Ollama) needs some time to think up a good response.
26
-
> - Generating high-quality voice with TTS is also a complex and not fast task.
27
-
>
28
-
> The key thing is: this framework is built on the most optimal and user-friendly local solutions available for each stage (STT, LLM, TTS). You have the freedom to choose and download your own models (like Whisper `.bin` for STT and any model for Ollama), to use any custom voice you want, to make it for different languages, to customize your agents and to find the perfect balance between speed and quality for your setup.
29
-
>
30
-
> Also this project is actively maintained. With every update, I'm working on making it faster, more optimized, and easier to use!
31
-
32
-
---
33
-
34
21
UnityNeuroSpeech is an open-source framework for creating **fully voice-interactive AI agents** inside Unity.
35
22
It connects:
36
23
@@ -76,7 +63,7 @@ No subscriptions, no accounts, no OpenAI API keys.
Copy file name to clipboardExpand all lines: docs/index.md
+74-9Lines changed: 74 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,21 +1,88 @@
1
-
# 🚀 Quick Start
1
+
# 🚀 Getting Started
2
2
3
3
---
4
4
5
5
## 🛠 Installing Requirements
6
6
7
+
UnityNeuroSpeech requires several things to be installed before using it in Unity. Here what you need to install:
8
+
7
9
---
8
10
9
-
UnityNeuroSpeech requires several programs to be installed.
10
-
You can simply run `setup.bat` — it will download everything automatically.
11
-
Then just import the `.unitypackage` into your project.
11
+
### 1. Ollama
12
+
13
+
**Ollama** is a platform for running large language models (LLMs) locally. You can use models like DeepSeek, Gemma, Qwen, etc.
14
+
Note that small models might affect accuracy and context understanding, but big models response can take a long time.
15
+
16
+
Install it from the [official website](https://ollama.com/download).
17
+
18
+
Then you need to download LLM model with this command:
19
+
```console
20
+
ollama pull modelname
21
+
```
22
+
> For quick test I recommend to download **qwen2.5:3b** - it responds very fast.
12
23
13
24
---
14
25
15
-
## 🎙️ Voice Files
26
+
### 2. STT
27
+
28
+
**Whisper** — Speech-To-Text model that transcribes and translates audio with high accuracy.
29
+
30
+
You need to download Whisper model from [here](https://huggingface.co/ggerganov/whisper.cpp/tree/main).
31
+
> For quick test I recommend to download `ggml-base.bin`.
32
+
33
+
---
34
+
35
+
### 3. TTS
36
+
37
+
**UV** — a modern, ultra-fast Python package and environment manager. It replaces traditional tools like `pip`.
38
+
**Coqui TTS**(runs XTTS) uses **UV** to simplify installation and allows running the TTS command directly, without manual Python setup.
39
+
40
+
**Coqui XTTS** — a Text-To-Speech model that can generate speech in any custom voice you want: Chester Bennington, Vito Corleone (The Godfather), Cyn (Murder Drones) or any other.
Now you can finally import **UnityNeuroSpeech** to your Unity project.
75
+
76
+
Download `UnityNeuroSpeech.X.X.X.unitypackage` and `UNS_StreamingAssets.unitypackage` from the [official repository](https://github.com/HardCodeDev777/UnityNeuroSpeech/releases). Then you can import them in your project.
77
+
78
+
They are splitted to avoid importing almost 2GB XTTS model in your project if you only need code/fixes.
79
+
80
+
**Don't forget to put your downloaded Whisper model(`.bin`) in `Assets/StreamingAssets/UnityNeuroSpeech/Whisper/` - yes, it's important.**
81
+
82
+
---
83
+
84
+
## 🎙️ Voice Files
85
+
19
86
Don’t forget that you need voice files for TTS speech.
20
87
Make sure your files meet the following requirements:
21
88
@@ -38,7 +105,5 @@ All voices must be placed in:
38
105
39
106
## 🖼️ Microphone Sprites
40
107
41
-
---
42
-
43
-
You’ll need two sprites for the microphone state (enabled/disabled).
44
-
Yes — without them, it won’t work 🤠
108
+
You’ll need two sprites for the microphone state (enabled/disabled).
109
+
But for quick test you can use random default sprites from Unity.
Copy file name to clipboardExpand all lines: docs/unity/steps-to-make-it-work.md
+3-19Lines changed: 3 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,40 +9,24 @@ You can find tooltips for each field directly in the Unity Editor.
9
9
10
10
## Step 1. 🧪 Settings
11
11
12
-
---
13
-
14
12
Go to **UnityNeuroSpeech → Main → Create Settings** in the Unity toolbar.
15
13
Default settings are recommended.
16
14
17
15
---
18
16
19
17
## Step 2. 👀 UNS Manager
20
18
21
-
---
22
-
23
19
**UnityNeuroSpeech Manager** is a GameObject in your scene that controls all non-agent scripts.
24
20
Without it, no agent (talkable AI) will work.
25
21
26
-
---
27
22
28
23
Create a `Dropdown` in your scene.
29
-
Then go to **UnityNeuroSpeech → Main → Create UNS Manager**.
30
-
31
-
The important setting there is:
32
-
33
-
-**Whisper model path in StreamingAssets** — path to your downloaded Whisper model (`.bin`) inside the `StreamingAssets` folder (without the `Assets` directory).
Copy file name to clipboardExpand all lines: docs/unity/useful-tools.md
+8-10Lines changed: 8 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,37 +2,35 @@
2
2
3
3
---
4
4
5
-
UnityNeuroSpeech provides several Editor tools to make development more convenient.
5
+
UnityNeuroSpeech provides several Editor tools to improve development experience.
6
6
7
7
---
8
8
9
9
## 🗒️ Prompts Test
10
10
11
-
---
12
11
13
12
Let’s say you want to check how a selected LLM model responds to a specific prompt.
14
-
Normally, you would have to run the game, wait for Whisper to load, say something into the microphone (and risk transcription errors), then wait for the LLM and TTS to finish — quite the hardcore workflow, right?
13
+
Normally, you would have to run the game, wait for Whisper to load, say something into the microphone,
14
+
then wait for the LLM and TTS to finish — quite the hardcore, right?
15
15
16
16
This tool allows you to test prompts instantly.
17
17
You only wait for the **LLM** (as usual) to generate a response — and you can even see the **generation time in milliseconds**!
18
18
19
-
---
20
19
21
-
To access it, go to **UnityNeuroSpeech → Tools → Prompts Test**.
20
+
Go to **UnityNeuroSpeech → Tools → Prompts Test**.
22
21
23
22
---
24
23
25
24
## 🕵️♂️ Decode Encoded
26
25
27
-
---
28
26
29
27
If you use AES encryption, your `.json` dialog history files will be encrypted.
30
28
But what if you want to view their contents?
31
29
This tool lets you decrypt and read them easily.
32
30
33
-
---
34
31
35
-
To access it, go to **UnityNeuroSpeech → Tools → Decode Encoded**.
32
+
Go to **UnityNeuroSpeech → Tools → Decode Encoded**.
33
+
34
+
Important setting:
36
35
37
-
Note about the **Key to encrypt** field:
38
-
You must use the same key you specified in your `AgentBehaviour` script.
36
+
-**Key to encrypt**: You must use the same key you specified in your `AgentBehaviour` script.
0 commit comments