The model doesn't respond properly

[SudarshanShetty1796]

I'm using the following model:
https://www.kaggle.com/models/google/gemma-3/gemmaCpp/3.0-4b-it-sfp

When I ask a question, the model starts generating gibberish text continuously and doesn’t stop at all. I’ve tried rebuilding the project multiple times, but the issue persists right from the first prompt.

Commands used:

make -j4 gemma

./gemma \     
--tokenizer tokenizer.spm \
--weights 4b-it-sfp.sbs \
--weight_type sfp \
--model 4b-it

System Details:
macOS (Apple Silicon - M4 chip)
CMake version: 3.22.0
Compiler: Apple Clang (Xcode CLT)
gemma.cpp repo from GitHub, built via standard cmake and make

Could this issue be related to the model itself, or is it likely caused by something in the build or runtime setup?

[jan-wassenberg]

Thanks for reporting. Tests are currently passing for me with Gemma 3 4B on x86, but there might be a difference with M4.

Let's first rule out partial/corrupted weights downloads - sha1sum of the weights should be c8af5c86600f9f389ff157fab76b497e5badf3d9 and for the tokenizer 8a8e05deb279bd5bdca094fb7ececabe732a9534.

Next, are you building from the dev branch? Does an asan build (add -fsanitize=address to CXXFLAGS) report any issues?

[SudarshanShetty1796]

sha1sum is the same as yours
4b-it-sfp.sbs => c8af5c86600f9f389ff157fab76b497e5badf3d9
tokenizer.spm => 8a8e05deb279bd5bdca094fb7ececabe732a9534

are you building from the dev branch?

yes

Does an asan build (add -fsanitize=address to CXXFLAGS) report any issues?

I'm very new to this AI environment — in fact, I'm just trying to learn and don’t yet fully understand many of the keywords, as my expertise lies in a different software domain.

I didn’t completely understand what needed to be done, but after doing some research, I followed the steps below. I ran into an error, so I’d appreciate it if you could confirm whether these steps were what you intended — or if not, could you please explain the correct steps in more detail?

1. I cleared my build/ folder, keeping only the model weight and tokenizer files inside it.
2. I went back to the root of the gemma.cpp project and ran this command: cmake -B build -DCMAKE_CXX_FLAGS="-fsanitize=address"
3. After that completed, I ran: cmake --build build
4. Then I moved into the build/ directory and executed the model with: ./gemma \ --tokenizer tokenizer.spm \ --weights 4b-it-sfp.sbs \ --weight_type sfp \ --model 4b-it
5. The model loaded and gave a prompt (Hello), but it threw an error.
   I’ve copied the full logs and error into a text file — let me know if that would help you debug.

Thanks in advance!

Error.txt

[jan-wassenberg]

Thank you, that is helpful. The asserts point to the problem: Highway had a mismatch between the feature check and usage of bf16 dot products. I expect to land a fix for that tomorrow.

[SudarshanShetty1796]

Hello,
Is this fixed? Can I try?

[jan-wassenberg]

The Highway fix has landed and #630 should go in soon as well.
Please let us know if that fixes it for you.

[SudarshanShetty1796]

I tried it, and it's working fine now! Thank you.

I'm able to chat with the model.

However, when I run the ./gemma command, it prints the following error. Is this something to be concerned about, or can it be ignored?

[jan-wassenberg]

Glad to hear, thanks for following up :)
The warnings are to be expected, OS X does not provide the APIs that would be required for pinning. We can update the code to skip these warnings on OS X.