Feature request
Thank you for adding support for embeddinggemma-300m. I am using it on fp32 and it works great. I wonder when we might have bfloat16 support? It is especially important for embeddinggemma-300m where fp16 does not work. I was surprised not to find an existing thread to follow along so feel free to mark dupe or close + redirect.
Motivation
TEI is convenient for serving models but the lack of bfloat16 support is a real problem, especially for models like embeddinggemma-300m which don't support float16. It means we leave a great deal of performance on the table.
Your contribution
I'm not sure how I would contribute but I'm open to suggestions
Feature request
Thank you for adding support for embeddinggemma-300m. I am using it on fp32 and it works great. I wonder when we might have bfloat16 support? It is especially important for embeddinggemma-300m where fp16 does not work. I was surprised not to find an existing thread to follow along so feel free to mark dupe or close + redirect.
Motivation
TEI is convenient for serving models but the lack of bfloat16 support is a real problem, especially for models like embeddinggemma-300m which don't support float16. It means we leave a great deal of performance on the table.
Your contribution
I'm not sure how I would contribute but I'm open to suggestions