fix: add Metal JIT warmup, cache inspect.signature, and add metal health label#665
fix: add Metal JIT warmup, cache inspect.signature, and add metal health label#665neuron-tech-ai wants to merge 1 commit into
Conversation
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Three small performance fixes for Metal (Apple Silicon) TTS:
Metal JIT warmup at startup — Metal compiles shaders JIT on first use, causing a multi-second stall on the first generation. Warming up at startup moves this cost to load time where it's expected, not to the first generation where it looks like a hang.
Cache
inspect.signature()— this was being called on every single TTS generation to introspect the model'sforward()signature. Caching the result eliminates the repeated reflection overhead.Add
metalhealth label — the/healthendpoint reported backend type but didn't distinguishmetalfrom other backends, making it harder to confirm Metal acceleration is active.