fix: add Metal JIT warmup, cache inspect.signature, and add metal health label#665
fix: add Metal JIT warmup, cache inspect.signature, and add metal health label#665neuron-tech-ai wants to merge 1 commit into
Conversation
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…ine#665, jamiepine#664, jamiepine#653) - Add index=True to 9 FK/date/status columns across 6 tables - Add idempotent _migrate_add_indexes migration using CREATE INDEX IF NOT EXISTS - Import Index from sqlalchemy for explicit index declarations - Cache inspect.signature at model load time in _supports_ref_audio - Add Metal kernel warmup on MLX TTS model load (shifts JIT to load time) - Use cached flag in generate() instead of per-call inspect.signature - Initialize _supports_ref_audio=False in __init__ - Clean up health.py: contextlib.suppress, remove unused imports, add metal backend variant, move backend_type after hf_constants import - Resolve conflict with jamiepine#789 ThreadPoolExecutor (kept both changes) - Emit cargo:warning instead of panicking on actool failure - Allow builds without full Xcode (CLT-only dev builds) - Add user-friendly messages for sm_120+ (Blackwell) vs older GPUs - Point users to Settings → Server → GPU Acceleration for re-download - Add cuda_arch_warning field to ModelStatus - Check CUDA compat once in get_model_status and pass to all models - Add get_cuda_arch() to platform_detect.py
jamiepine#666, jamiepine#665, jamiepine#664, jamiepine#653) with index migration test coverage
1 participant
Three small performance fixes for Metal (Apple Silicon) TTS:
Metal JIT warmup at startup — Metal compiles shaders JIT on first use, causing a multi-second stall on the first generation. Warming up at startup moves this cost to load time where it's expected, not to the first generation where it looks like a hang.
Cache
inspect.signature()— this was being called on every single TTS generation to introspect the model'sforward()signature. Caching the result eliminates the repeated reflection overhead.Add
metalhealth label — the/healthendpoint reported backend type but didn't distinguishmetalfrom other backends, making it harder to confirm Metal acceleration is active.