[Showcase] Eco-Metal — 63 Modular Plugins for Advanced LLM Inference natively on MSL

[helgklaizar]

Title: Show and Tell: Eco-Metal — 63 Modular Plugins for Advanced LLM Inference natively on MSL

Body:
Hey MLX team! First of all, huge thanks for the amazing work on MLX. Apple Silicon is absolutely game-changing for local AI.

I've spent the past few months heavily utilizing your framework to build Eco-Metal (https://github.com/helgklaizar/Eco-Metal), a production-ready ecosystem of 63 modular AI components fully optimized for Mac.

Our main focus was to eliminate slow Python overhead and CUDA-wrappers. We've successfully ported and hardened custom Metal Shading Language (MSL) kernels and native mx.fast paths for several SOTA algorithms:

• Paged Attention & KV Traversal: True zero-copy caching architectures taking advantage of the 800 GB/s Unified Memory.
• H2O Heavy-Hitters KV Predictors: Dynamically drops non-essential KV tokens for infinite contexts.
• Extreme Quantization: (1-2 bit multi-codebook lookups natively on Metal)
• Rocket Pruning & Tri-Attention: Custom Metal branches handling sparse retrieval.

We ensure 100% test coverage and native JIT execution via mx.fast.metal_kernel and mx.fast.scaled_dot_product_attention.

I would be incredibly honored if someone from the core AMLR team (or anyone else!) could check out the project. I'm completely open to submitting any of our high-performance Metal MSL kernels directly upstream to MLX if you feel they fit the primary framework requirements!

Ecosystem Repository: https://github.com/helgklaizar/Eco-Metal

Cheers, and keep up the brilliant work!