Project Information

• Project name: nanoserve
• Github URL: https://github.com/pjdurden/nanoserve
• Homepage URL: https://pjdurden.github.io/nanoserve/
• Logo URL:

Category

• Inference
  • Inference Engine
  • Inference Platform
  • Middleware
  • LLM Router
  • AI Gateway
  • Output
  • Simulator
  • Benchmark
• Orchestration
  • Application Framework
  • Agent Framework
  • Evolutionary Framework
  • RAG
  • Workflow
• Runtime
  • AI Terminal
  • AI Agent
  • Chatbot
  • Code Agent
  • Database
  • Evolve Agent
  • Observation
  • Sandbox
  • Tool
• Training
  • Framework
  • FineTune
  • RLHF
  • Agentic RL
  • Benchmark
  • Workflow

nanoserve is a from-scratch minimal LLM inference engine (~2k readable lines): paged KV cache, continuous batching, and a hand-written Triton paged-attention kernel; serves Llama-3.2-1B with an OpenAI-compatible endpoint. MIT licensed, built in public.