Unauthenticated /dev/unload endpoint can unload the active model

[Galaxync]

Unauthenticated /dev/unload endpoint can unload the active model

Describe the bug

POST /dev/unload is mounted by default and does not require authentication. Any remote caller who can reach the API can unload the active model from memory. The model reloads automatically on the next inference request, so repeated calls can continuously degrade service availability and force expensive cold starts.

Confirmed affected commit: 4129920330e4a392bd594e7ba6310b1db85334c8

Relevant code:

• api/src/main.py unconditionally includes dev_router.
• api/src/routers/development.py exposes POST /dev/unload without an auth dependency.
• api/src/inference/model_manager.py unloads the backend and clears CUDA cache.

Screenshots or console output

curl -X POST 'http://127.0.0.1:8880/dev/unload'

Vulnerable response:

{"status":"unloaded"}

After this request, the next TTS request needs to reload the model. Repeating the unauthenticated request can keep forcing unload/reload cycles.

Branch / Deployment used

Source review against the master branch at commit 4129920330e4a392bd594e7ba6310b1db85334c8.

The vulnerable route is included by default in the FastAPI application:

app.include_router(dev_router)

Operating System

Not OS-specific. The impact is stronger on GPU deployments because unloading and reloading model weights can cause latency spikes and resource churn.

Additional context

Expected behavior:

• Development endpoints should not be mounted by default in production deployments; or
• /dev/unload should require an administrator credential; and
• repeated unload requests should be rate limited.

Suggested fix:

• Gate dev_router behind an explicit development/admin setting.
• Require authentication/authorization for /dev/unload.
• Consider binding development endpoints to localhost only.
• Add rate limiting for model lifecycle operations.

Source references:

• Kokoro-FastAPI/api/src/main.py

  Lines 132 to 134 in 4129920

  	allow_headers=["*"],
  	)

• Kokoro-FastAPI/api/src/routers/development.py

  Lines 417 to 430 in 4129920

  	@router.post("/dev/unload")
  	async def unload_model(
  	tts_service: TTSService = Depends(get_tts_service),
  	):
  	"""Release the model from GPU VRAM without stopping the container.
  	The model reloads automatically on the next inference request.
  	Useful for homelab deployments where GPU memory is shared across services.
  	"""
  	try:
  	if tts_service.model_manager is None:
  	raise HTTPException(status_code=503, detail={"error": "Model manager not initialized"})
  	await tts_service.model_manager.unload()
  	return JSONResponse({"status": "unloaded"})

• Kokoro-FastAPI/api/src/inference/model_manager.py

  Lines 168 to 176 in 4129920

  	async def unload(self) -> None:
  	"""Release model from GPU memory. Reloads automatically on next request."""
  	async with self._lock:
  	if self._backend is not None:
  	self._backend.unload()
  	self._backend = None
  	if torch.cuda.is_available():
  	torch.cuda.empty_cache()
  	logger.info("Model unloaded from GPU memory")

[remsky]

Makes sense. We'd probably want to move the phenome and other similar endpoints off the dev router if we were flagging it off entirely though. Should be able to at least add the unload flag gate in the interim