Claude Code ran into this problem and created the following after debugging - Jeff

Spawned session subprocess can't recompile Kaimon when target project's manifest invalidates the precompile cache

Symptom

Calling start_session(project_path=<P>) against a project P whose Manifest.toml has drifted enough to invalidate Kaimon's precompile cache returns:

Error: Session failed to start — Process died at <ts>
Check log: /home/<user>/.cache/kaimon/sessions/<P>.log

The reported log path does not exist (see #43 — log file is never written), and the server log only records:

Session '<P>' subprocess spawning (PID=<pid>)
start_session ✓ (4.5s)
... [silence] ...
Session 'P' subprocess exited unexpectedly

…with no underlying stack trace.

What's actually happening

Reproducing the spawn manually (the boot script from _build_session_script plus the env overlay from spawn_session!):

JULIA_LOAD_PATH='@:@v#.#:@stdlib' JULIA_PROJECT='' \
julia -i -t auto --startup-file=no \
  --project=/path/to/target \
  -e 'try; using Revise; catch; end;
      insert!(LOAD_PATH, 1, "<kaimon_pkgdir>");
      using Kaimon;
      import Pkg; Pkg.instantiate(; io=devnull);
      @async Kaimon.Gate.serve(force=true, allow_mirror=true, allow_restart=true, spawned_by="agent")'

surfaces:

[ Info: Precompiling Kaimon [d3856c55-...] (cache misses: wrong dep version
loaded (1), wrong source (2), incompatible header (5))
ERROR: LoadError: ArgumentError: Package JSON [682c06a0-...] is required
but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Root cause

spawn_session! sets the subprocess JULIA_LOAD_PATH to @:@v#.#:@stdlib — the shell default for julia --project=<target>. When Kaimon's precompile cache is valid for that target, this is fine: the cache loads as a binary blob without any dep resolution.

When the cache is invalid (changes in the target's Manifest can invalidate it via wrong dep version loaded / wrong source / incompatible header), Julia rebuilds Kaimon from source in a worker. That rebuild needs to resolve Kaimon's using JSON (and all other direct deps) via the worker's LOAD_PATH — none of which has Kaimon's own deps:

• @ = target's Project.toml (no JSON unless the user happens to depend on it)
• @v#.# = global v1.12 env (rarely populated)
• @stdlib = stdlib only

The insert!(LOAD_PATH, 1, pkgdir(Kaimon)) line in the boot script doesn't help: a bare pkgdir is not an env, has no manifest, and Julia uses it only to find Kaimon's source — falling back to the active project for dep resolution.

Reproduction recipe

1. In a non-Kaimon project, do anything that perturbs the dep graph relative to Kaimon's cache (Pkg.update, Pkg.rm of a transitive dep, switching Julia patch versions, etc.). In my case it was Pkg.rm on heavy deps from a project that had grown stale, which caused JLLWrappers / Parsers to resolve to versions Kaimon wasn't built against.
2. Call start_session(project_path=<that project>).
3. Observe "Process died" with no log.