Skip to content

Commit d92b2d7

Browse files
u9gtoubatbriancursoragent
authored
simulate: Node agent support, --view mode, and TUI/start UX fixes (#878)
* feat(session): support Node.js agents in the local session daemon The `lk agent session` daemon could previously only spawn Python agents; Node/JS projects were rejected at detection time. This wires up the Node path so a JS/TS agent can be driven over the same text-mode console protocol as Python. - agent_utils.go: stop gating `detectProject` on Python-only. Accept Node projects too (DetectProjectRoot already recognizes them via package.json) and update the error to reflect both supported runtimes. - simulate_subprocess.go: extract interpreter/argv resolution into `buildAgentCommand`, branching on project type. Python keeps `<python> <entry> <args>` (uv prefixes `run python`); Node runs `node [--experimental-strip-types] <entry> <args>`. Add `findNodeBinary` (resolves `node` from PATH) and `isTypeScriptEntry` so a `.ts`/`.mts`/ `.cts` entrypoint runs directly via the type-stripping loader with no build step. - session_daemon.go: drop the stale TODO; the daemon now spawns either runtime via `buildConsoleArgs` (`console --connect-addr <addr>`). Verified end-to-end: `lk agent session start examples/src/console_text_agent.ts` spawns the JS agent, connects to the Go TCP console server, completes the text-mode handshake, and round-trips multi-turn `say` requests including a function-tool call. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(agent): Node.js agent support for lk agent dev/start/console (#868) * feat(agent): support Node.js agents in lk agent dev/start/console Builds on the Node project detection and spawn helpers from the session daemon work (#861) to wire the remaining agent run commands for Node: - `lk agent dev`/`start`: spawn the agents-js `dev` subcommand (Python keeps `start --dev`), normalize --log-level casing per runtime (agents-js only accepts lowercase, Python expects uppercase), and make the reload-server handshake (--reload-addr) Python-only — Node reloads are a plain kill+respawn since agents-js has no job-restore protocol. - Entrypoint discovery: probe per-runtime candidate lists (agent.ts, agent.js, src/agent.{ts,js} for Node; agent.py, src/agent.py for Python) instead of a single root default, and make the not-found message runtime-aware. - Probe `node --version` before running a TypeScript entrypoint and fail with a clear message on Node < 22.6 (no --experimental-strip-types). - resolveCredentials: ignore the global --url flag's default value so the project config (--project) can supply the URL; previously the spawned agent was always pointed at http://localhost:7880 unless --url or LIVEKIT_URL was set explicitly. `lk agent console` needed no changes beyond #861 — the spawn path was already runtime-agnostic. Tested end to end against agents-js' console CLI branch (#1706): console text mode, audio mode (mic/speaker + playback-finished handshake), ctrl-T mode toggle, --record output, dev worker registration, file-watch reload, and clean shutdown. * feat(agent): load discovered .env files into Node agents via --env-file Node agents spawned by lk agent dev/start/console inherited only the shell environment, so credentials in a project .env file were never seen (Python agents conventionally load_dotenv() themselves). Discover a known env file in the project dir and pass it to Node's built-in dotenv with --env-file. The flag requires Node >= 20.6, so the type-stripping version check is refactored into a shared nodeVersion probe and the flag is skipped on older Nodes. Shell-exported variables still take precedence over file values. * feat(console): fail fast when the agent job crashes before connecting A pre-connect job crash (e.g. missing API key in the entrypoint) leaves the worker process alive, so lk agent console sat on the spinner for the full 60s connect timeout before surfacing the traceback. Scan agent output for crash markers (Python's "job crashed" shutdown reason, agents-js's FATAL "console mode failed:") via a new FailSignals option on AgentStartConfig, and abort the connect wait immediately with the recent logs. The session daemon's wait gets the same treatment, reporting the log path through the ready pipe. Verified against agents @ a61578853: a missing .env now fails in ~4s with the full traceback instead of timing out at 60s. * Revert "feat(agent): load discovered .env files into Node agents via --env-file" This reverts commit 2430a24. Per PR review, agent code is expected to load its own env; auto-loading a discovered .env file second-guesses that. Arbitrary arg forwarding (e.g. a -- separator) is the preferred ease-of-use mechanism instead. * feat(agent): forward runtime args after -- to node/python Per PR review, replace the reverted .env auto-loading with explicit arg forwarding: everything after a -- separator is passed to the runtime interpreter ahead of the entrypoint in lk agent dev/start/console and the session daemon, so `lk agent console agent.ts -- --env-file=.env` runs `node --env-file=.env agent.ts console ...`. urfave/cli strips the -- and folds the trailing args into the positionals, so splitForwardedArgs recovers the split from os.Args; detectProject uses it so a forwarded flag with no entrypoint argument isn't mistaken for one. The detached session daemon receives the args JSON-encoded via LK_SESSION_FWD. * test(agent): trim agent run tests to the fail-signal case Remove the log-level, entrypoint-resolution, arg-splitting, and command- building unit tests along with the session forwarded-args round-trip test, keeping TestAgentProcessFailSignal. * simulate: Node agent support, --view mode, and TUI/start UX fixes Ports our simulate work onto the Node-support branch: - Node agent start: normalizeLogLevel, --log-format only for Python, main.ts/main.js entrypoints, and refuse entrypoint discovery when package.json defines a build task; drop the Python-only guard. - `lk agent simulate --view <runID>`: open a pre-existing simulation and print the reopen command on close. - TUI: ←/→ arrow navigation in/out of simulation detail, richer agent-exit errors (start command + recent output), and quit when the agent fails to register. * fix(agent): resolve Node SDK version via the runtime, gate Node separately The local-run version gate ran main's file-based CheckSDKVersion for Node too, which (a) couldn't find @livekit/agents in monorepo/workspace layouts where the dep is a `workspace:*` symlink rather than a versioned entry, and (b) gated on the Python thin-CLI baseline (1.6.0), the wrong floor for the Node SDK. Resolve the installed @livekit/agents via Node's own module resolution (embedded node_resolve_version.js), starting from the entrypoint's directory so pnpm/workspace symlinks and hoisting resolve exactly as they will at runtime, and gate Node on its own nodeAgentMinVersion. Python keeps the existing CheckSDKVersion path. Export IsVersionSatisfied for the reused comparison. --------- Co-authored-by: Toubat <brian.yin@livekit.io> Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent e88443e commit d92b2d7

14 files changed

Lines changed: 579 additions & 144 deletions

cmd/lk/agent_run.go

Lines changed: 27 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ import (
2222
"syscall"
2323

2424
"github.com/urfave/cli/v3"
25+
26+
"github.com/livekit/livekit-cli/v2/pkg/agentfs"
2527
)
2628

2729
func init() {
@@ -38,15 +40,15 @@ var agentRunFlags = []cli.Flag{
3840
var startCommand = &cli.Command{
3941
Name: "start",
4042
Usage: "Run an agent in production mode",
41-
ArgsUsage: "[entrypoint]",
43+
ArgsUsage: "[entrypoint] [-- node/python-args...]",
4244
Flags: agentRunFlags,
4345
Action: runAgentStart,
4446
}
4547

4648
var devCommand = &cli.Command{
4749
Name: "dev",
4850
Usage: "Run an agent in development mode with auto-reload",
49-
ArgsUsage: "[entrypoint]",
51+
ArgsUsage: "[entrypoint] [-- node/python-args...]",
5052
Flags: append(agentRunFlags, &cli.BoolFlag{
5153
Name: "no-reload",
5254
Usage: "Disable auto-reload on file changes",
@@ -57,6 +59,11 @@ var devCommand = &cli.Command{
5759
// resolveCredentials returns CLI args (--url, --api-key, --api-secret) for the agent subprocess.
5860
func resolveCredentials(cmd *cli.Command, loadOpts ...loadOption) ([]string, error) {
5961
url := cmd.String("url")
62+
if !cmd.IsSet("url") {
63+
// Ignore the global flag's default (http://localhost:7880) so the
64+
// project config can supply the URL.
65+
url = ""
66+
}
6067
apiKey := cmd.String("api-key")
6168
apiSecret := cmd.String("api-secret")
6269

@@ -93,10 +100,10 @@ func resolveCredentials(cmd *cli.Command, loadOpts ...loadOption) ([]string, err
93100
return args, nil
94101
}
95102

96-
func buildCLIArgs(subcmd string, cmd *cli.Command, loadOpts ...loadOption) ([]string, error) {
103+
func buildCLIArgs(projectType agentfs.ProjectType, subcmd string, cmd *cli.Command, loadOpts ...loadOption) ([]string, error) {
97104
args := []string{subcmd}
98105
if logLevel := cmd.String("log-level"); logLevel != "" {
99-
args = append(args, "--log-level", logLevel)
106+
args = append(args, "--log-level", normalizeLogLevel(projectType, logLevel))
100107
}
101108
creds, err := resolveCredentials(cmd, loadOpts...)
102109
if err != nil {
@@ -113,7 +120,7 @@ func runAgentStart(ctx context.Context, cmd *cli.Command) error {
113120
}
114121
out.Statusf("Detected %s agent (%s in %s)", projectType.Lang(), entrypoint, projectDir)
115122

116-
cliArgs, err := buildCLIArgs("start", cmd)
123+
cliArgs, err := buildCLIArgs(projectType, "start", cmd)
117124
if err != nil {
118125
return err
119126
}
@@ -122,6 +129,7 @@ func runAgentStart(ctx context.Context, cmd *cli.Command) error {
122129
Dir: projectDir,
123130
Entrypoint: entrypoint,
124131
ProjectType: projectType,
132+
RuntimeArgs: forwardedArgs(cmd),
125133
CLIArgs: cliArgs,
126134
ForwardOutput: os.Stdout,
127135
})
@@ -134,7 +142,7 @@ func runAgentStart(ctx context.Context, cmd *cli.Command) error {
134142
sigCh := make(chan os.Signal, 1)
135143
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
136144

137-
// Forward every signal to the agent — Python decides
145+
// Forward every signal to the agent — the agent decides:
138146
// first = graceful shutdown, second = force exit.
139147
go func() {
140148
for range sigCh {
@@ -154,19 +162,28 @@ func runAgentDev(ctx context.Context, cmd *cli.Command) error {
154162
return err
155163
}
156164

157-
cliArgs, err := buildCLIArgs("start", cmd)
165+
// Python has no dedicated dev subcommand: dev mode is `start --dev`.
166+
// agents-js has a `dev` subcommand that already defaults to debug logs.
167+
subcmd := "dev"
168+
if projectType.IsPython() {
169+
subcmd = "start"
170+
}
171+
cliArgs, err := buildCLIArgs(projectType, subcmd, cmd)
158172
if err != nil {
159173
return err
160174
}
161-
cliArgs = append(cliArgs, "--dev")
162-
if cmd.String("log-level") == "" {
163-
cliArgs = append(cliArgs, "--log-level", "DEBUG")
175+
if projectType.IsPython() {
176+
cliArgs = append(cliArgs, "--dev")
177+
if cmd.String("log-level") == "" {
178+
cliArgs = append(cliArgs, "--log-level", "DEBUG")
179+
}
164180
}
165181

166182
cfg := AgentStartConfig{
167183
Dir: projectDir,
168184
Entrypoint: entrypoint,
169185
ProjectType: projectType,
186+
RuntimeArgs: forwardedArgs(cmd),
170187
CLIArgs: cliArgs,
171188
ForwardOutput: os.Stdout,
172189
}

cmd/lk/agent_run_test.go

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
// Copyright 2026 LiveKit, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
package main
16+
17+
import (
18+
"os"
19+
"os/exec"
20+
"path/filepath"
21+
"testing"
22+
"time"
23+
24+
"github.com/stretchr/testify/require"
25+
26+
"github.com/livekit/livekit-cli/v2/pkg/agentfs"
27+
)
28+
29+
func TestAgentProcessFailSignal(t *testing.T) {
30+
if _, err := exec.LookPath("node"); err != nil {
31+
t.Skip("node not on PATH")
32+
}
33+
34+
// An agent whose job crashes logs a marker but keeps the process alive;
35+
// Failed() must fire without waiting for exit.
36+
dir := t.TempDir()
37+
script := `console.log('shutting down job task {"reason": "job crashed"}'); setTimeout(() => {}, 30000);`
38+
require.NoError(t, os.WriteFile(filepath.Join(dir, "agent.js"), []byte(script), 0o644))
39+
// startAgent gates on the SDK version, which Node resolves from node_modules,
40+
// so install a satisfying @livekit/agents stub.
41+
agentsDir := filepath.Join(dir, "node_modules", "@livekit", "agents")
42+
require.NoError(t, os.MkdirAll(agentsDir, 0o755))
43+
require.NoError(t, os.WriteFile(filepath.Join(agentsDir, "package.json"),
44+
[]byte(`{"name": "@livekit/agents", "version": "1.6.0"}`), 0o644))
45+
46+
ap, err := startAgent(AgentStartConfig{
47+
Dir: dir,
48+
Entrypoint: "agent.js",
49+
ProjectType: agentfs.ProjectTypeNode,
50+
FailSignals: consoleCrashSignals,
51+
})
52+
require.NoError(t, err)
53+
defer ap.Kill()
54+
55+
select {
56+
case <-ap.Failed():
57+
case <-time.After(10 * time.Second):
58+
t.Fatal("Failed() did not fire on crash marker")
59+
}
60+
}

cmd/lk/agent_utils.go

Lines changed: 54 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,14 +18,46 @@ import (
1818
"fmt"
1919
"os"
2020
"path/filepath"
21+
"strings"
2122

2223
"github.com/urfave/cli/v3"
2324

2425
"github.com/livekit/livekit-cli/v2/pkg/agentfs"
2526
)
2627

28+
// splitForwardedArgs recovers the argument split around a "--" separator.
29+
// urfave/cli strips the separator and appends everything after it to the
30+
// parsed positional args, so the forwarded tail is recovered from the raw
31+
// process argv and trimmed off the positionals.
32+
func splitForwardedArgs(rawArgs, positional []string) (entryArgs, forwarded []string) {
33+
for i, a := range rawArgs {
34+
if a != "--" {
35+
continue
36+
}
37+
forwarded = rawArgs[i+1:]
38+
if len(forwarded) > len(positional) {
39+
// The "--" was consumed as a flag's value, not a separator.
40+
return positional, nil
41+
}
42+
return positional[:len(positional)-len(forwarded)], forwarded
43+
}
44+
return positional, nil
45+
}
46+
47+
// forwardedArgs returns the args the user passed after a "--" separator,
48+
// forwarded to the runtime interpreter (node/python) ahead of the
49+
// entrypoint, e.g. `lk agent console agent.ts -- --env-file=.env`.
50+
func forwardedArgs(cmd *cli.Command) []string {
51+
_, fwd := splitForwardedArgs(os.Args, cmd.Args().Slice())
52+
return fwd
53+
}
54+
2755
func detectProject(cmd *cli.Command) (string, agentfs.ProjectType, string, error) {
28-
explicit := cmd.Args().First()
56+
entryArgs, _ := splitForwardedArgs(os.Args, cmd.Args().Slice())
57+
var explicit string
58+
if len(entryArgs) > 0 {
59+
explicit = entryArgs[0]
60+
}
2961

3062
detectFrom := "."
3163
if explicit != "" {
@@ -44,11 +76,8 @@ func detectProject(cmd *cli.Command) (string, agentfs.ProjectType, string, error
4476
return "", "", "", noAgentError()
4577
}
4678

47-
// TODO(node): support JS/Node agents here. DetectProjectRoot already
48-
// recognizes Node projects; once the session daemon can spawn a Node
49-
// agent in console mode, drop this gate and branch on projectType.
50-
if !projectType.IsPython() {
51-
return "", "", "", fmt.Errorf("currently only supports Python agents (detected: %s)", projectType)
79+
if !projectType.IsPython() && !projectType.IsNode() {
80+
return "", "", "", fmt.Errorf("only Python and Node agents are supported (detected: %s)", projectType)
5281
}
5382

5483
if explicit != "" {
@@ -67,6 +96,16 @@ func detectProject(cmd *cli.Command) (string, agentfs.ProjectType, string, error
6796
return projectDir, projectType, entrypoint, nil
6897
}
6998

99+
// consoleCrashSignals are output markers meaning the console job died even
100+
// though the worker process may stay alive: the Python SDK keeps the worker
101+
// running after the job task crashes (logging `"reason": "job crashed"`), and
102+
// agents-js logs FATAL `console mode failed:` before exiting. Without these,
103+
// a pre-connect crash leaves the user waiting out the full connect timeout.
104+
var consoleCrashSignals = []string{
105+
`"job crashed"`,
106+
"console mode failed:",
107+
}
108+
70109
// buildConsoleArgs builds the agent subprocess argv for console mode, shared by
71110
// `lk agent console` and the `lk agent daemon` daemon.
72111
func buildConsoleArgs(addr string, record bool) []string {
@@ -76,3 +115,12 @@ func buildConsoleArgs(addr string, record bool) []string {
76115
}
77116
return args
78117
}
118+
119+
// normalizeLogLevel adapts the log level to the agent runtime's convention:
120+
// agents-js accepts only lowercase levels, Python expects uppercase.
121+
func normalizeLogLevel(projectType agentfs.ProjectType, level string) string {
122+
if projectType.IsNode() {
123+
return strings.ToLower(level)
124+
}
125+
return strings.ToUpper(level)
126+
}

cmd/lk/agent_watcher.go

Lines changed: 35 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -80,15 +80,19 @@ func newAgentWatcher(config AgentStartConfig) (*agentWatcher, error) {
8080
return nil, fmt.Errorf("failed to setup file watcher: %w", err)
8181
}
8282

83-
rs, err := newReloadServer()
84-
if err != nil {
85-
w.Close()
86-
return nil, err
83+
// The reload protocol (capture running jobs from the old process, restore
84+
// them in the new one) is Python-only; Node reloads are a plain kill+respawn.
85+
var rs *reloadServer
86+
if config.ProjectType.IsPython() {
87+
rs, err = newReloadServer()
88+
if err != nil {
89+
w.Close()
90+
return nil, err
91+
}
92+
// Append --reload-addr to CLI args so the Python process connects back
93+
config.CLIArgs = append(config.CLIArgs, "--reload-addr", rs.addr())
8794
}
8895

89-
// Append --dev and --reload-addr to CLI args so the Python process connects back
90-
config.CLIArgs = append(config.CLIArgs, "--dev", "--reload-addr", rs.addr())
91-
9296
return &agentWatcher{
9397
config: config,
9498
exts: watchExtensions(config.ProjectType),
@@ -107,15 +111,17 @@ func (aw *agentWatcher) start() error {
107111
aw.agent = agent
108112

109113
// Accept connection from new Python process in background
110-
go func() {
111-
conn, err := aw.reloadSrv.listener.Accept()
112-
if err != nil {
113-
return
114-
}
115-
aw.conn = conn
116-
// Serve the initial restore request (will be empty on first start)
117-
go aw.reloadSrv.serveNewProcess(conn)
118-
}()
114+
if aw.reloadSrv != nil {
115+
go func() {
116+
conn, err := aw.reloadSrv.listener.Accept()
117+
if err != nil {
118+
return
119+
}
120+
aw.conn = conn
121+
// Serve the initial restore request (will be empty on first start)
122+
go aw.reloadSrv.serveNewProcess(conn)
123+
}()
124+
}
119125

120126
return nil
121127
}
@@ -143,14 +149,16 @@ func (aw *agentWatcher) restart() error {
143149
aw.agent = agent
144150

145151
// 4. Accept new connection and serve restored jobs
146-
go func() {
147-
conn, err := aw.reloadSrv.listener.Accept()
148-
if err != nil {
149-
return
150-
}
151-
aw.conn = conn
152-
go aw.reloadSrv.serveNewProcess(conn)
153-
}()
152+
if aw.reloadSrv != nil {
153+
go func() {
154+
conn, err := aw.reloadSrv.listener.Accept()
155+
if err != nil {
156+
return
157+
}
158+
aw.conn = conn
159+
go aw.reloadSrv.serveNewProcess(conn)
160+
}()
161+
}
154162

155163
return nil
156164
}
@@ -176,7 +184,9 @@ func (aw *agentWatcher) Run(done <-chan struct{}) error {
176184
if aw.conn != nil {
177185
aw.conn.Close()
178186
}
179-
aw.reloadSrv.close()
187+
if aw.reloadSrv != nil {
188+
aw.reloadSrv.close()
189+
}
180190
aw.watcher.Close()
181191
}()
182192

cmd/lk/console.go

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ func init() {
4141
var consoleCommand = &cli.Command{
4242
Name: "console",
4343
Usage: "Voice chat with an agent via mic/speakers",
44-
ArgsUsage: "[entrypoint]",
44+
ArgsUsage: "[entrypoint] [-- node/python-args...]",
4545
Category: "Core",
4646
Flags: []cli.Flag{
4747
&cli.IntFlag{
@@ -141,7 +141,9 @@ func runConsole(ctx context.Context, cmd *cli.Command) error {
141141
Dir: projectDir,
142142
Entrypoint: entrypoint,
143143
ProjectType: projectType,
144+
RuntimeArgs: forwardedArgs(cmd),
144145
CLIArgs: buildConsoleArgs(actualAddr, cmd.Bool("record")),
146+
FailSignals: consoleCrashSignals,
145147
})
146148
if err != nil {
147149
stopSpinner()
@@ -171,9 +173,17 @@ func runConsole(ctx context.Context, cmd *cli.Command) error {
171173
return fmt.Errorf("agent connection: %w", res.err)
172174
}
173175
conn = res.conn
174-
case <-agentProc.Done():
176+
case err := <-agentProc.Done():
175177
stopSpinner()
178+
if err != nil {
179+
return fmt.Errorf("the agent exited before connecting: %w\n\n%s", err, agentExitDetail(agentProc))
180+
}
176181
return fmt.Errorf("the agent exited before connecting.\n\n%s", agentExitDetail(agentProc))
182+
case <-agentProc.Failed():
183+
stopSpinner()
184+
// The crash marker arrives mid-traceback; give trailing output a moment.
185+
time.Sleep(500 * time.Millisecond)
186+
return fmt.Errorf("the agent job crashed before connecting.\n\n%s", agentExitDetail(agentProc))
177187
case <-time.After(60 * time.Second):
178188
stopSpinner()
179189
return fmt.Errorf("timed out waiting for the agent to connect.\n\n%s", agentExitDetail(agentProc))

0 commit comments

Comments
 (0)