feat(eval): honor UIPATH_AGENT_ID, UIPATH_CLOUD_USER_ID, UIPATH_PROJECT_FILES_SOURCE env vars [AE-1396]

[Chibionos]

What

The SDK now distinguishes the logical agent the user authored from the file-source project the worker fetches files from. Previously these were collapsed onto UIPATH_PROJECT_ID, which works for cloud projects but is wrong for local-workspace eval runs (where prepareForCustomDebug creates a separate cloud debug project).

Three behaviors change:

1. Eval API callbacks route by AgentId, not file-source project. The URL /api/execution/agents/{id}/evalRun and the agentId payload field now use the logical agent ID.
2. Telemetry tags AgentId with the logical agent, not the file-source project — dashboards group by what the user authored.
3. CloudUserId prefers the explicit env var over the JWT sub claim, so service-account-spawned workers tag the real triggering user.

File fetching (UiPathConfig.project_id, used by StudioClient / ResourceOverwritesContext in cli_eval.py and cli_debug.py) stays on the file-source project — that's where the worker's package files actually live.

Env var contract

Env var	Meaning	Notes
UIPATH_PROJECT_ID	Cloud project the worker fetches package files from. Equals the agent ID for cloud projects; for local-workspace runs it is the debug project's GUID.	UiPathConfig.project_id; StudioClient / ResourceOverwritesContext use it for file fetching.
UIPATH_AGENT_ID	Logical agent the user authored.	UiPathConfig.agent_id (falls back to project_id). Used for eval API URL routing in StudioWebProgressReporter and telemetry AgentId tagging.
UIPATH_CLOUD_USER_ID	Real user who triggered the run.	Eval + CLI telemetry prefer this over the JWT sub claim.
UIPATH_PROJECT_FILES_SOURCE	Local or Cloud discriminator the UI filters by.	Sent on every eval-run / eval-set-run POST/PUT payload and the resume GET as a query parameter, so rows match the UI's ?projectFilesSource=1 filter.

How env vars flow

sequenceDiagram
    autonumber
    actor U as User
    participant FE as Studio Web (FE)
    participant API as Agents Backend
    participant ORC as Orchestrator
    participant HDN as hdens worker (Python runtime)
    participant SDK as uipath-python SDK

    U->>FE: Start eval run
    FE->>API: POST /startMultipleEvaluators<br/>?debugProjectId={debugId}
    Note over API: Builds env vars:<br/>UIPATH_PROJECT_ID = debugProjectId ?? agentId<br/>UIPATH_AGENT_ID = real agentId<br/>UIPATH_CLOUD_USER_ID = session.UserId<br/>UIPATH_PROJECT_FILES_SOURCE = Local/Cloud
    API->>ORC: ServerlessStartJobDto envVars newline-joined
    ORC->>HDN: Dispatch job
    Note over HDN: PythonCodedJobExecutor<br/>EnsureEnvironmentVariablesParsed()<br/>(no allowlist - every UIPATH_* flows through)
    HDN->>HDN: os.environ.update(env_vars)<br/>(via cli_server.py for pooled mode)
    HDN->>SDK: Invoke uipath eval

    rect rgb(240, 248, 255)
        Note over SDK: API callbacks use UIPATH_AGENT_ID
        SDK->>API: POST /api/execution/agents/{UIPATH_AGENT_ID}/evalSetRun<br/>{ projectFilesSource: 1, ... }
        SDK->>API: POST /api/execution/agents/{UIPATH_AGENT_ID}/evalRun<br/>{ projectFilesSource: 1, ... }
        SDK->>API: GET /api/execution/agents/.../evalRuns<br/>?projectFilesSource=1
    end

    rect rgb(245, 245, 245)
        Note over SDK: File fetching uses UIPATH_PROJECT_ID (file source)
        SDK->>API: StudioClient(UIPATH_PROJECT_ID)<br/>(ResourceOverwritesContext)
    end

    rect rgb(248, 240, 255)
        Note over SDK: Telemetry tagged with both
        SDK->>SDK: properties[AgentId] = UIPATH_AGENT_ID<br/>properties[ProjectId] = UIPATH_PROJECT_ID<br/>properties[CloudUserId] = UIPATH_CLOUD_USER_ID
    end

Loading

For cloud runs, UIPATH_AGENT_ID and UIPATH_PROJECT_ID are equal — the diagram collapses to today's behaviour. For local-workspace runs they differ and the SDK keeps them straight.

Why this matters

• Codex adversarial review flagged "the route now lies" — eval-run callbacks were hitting /agents/{debug-project}/evalRun even though the row should be attributed to the logical agent. Authz, telemetry dimensions, cache keys, and any future code consuming the route param would all see the debug project as if it were the agent.
• Pooled cli_server.py cross-contamination is mitigated by the existing os.environ.clear() + update(baseline) + update(env_vars) pattern — every job request gets a fresh env. The new variables follow the same pipeline.
• Backwards compatibility is preserved at every layer: missing env vars fall back to the legacy single-id behaviour. A worker running this SDK against an old backend (without UIPATH_AGENT_ID) falls back to UIPATH_PROJECT_ID and behaves identically to today.

Files touched

packages/uipath-platform/src/uipath/platform/common/constants.py        +5
packages/uipath-platform/src/uipath/platform/common/_config.py          +25
packages/uipath/src/uipath/_cli/_evals/_progress_reporter.py            +5/-3
packages/uipath/src/uipath/_cli/_evals/_telemetry.py                    +6/-7
packages/uipath/src/uipath/_cli/_telemetry.py                           +6/-7
packages/uipath/pyproject.toml                                          version + dep pin
packages/uipath-platform/pyproject.toml                                 version
packages/uipath-platform/tests/common/test_config_env_vars.py           +60 (new)
packages/uipath/tests/cli/eval/test_progress_reporter.py                +81 (new tests)

Tests

• 10 test_config_env_vars.py — fallback chain for each property
• 3 TestAgentIdRouting in test_progress_reporter.py — URL routing, payload field, fallback to project_id

1826 passed, 6 warnings in 12.65s

Ship sequence

1. hdens #7038 — restores eval args after the #6929 regression. ✅ Merged 2026-05-01. Required for evals to run at all.
2. This PR — SDK reads new env vars (gracefully degrades when unset).
3. Agents #5160 — backend emits the env vars. After this, local-workspace telemetry tags the right user + agent.

Jira

• AE-1396 — local-workspace eval regression bug
• Spec: Local-workspace eval runs — clean fix design

🤖 Generated with Claude Code

[AAgnihotry]

where are we passing these values to the eval runs and evalset runs

[github-actions]

🚨 Heads up: uipath-langchain cross-tests are FAILING 🚨

Your changes may break the uipath-langchain-python integration.

⚠️ These checks are NOT enforced by branch protection rules. Please review the failures before merging.

🔍 Inspect the failed run →