Skip to content

Commit e3cfe5b

Browse files
radugheoradugheo
committed
test(uipath-agents): coded HITL / process / RAG / tracing tests
- skill-agent-coded-hitl-create-task (e2e) — LangGraph agent with interrupt(CreateTask(app_name="ExpenseReview", app_folder_path="Finance", ...)) and a MemorySaver checkpointer. Asserts the imports, the call shape, the checkpointer wiring, and the `app` binding for ExpenseReview/Finance. - skill-agent-coded-process-invoke (e2e) — LangGraph agent with interrupt(InvokeProcess(name="DataScraper", process_folder_path="Workflows", ...)). Asserts the call shape and the `process` binding. - skill-agent-coded-rag-langgraph (e2e) — LangGraph RAG agent with ContextGroundingRetriever(index_name="company_docs", folder_path="Shared") inside a node. Asserts the import path, retriever args, lazy-init, and the `index` binding. - skill-agent-coded-tracing-redaction (e2e) — Simple Function agent whose `main` is decorated with @Traced(name=..., input_processor=..., output_processor=...). Asserts all three kwargs are present and rejects the conflicting hide_input/ hide_output combo.
1 parent 2ea8a8d commit e3cfe5b

8 files changed

Lines changed: 626 additions & 0 deletions

File tree

‎tests/tasks/uipath-agents/hitl_create_task/check_hitl_create_task.py‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
#!/usr/bin/env python3
2+
"""HITL coded-agent shape check.
3+
4+
Asserts the four primitives that together make
5+
`interrupt(CreateEscalation)` work end-to-end:
6+
7+
1. `main.py` imports `interrupt` from `langgraph.types`.
8+
2. `main.py` imports `CreateEscalation` from
9+
`uipath.platform.common` and references it inside `interrupt(...)`.
10+
The prompt is an explicit escalation — the skill prescribes
11+
`CreateEscalation` for that path (`CreateTask` is the general
12+
form for non-escalation HITL).
13+
3. The graph is compiled with a `MemorySaver` checkpointer (or
14+
equivalent — the skill teaches `MemorySaver`, but
15+
`InMemorySaver` is acceptable). Without a checkpointer the
16+
interrupt cannot pause/resume.
17+
4. `bindings.json` declares the `app` resource for the Action
18+
Center app the escalation targets — `app_name=ExpenseReview`,
19+
`app_folder_path=Finance`. Without this binding,
20+
`uipath push` would not create the virtual placeholder.
21+
22+
Also runs the lazy-LLM-init AST scan as a hygiene check.
23+
"""
24+
25+
from __future__ import annotations
26+
27+
import json
28+
import os
29+
import re
30+
import sys
31+
from pathlib import Path
32+
33+
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
34+
from _shared.project_root import find_project_root # noqa: E402
35+
36+
ROOT = find_project_root("expense-approver")
37+
38+
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
39+
from _shared.bindings_assertions import ( # noqa: E402
40+
load_bindings,
41+
find_resource,
42+
assert_value_field,
43+
assert_metadata_field,
44+
)
45+
from _shared.ast_lazy_init_check import find_module_level_llm_clients # noqa: E402
46+
47+
48+
def _read_text(path: Path) -> str:
49+
if not path.is_file():
50+
sys.exit(f"FAIL: Missing {path}")
51+
return path.read_text(encoding="utf-8")
52+
53+
54+
def find_graph_module() -> Path:
55+
for candidate in ("main.py", "graph.py"):
56+
path = ROOT / candidate
57+
if path.is_file():
58+
return path
59+
sys.exit(f"FAIL: neither main.py nor graph.py found under {ROOT}")
60+
61+
62+
def check_imports_and_calls(text: str) -> None:
63+
if not re.search(r"from\s+langgraph\.types\s+import\s+[^\n]*\binterrupt\b", text):
64+
sys.exit("FAIL: missing `from langgraph.types import interrupt`")
65+
print("OK: imports `interrupt` from langgraph.types")
66+
if not re.search(r"from\s+uipath\.platform\.common\s+import\s+[^\n]*\bCreateEscalation\b", text):
67+
sys.exit(
68+
"FAIL: missing `from uipath.platform.common import CreateEscalation`. "
69+
"The prompt describes an explicit escalation — the skill prescribes "
70+
"`CreateEscalation` for that path."
71+
)
72+
print("OK: imports CreateEscalation from uipath.platform.common")
73+
if not re.search(r"interrupt\s*\(\s*CreateEscalation\s*\(", text):
74+
sys.exit("FAIL: no `interrupt(CreateEscalation(...))` call site found")
75+
print("OK: graph node calls interrupt(CreateEscalation(...))")
76+
if not re.search(r'app_name\s*=\s*["\']ExpenseReview["\']', text):
77+
sys.exit('FAIL: escalation must target `app_name="ExpenseReview"`')
78+
if not re.search(r'app_folder_path\s*=\s*["\']Finance["\']', text):
79+
sys.exit('FAIL: escalation must target `app_folder_path="Finance"`')
80+
print('OK: escalation targets app_name="ExpenseReview" / app_folder_path="Finance"')
81+
82+
83+
def check_checkpointer(text: str) -> None:
84+
if not re.search(r"\b(MemorySaver|InMemorySaver)\b", text):
85+
sys.exit(
86+
"FAIL: graph is not compiled with a MemorySaver / InMemorySaver "
87+
"checkpointer. Without one the interrupt cannot pause/resume."
88+
)
89+
if not re.search(r"\.compile\s*\([^)]*checkpointer\s*=", text):
90+
sys.exit(
91+
"FAIL: `.compile()` call does not pass `checkpointer=` — the "
92+
"MemorySaver is unused."
93+
)
94+
print("OK: graph is compiled with a checkpointer")
95+
96+
97+
def check_app_binding() -> None:
98+
doc = load_bindings(ROOT / "bindings.json")
99+
entry = find_resource(doc, resource="app", key="ExpenseReview.Finance")
100+
assert_value_field(entry, field="name", expected="ExpenseReview")
101+
assert_value_field(entry, field="folderPath", expected="Finance")
102+
assert_metadata_field(entry, field="ActivityName", expected="create_async")
103+
assert_metadata_field(entry, field="DisplayLabel", expected="ExpenseReview")
104+
print("OK: bindings.json declares the ExpenseReview/Finance `app` resource")
105+
106+
107+
def main() -> None:
108+
if not ROOT.is_dir():
109+
sys.exit(f"FAIL: project directory {ROOT} does not exist")
110+
module = find_graph_module()
111+
text = _read_text(module)
112+
check_imports_and_calls(text)
113+
check_checkpointer(text)
114+
violations = find_module_level_llm_clients(module)
115+
if violations:
116+
sys.exit("FAIL: " + " | ".join(violations))
117+
print("OK: no module-level UiPath* construction")
118+
check_app_binding()
119+
120+
121+
if __name__ == "__main__":
122+
main()

‎tests/tasks/uipath-agents/hitl_create_task/hitl_create_task.yaml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
task_id: skill-agent-coded-hitl-create-task
2+
description: >
3+
Coded-agent HITL via `interrupt(CreateEscalation(...))`. Verifies
4+
the agent imports `interrupt` from `langgraph.types` and
5+
`CreateEscalation` from `uipath.platform.common`, compiles the
6+
graph with a `MemorySaver` checkpointer, and emits the `app`
7+
binding for the Action Center app the escalation targets.
8+
tags: [uipath-agents, e2e, coded, lifecycle:generate, feature:hitl-coded]
9+
max_iterations: 1
10+
11+
agent:
12+
type: claude-code
13+
permission_mode: acceptEdits
14+
allowed_tools: ["Skill", "Bash", "Read", "Write", "Edit", "Glob", "Grep"]
15+
turn_timeout: 1200
16+
17+
sandbox:
18+
driver: tempdir
19+
python: {}
20+
21+
initial_prompt: |
22+
Build a LangGraph UiPath coded agent named `expense-approver` that
23+
escalates expense-reimbursement requests over `$1000` to a human
24+
reviewer in Action Center. Below the threshold, the agent
25+
auto-approves.
26+
27+
The escalation should target an Action Center app named
28+
`ExpenseReview` in folder `Finance`, passing the full request
29+
payload to the reviewer. The graph must support pause/resume so
30+
the escalation can wait for the human response.
31+
32+
Input: `amount` (number), `description` (string), `submitter`
33+
(string).
34+
Output: `approved` (bool), `reviewer_decision` (string|null).
35+
36+
Sync `bindings.json` so the Action Center resource is declared.
37+
38+
Take the agent through scaffold → init. Do NOT run, publish,
39+
upload, or deploy. Do NOT call `uip login`. Do NOT pause between
40+
planning and implementation. Complete end-to-end in a single pass.
41+
42+
success_criteria:
43+
- type: command_executed
44+
description: "Agent scaffolded the project with uip codedagent new"
45+
tool_name: "Bash"
46+
command_pattern: 'uip\s+codedagent\s+new'
47+
min_count: 1
48+
weight: 1.0
49+
pass_threshold: 1.0
50+
51+
- type: command_executed
52+
description: "Agent ran uip codedagent init"
53+
tool_name: "Bash"
54+
command_pattern: 'uip\s+codedagent\s+init'
55+
min_count: 1
56+
weight: 1.5
57+
pass_threshold: 1.0
58+
59+
- type: run_command
60+
description: "interrupt(CreateEscalation) shape, MemorySaver checkpointer, app binding"
61+
command: "python3 $TASK_DIR/check_hitl_create_task.py"
62+
timeout: 30
63+
expected_exit_code: 0
64+
weight: 6.0
65+
pass_threshold: 1.0

‎tests/tasks/uipath-agents/process_invoke/check_process_invoke.py‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
#!/usr/bin/env python3
2+
"""InvokeProcess coded-agent shape check.
3+
4+
Asserts:
5+
1. `main.py` imports `InvokeProcess` from `uipath.platform.common`
6+
and references it inside `interrupt(...)`.
7+
2. The call site passes both `name="DataScraper"` and
8+
`process_folder_path="Workflows"`.
9+
3. `bindings.json` declares the `process` resource for
10+
DataScraper / Workflows.
11+
4. No module-level UiPath* construction.
12+
"""
13+
14+
from __future__ import annotations
15+
16+
import os
17+
import re
18+
import sys
19+
from pathlib import Path
20+
21+
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
22+
from _shared.project_root import find_project_root # noqa: E402
23+
24+
ROOT = find_project_root("data-orchestrator")
25+
26+
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
27+
from _shared.bindings_assertions import ( # noqa: E402
28+
load_bindings,
29+
find_resource,
30+
assert_value_field,
31+
assert_metadata_field,
32+
)
33+
from _shared.ast_lazy_init_check import find_module_level_llm_clients # noqa: E402
34+
35+
36+
def _read_text(path: Path) -> str:
37+
if not path.is_file():
38+
sys.exit(f"FAIL: Missing {path}")
39+
return path.read_text(encoding="utf-8")
40+
41+
42+
def find_graph_module() -> Path:
43+
for candidate in ("main.py", "graph.py"):
44+
path = ROOT / candidate
45+
if path.is_file():
46+
return path
47+
sys.exit(f"FAIL: neither main.py nor graph.py found under {ROOT}")
48+
49+
50+
def check_imports_and_calls(text: str) -> None:
51+
if not re.search(r"from\s+uipath\.platform\.common\s+import\s+[^\n]*\bInvokeProcess\b", text):
52+
sys.exit("FAIL: missing `from uipath.platform.common import InvokeProcess`")
53+
if not re.search(r"from\s+langgraph\.types\s+import\s+[^\n]*\binterrupt\b", text):
54+
sys.exit("FAIL: missing `from langgraph.types import interrupt`")
55+
if not re.search(r"interrupt\s*\(\s*InvokeProcess\s*\(", text):
56+
sys.exit("FAIL: no `interrupt(InvokeProcess(...))` call site found")
57+
if not re.search(r'name\s*=\s*["\']DataScraper["\']', text):
58+
sys.exit('FAIL: InvokeProcess call does not pass name="DataScraper"')
59+
if not re.search(r'process_folder_path\s*=\s*["\']Workflows["\']', text):
60+
sys.exit('FAIL: InvokeProcess call does not pass process_folder_path="Workflows"')
61+
print('OK: graph node calls `interrupt(InvokeProcess(name="DataScraper", process_folder_path="Workflows", ...))`')
62+
63+
64+
def check_process_binding() -> None:
65+
doc = load_bindings(ROOT / "bindings.json")
66+
entry = find_resource(doc, resource="process", key="DataScraper.Workflows")
67+
assert_value_field(entry, field="name", expected="DataScraper")
68+
assert_value_field(entry, field="folderPath", expected="Workflows")
69+
assert_metadata_field(entry, field="ActivityName", expected="invoke_async")
70+
print("OK: bindings.json declares the DataScraper/Workflows `process` resource")
71+
72+
73+
def main() -> None:
74+
if not ROOT.is_dir():
75+
sys.exit(f"FAIL: project directory {ROOT} does not exist")
76+
module = find_graph_module()
77+
text = _read_text(module)
78+
check_imports_and_calls(text)
79+
violations = find_module_level_llm_clients(module)
80+
if violations:
81+
sys.exit("FAIL: " + " | ".join(violations))
82+
print("OK: no module-level UiPath* construction")
83+
check_process_binding()
84+
85+
86+
if __name__ == "__main__":
87+
main()

‎tests/tasks/uipath-agents/process_invoke/process_invoke.yaml‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
task_id: skill-agent-coded-process-invoke
2+
description: >
3+
Coded-agent process invocation via `interrupt(InvokeProcess(...))`.
4+
Verifies the agent imports `InvokeProcess` from
5+
`uipath.platform.common`, calls it from a LangGraph node with both
6+
`name` and `process_folder_path` set, and emits the `process`
7+
binding for the invoked process.
8+
tags: [uipath-agents, e2e, coded, lifecycle:generate, feature:process-invocation]
9+
max_iterations: 1
10+
11+
agent:
12+
type: claude-code
13+
permission_mode: acceptEdits
14+
allowed_tools: ["Skill", "Bash", "Read", "Write", "Edit", "Glob", "Grep"]
15+
turn_timeout: 1200
16+
17+
sandbox:
18+
driver: tempdir
19+
python: {}
20+
21+
initial_prompt: |
22+
Build a LangGraph UiPath coded agent named `data-orchestrator`
23+
that delegates data scraping to an existing RPA process named
24+
`DataScraper` in folder `Workflows`. The agent forwards a target
25+
URL to the process and waits for the rows it produces, then
26+
returns them.
27+
28+
Input: `target_url` (string).
29+
Output: `row_count` (int), `rows` (list of dict).
30+
31+
Sync `bindings.json` so the process resource is declared.
32+
33+
Take the agent through scaffold → init. Do NOT run, publish,
34+
upload, or deploy. Do NOT pause between planning and
35+
implementation. Complete end-to-end in a single pass.
36+
37+
success_criteria:
38+
- type: command_executed
39+
description: "Agent ran uip codedagent init"
40+
tool_name: "Bash"
41+
command_pattern: 'uip\s+codedagent\s+init'
42+
min_count: 1
43+
weight: 1.5
44+
pass_threshold: 1.0
45+
46+
- type: run_command
47+
description: "InvokeProcess shape, lazy-LLM-init, process binding"
48+
command: "python3 $TASK_DIR/check_process_invoke.py"
49+
timeout: 30
50+
expected_exit_code: 0
51+
weight: 6.0
52+
pass_threshold: 1.0

0 commit comments

Comments
 (0)