Skip to content

Commit fa8bc6b

Browse files
radugheoradugheo
committed
test(uipath-agents): coded HITL / process / RAG / tracing tests
- skill-agent-coded-hitl-create-task (e2e) — LangGraph agent with interrupt(CreateTask(app_name="ExpenseReview", app_folder_path="Finance", ...)) and a MemorySaver checkpointer. Asserts the imports, the call shape, the checkpointer wiring, and the `app` binding for ExpenseReview/Finance. - skill-agent-coded-process-invoke (e2e) — LangGraph agent with interrupt(InvokeProcess(name="DataScraper", process_folder_path="Workflows", ...)). Asserts the call shape and the `process` binding. - skill-agent-coded-rag-langgraph (e2e) — LangGraph RAG agent with ContextGroundingRetriever(index_name="company_docs", folder_path="Shared") inside a node. Asserts the import path, retriever args, lazy-init, and the `index` binding. - skill-agent-coded-tracing-redaction (e2e) — Simple Function agent whose `main` is decorated with @Traced(name=..., input_processor=..., output_processor=...). Asserts all three kwargs are present and rejects the conflicting hide_input/ hide_output combo.
1 parent 891fc53 commit fa8bc6b

8 files changed

Lines changed: 615 additions & 0 deletions

File tree

‎tests/tasks/uipath-agents/hitl_create_task/check_hitl_create_task.py‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
#!/usr/bin/env python3
2+
"""HITL coded-agent shape check.
3+
4+
Asserts the four primitives that together make `interrupt(CreateTask)`
5+
work end-to-end:
6+
7+
1. `main.py` imports `interrupt` from `langgraph.types`.
8+
2. `main.py` imports `CreateTask` from `uipath.platform.common`
9+
and references it inside `interrupt(...)`.
10+
3. The graph is compiled with a `MemorySaver` checkpointer (or
11+
equivalent — the skill teaches `MemorySaver`, but
12+
`InMemorySaver` is acceptable). Without a checkpointer the
13+
interrupt cannot pause/resume.
14+
4. `bindings.json` declares the `app` resource for the Action
15+
Center app the escalation targets — `app_name=ExpenseReview`,
16+
`app_folder_path=Finance`. Without this binding,
17+
`uipath push` would not create the virtual placeholder.
18+
19+
Also runs the lazy-LLM-init AST scan as a hygiene check.
20+
"""
21+
22+
from __future__ import annotations
23+
24+
import json
25+
import os
26+
import re
27+
import sys
28+
from pathlib import Path
29+
30+
ROOT = Path(os.getcwd())
31+
32+
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
33+
from _shared.bindings_assertions import ( # noqa: E402
34+
load_bindings,
35+
find_resource,
36+
assert_value_field,
37+
assert_metadata_field,
38+
)
39+
from _shared.ast_lazy_init_check import find_module_level_llm_clients # noqa: E402
40+
41+
42+
def _read_text(path: Path) -> str:
43+
if not path.is_file():
44+
sys.exit(f"FAIL: Missing {path}")
45+
return path.read_text(encoding="utf-8")
46+
47+
48+
def find_graph_module() -> Path:
49+
for candidate in ("main.py", "graph.py"):
50+
path = ROOT / candidate
51+
if path.is_file():
52+
return path
53+
sys.exit(f"FAIL: neither main.py nor graph.py found under {ROOT}")
54+
55+
56+
def check_imports_and_calls(text: str) -> None:
57+
if not re.search(r"from\s+langgraph\.types\s+import\s+[^\n]*\binterrupt\b", text):
58+
sys.exit("FAIL: missing `from langgraph.types import interrupt`")
59+
print("OK: imports `interrupt` from langgraph.types")
60+
if not re.search(r"from\s+uipath\.platform\.common\s+import\s+[^\n]*\bCreateTask\b", text):
61+
sys.exit("FAIL: missing `from uipath.platform.common import CreateTask`")
62+
print("OK: imports `CreateTask` from uipath.platform.common")
63+
if not re.search(r"interrupt\s*\(\s*CreateTask\s*\(", text):
64+
sys.exit("FAIL: no `interrupt(CreateTask(...))` call site found")
65+
print("OK: graph node calls `interrupt(CreateTask(...))`")
66+
if not re.search(r'app_name\s*=\s*["\']ExpenseReview["\']', text):
67+
sys.exit('FAIL: CreateTask call does not pass app_name="ExpenseReview"')
68+
if not re.search(r'app_folder_path\s*=\s*["\']Finance["\']', text):
69+
sys.exit('FAIL: CreateTask call does not pass app_folder_path="Finance"')
70+
print('OK: CreateTask targets app_name="ExpenseReview" / app_folder_path="Finance"')
71+
72+
73+
def check_checkpointer(text: str) -> None:
74+
if not re.search(r"\b(MemorySaver|InMemorySaver)\b", text):
75+
sys.exit(
76+
"FAIL: graph is not compiled with a MemorySaver / InMemorySaver "
77+
"checkpointer. Without one the interrupt cannot pause/resume."
78+
)
79+
if not re.search(r"\.compile\s*\([^)]*checkpointer\s*=", text):
80+
sys.exit(
81+
"FAIL: `.compile()` call does not pass `checkpointer=` — the "
82+
"MemorySaver is unused."
83+
)
84+
print("OK: graph is compiled with a checkpointer")
85+
86+
87+
def check_app_binding() -> None:
88+
doc = load_bindings(ROOT / "bindings.json")
89+
entry = find_resource(doc, resource="app", key="ExpenseReview.Finance")
90+
assert_value_field(entry, field="name", expected="ExpenseReview")
91+
assert_value_field(entry, field="folderPath", expected="Finance")
92+
assert_metadata_field(entry, field="ActivityName", expected="create_async")
93+
assert_metadata_field(entry, field="DisplayLabel", expected="ExpenseReview")
94+
print("OK: bindings.json declares the ExpenseReview/Finance `app` resource")
95+
96+
97+
def main() -> None:
98+
if not ROOT.is_dir():
99+
sys.exit(f"FAIL: project directory {ROOT} does not exist")
100+
module = find_graph_module()
101+
text = _read_text(module)
102+
check_imports_and_calls(text)
103+
check_checkpointer(text)
104+
violations = find_module_level_llm_clients(module)
105+
if violations:
106+
sys.exit("FAIL: " + " | ".join(violations))
107+
print("OK: no module-level UiPath* construction")
108+
check_app_binding()
109+
110+
111+
if __name__ == "__main__":
112+
main()

‎tests/tasks/uipath-agents/hitl_create_task/hitl_create_task.yaml‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
task_id: skill-agent-coded-hitl-create-task
2+
description: >
3+
Coded-agent HITL via `interrupt(CreateTask(...))`. Verifies the
4+
skill guides the agent to import `interrupt` from
5+
`langgraph.types`, `CreateTask` from `uipath.platform.common`,
6+
attach a `MemorySaver` checkpointer when compiling the graph, and
7+
emit an `app` binding for the Action Center app referenced by the
8+
escalation. Closes report Top-10 #5 — `interrupt(CreateTask)` and
9+
`MemorySaver` are foundational coded HITL primitives that were
10+
wholly untested.
11+
tags: [uipath-agents, e2e, coded, lifecycle:generate, feature:hitl-coded]
12+
max_iterations: 1
13+
14+
agent:
15+
type: claude-code
16+
permission_mode: acceptEdits
17+
allowed_tools: ["Skill", "Bash", "Read", "Write", "Edit", "Glob", "Grep"]
18+
turn_timeout: 1200
19+
20+
sandbox:
21+
driver: tempdir
22+
python: {}
23+
24+
initial_prompt: |
25+
Build a LangGraph UiPath coded agent named `expense-approver` that
26+
escalates expense-reimbursement requests over `$1000` to a human
27+
reviewer in Action Center. Below the threshold, the agent
28+
auto-approves.
29+
30+
The escalation should target an Action Center app named
31+
`ExpenseReview` in folder `Finance`, passing the full request
32+
payload to the reviewer. The graph must support pause/resume so
33+
the escalation can wait for the human response.
34+
35+
Input: `amount` (number), `description` (string), `submitter`
36+
(string).
37+
Output: `approved` (bool), `reviewer_decision` (string|null).
38+
39+
Sync `bindings.json` so the Action Center resource is declared.
40+
41+
Take the agent through scaffold → init. Do NOT run, publish,
42+
upload, or deploy. Do NOT call `uip login`. Do NOT pause between
43+
planning and implementation. Complete end-to-end in a single pass.
44+
45+
success_criteria:
46+
- type: command_executed
47+
description: "Agent scaffolded the project with uip codedagent new"
48+
tool_name: "Bash"
49+
command_pattern: 'uip\s+codedagent\s+new'
50+
min_count: 1
51+
weight: 1.0
52+
pass_threshold: 1.0
53+
54+
- type: command_executed
55+
description: "Agent ran uip codedagent init"
56+
tool_name: "Bash"
57+
command_pattern: 'uip\s+codedagent\s+init'
58+
min_count: 1
59+
weight: 1.5
60+
pass_threshold: 1.0
61+
62+
- type: run_command
63+
description: "interrupt(CreateTask) shape, MemorySaver checkpointer, app binding"
64+
command: "python3 $TASK_DIR/check_hitl_create_task.py"
65+
timeout: 30
66+
expected_exit_code: 0
67+
weight: 6.0
68+
pass_threshold: 1.0

‎tests/tasks/uipath-agents/process_invoke/check_process_invoke.py‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
#!/usr/bin/env python3
2+
"""InvokeProcess coded-agent shape check.
3+
4+
Asserts:
5+
1. `main.py` imports `InvokeProcess` from `uipath.platform.common`
6+
and references it inside `interrupt(...)`.
7+
2. The call site passes both `name="DataScraper"` and
8+
`process_folder_path="Workflows"`.
9+
3. `bindings.json` declares the `process` resource for
10+
DataScraper / Workflows.
11+
4. No module-level UiPath* construction.
12+
"""
13+
14+
from __future__ import annotations
15+
16+
import os
17+
import re
18+
import sys
19+
from pathlib import Path
20+
21+
ROOT = Path(os.getcwd())
22+
23+
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
24+
from _shared.bindings_assertions import ( # noqa: E402
25+
load_bindings,
26+
find_resource,
27+
assert_value_field,
28+
assert_metadata_field,
29+
)
30+
from _shared.ast_lazy_init_check import find_module_level_llm_clients # noqa: E402
31+
32+
33+
def _read_text(path: Path) -> str:
34+
if not path.is_file():
35+
sys.exit(f"FAIL: Missing {path}")
36+
return path.read_text(encoding="utf-8")
37+
38+
39+
def find_graph_module() -> Path:
40+
for candidate in ("main.py", "graph.py"):
41+
path = ROOT / candidate
42+
if path.is_file():
43+
return path
44+
sys.exit(f"FAIL: neither main.py nor graph.py found under {ROOT}")
45+
46+
47+
def check_imports_and_calls(text: str) -> None:
48+
if not re.search(r"from\s+uipath\.platform\.common\s+import\s+[^\n]*\bInvokeProcess\b", text):
49+
sys.exit("FAIL: missing `from uipath.platform.common import InvokeProcess`")
50+
if not re.search(r"from\s+langgraph\.types\s+import\s+[^\n]*\binterrupt\b", text):
51+
sys.exit("FAIL: missing `from langgraph.types import interrupt`")
52+
if not re.search(r"interrupt\s*\(\s*InvokeProcess\s*\(", text):
53+
sys.exit("FAIL: no `interrupt(InvokeProcess(...))` call site found")
54+
if not re.search(r'name\s*=\s*["\']DataScraper["\']', text):
55+
sys.exit('FAIL: InvokeProcess call does not pass name="DataScraper"')
56+
if not re.search(r'process_folder_path\s*=\s*["\']Workflows["\']', text):
57+
sys.exit('FAIL: InvokeProcess call does not pass process_folder_path="Workflows"')
58+
print('OK: graph node calls `interrupt(InvokeProcess(name="DataScraper", process_folder_path="Workflows", ...))`')
59+
60+
61+
def check_process_binding() -> None:
62+
doc = load_bindings(ROOT / "bindings.json")
63+
entry = find_resource(doc, resource="process", key="DataScraper.Workflows")
64+
assert_value_field(entry, field="name", expected="DataScraper")
65+
assert_value_field(entry, field="folderPath", expected="Workflows")
66+
assert_metadata_field(entry, field="ActivityName", expected="invoke_async")
67+
print("OK: bindings.json declares the DataScraper/Workflows `process` resource")
68+
69+
70+
def main() -> None:
71+
if not ROOT.is_dir():
72+
sys.exit(f"FAIL: project directory {ROOT} does not exist")
73+
module = find_graph_module()
74+
text = _read_text(module)
75+
check_imports_and_calls(text)
76+
violations = find_module_level_llm_clients(module)
77+
if violations:
78+
sys.exit("FAIL: " + " | ".join(violations))
79+
print("OK: no module-level UiPath* construction")
80+
check_process_binding()
81+
82+
83+
if __name__ == "__main__":
84+
main()

‎tests/tasks/uipath-agents/process_invoke/process_invoke.yaml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
task_id: skill-agent-coded-process-invoke
2+
description: >
3+
Coded-agent process invocation via `interrupt(InvokeProcess(...))`.
4+
Verifies the skill guides the agent to import `InvokeProcess` from
5+
`uipath.platform.common`, call it from a LangGraph node with both
6+
`name` and `process_folder_path` set, and emit a `process` binding
7+
for the invoked process. Closes report Top-10 #21 — process
8+
invocation is currently only covered indirectly (via bindings_sync
9+
scanning), never as a real execute path.
10+
tags: [uipath-agents, e2e, coded, lifecycle:generate, feature:process-invocation]
11+
max_iterations: 1
12+
13+
agent:
14+
type: claude-code
15+
permission_mode: acceptEdits
16+
allowed_tools: ["Skill", "Bash", "Read", "Write", "Edit", "Glob", "Grep"]
17+
turn_timeout: 1200
18+
19+
sandbox:
20+
driver: tempdir
21+
python: {}
22+
23+
initial_prompt: |
24+
Build a LangGraph UiPath coded agent named `data-orchestrator`
25+
that delegates data scraping to an existing RPA process named
26+
`DataScraper` in folder `Workflows`. The agent forwards a target
27+
URL to the process and waits for the rows it produces, then
28+
returns them.
29+
30+
Input: `target_url` (string).
31+
Output: `row_count` (int), `rows` (list of dict).
32+
33+
Sync `bindings.json` so the process resource is declared.
34+
35+
Take the agent through scaffold → init. Do NOT run, publish,
36+
upload, or deploy. Do NOT pause between planning and
37+
implementation. Complete end-to-end in a single pass.
38+
39+
success_criteria:
40+
- type: command_executed
41+
description: "Agent ran uip codedagent init"
42+
tool_name: "Bash"
43+
command_pattern: 'uip\s+codedagent\s+init'
44+
min_count: 1
45+
weight: 1.5
46+
pass_threshold: 1.0
47+
48+
- type: run_command
49+
description: "InvokeProcess shape, lazy-LLM-init, process binding"
50+
command: "python3 $TASK_DIR/check_process_invoke.py"
51+
timeout: 30
52+
expected_exit_code: 0
53+
weight: 6.0
54+
pass_threshold: 1.0

0 commit comments

Comments
 (0)