Skip to content

test(uipath-agents): coded HITL / process / RAG / tracing tests (PR 4/5)#475

Merged
radugheo merged 1 commit into
mainfrom
test/coded-tests-capabilities
May 5, 2026
Merged

test(uipath-agents): coded HITL / process / RAG / tracing tests (PR 4/5)#475
radugheo merged 1 commit into
mainfrom
test/coded-tests-capabilities

Conversation

@radugheo
Copy link
Copy Markdown
Contributor

Summary

Adds four coded-agent tests for the platform capabilities the existing suite never exercised — human-in-the-loop, process invocation, RAG via Context Grounding, and @traced redaction.

Task Tier What it covers
skill-agent-coded-hitl-create-task e2e LangGraph agent that escalates large expenses through interrupt(CreateTask(app_name="ExpenseReview", app_folder_path="Finance", ...)), compiled with a MemorySaver checkpointer so the interrupt can pause/resume.
skill-agent-coded-process-invoke e2e LangGraph agent that delegates work to an existing RPA process via interrupt(InvokeProcess(name="DataScraper", process_folder_path="Workflows", ...)).
skill-agent-coded-rag-langgraph e2e LangGraph RAG agent that retrieves passages from the company_docs Context Grounding index (ContextGroundingRetriever(index_name=..., folder_path="Shared")) and composes the answer with UiPathChat.
skill-agent-coded-tracing-redaction e2e Simple Function agent whose main is decorated with @traced(name=..., input_processor=..., output_processor=...) to redact passwords and tokens before they reach the trace store.

What each check script asserts

  • HITL — imports of interrupt and CreateTask; interrupt(CreateTask(app_name="ExpenseReview", app_folder_path="Finance", ...)) call shape; .compile(checkpointer=MemorySaver()) (or InMemorySaver); bindings.json declares the app resource with ActivityName=create_async and DisplayLabel=ExpenseReview. Negative case verified: removing the checkpointer makes the check fail.
  • Process invokeinterrupt(InvokeProcess(name="DataScraper", process_folder_path="Workflows", ...)) call shape; bindings.json process resource with ActivityName=invoke_async.
  • RAGContextGroundingRetriever imported from uipath_langchain.* and constructed with index_name="company_docs" / folder_path="Shared"; bindings.json index resource with ActivityName=retrieve_async; both retriever and UiPathChat are lazy (no module-level construction).
  • Tracingmain carries exactly one @traced(...) decorator with all three kwargs: name, input_processor, output_processor. Rejects the conflicting hide_input=True / hide_output=True on the same call site (the skill recommends processors over hide_* for partial visibility). Negative case verified.

Stacked on #474.

Test plan

  • All four check_*.py dry-run green against synthetic well-formed projects.
  • Negative cases verified: HITL without a checkpointer fails; tracing with hide_input=True alongside processors fails.
  • All twelve task YAMLs in coded/ parse and tag lists are consistent.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 29, 2026

Claude finished @radugheo's task in 2m 34s —— View job


PR Review: test(uipath-agents): coded HITL / process / RAG / tracing tests

  • Gather context and read review guidelines
  • Read full diff and all changed files
  • Check test task structure and conventions
  • Check for secrets, cross-skill references, and link validity
  • Post review

Summary

Adds four new coded-agent e2e test tasks covering human-in-the-loop (interrupt(CreateTask)), process invocation (interrupt(InvokeProcess)), RAG via Context Grounding (ContextGroundingRetriever), and @traced redaction. Each task has a YAML definition and a Python check script. The tests are well-structured, follow established patterns from existing coded tests, and make good use of shared utilities (_shared/bindings_assertions, _shared/ast_lazy_init_check).

Change-by-Change Review

1. tests/tasks/uipath-agents/coded/hitl_create_task/check_hitl_create_task.py

Severity: Low

Solid check script. Validates all four HITL primitives (interrupt import, CreateTask import + call shape, checkpointer, app binding). Good use of shared helpers.

One issue: unused import json on line 24. The script imports json but never uses it directly — JSON parsing is handled by load_bindings in the shared module.

Fix this →

2. tests/tasks/uipath-agents/coded/hitl_create_task/hitl_create_task.yaml

Severity: OK

Well-structured task YAML. Tags are consistent with repo conventions ([uipath-agents, e2e, coded, lifecycle:generate, feature:hitl-coded]). This is the only new task that includes a command_executed criterion for uip codedagent new (scaffold step) — the other three only check uip codedagent init. This is fine since the prompt says "scaffold → init", but worth noting the inconsistency (see #5 below).

Weight distribution is reasonable: scaffold (1.0) + init (1.5) + shape check (6.0).

3. tests/tasks/uipath-agents/coded/process_invoke/check_process_invoke.py

Severity: Low

Clean and focused. Validates interrupt(InvokeProcess(...)) call shape, parameter values, binding, and lazy-init.

Minor observation: the regex name\s*=\s*["\']DataScraper["\'] on line 54 would technically match a suffix like other_name="DataScraper". In practice the false-positive risk is negligible since the value DataScraper is unique to this context, but a \b anchor before name (i.e. \bname\s*=) would make it precise. Same pattern used in HITL for app_name and app_folder_path where the kwarg names are longer and inherently more specific.

4. tests/tasks/uipath-agents/coded/process_invoke/process_invoke.yaml

Severity: OK

Clean. Tags consistent. Only checks uip codedagent init (no scaffold step) — matches the other three non-HITL tasks.

5. tests/tasks/uipath-agents/coded/rag_langgraph/check_rag_langgraph.py

Severity: OK

Good two-stage import validation: first checks the class name is referenced at all, then verifies the import source is from uipath_langchain.*. The flexible import regex (uipath_langchain\.[\w.]+) correctly handles both uipath_langchain.agent.tools and uipath_langchain.retrievers.

6. tests/tasks/uipath-agents/coded/rag_langgraph/rag_langgraph.yaml

Severity: OK

Clean. Well-written prompt that specifies two-node architecture (retrieve + respond) with lazy-init requirement.

7. tests/tasks/uipath-agents/coded/tracing_redaction/check_tracing_redaction.py

Severity: Low

Good AST-based validation — properly parses decorator kwargs rather than relying on regex. The negative check for hide_input=True / hide_output=True alongside processors is a nice touch that enforces the skill's recommendation.

One observation on line 49: the import check uses string containment ("from uipath.tracing import traced" not in text and "uipath.tracing" not in text). This is looser than the regex-based import checks in the other scripts. A comment string containing uipath.tracing would pass the gate. Since the AST check downstream validates the actual @traced(...) decorator, this is not a functional issue — the import check is more of a diagnostic hint. But for consistency with the other check scripts, a regex like r"from\s+uipath\.tracing\s+import\s+[^\n]*\btraced\b" would be tighter.

8. tests/tasks/uipath-agents/coded/tracing_redaction/tracing_redaction.yaml

Severity: OK

Clean. The prompt correctly specifies a Simple Function agent (not LangGraph), matching the @traced decorator use case. No secrets — "password" and "token" references are instructional only.

What's Missing

  • Scaffold (uip codedagent new) criterion inconsistency: hitl_create_task.yaml includes a command_executed check for uip codedagent new but the other three tasks (process_invoke, rag_langgraph, tracing_redaction) do not, despite all four prompts saying "scaffold → init". This is not a bug — the shape check scripts validate the project directory exists — but it's a coverage gap. Consider adding the scaffold criterion to the other three for consistency, or removing it from HITL. (Low)
  • No smoke-tier task in this batch: All four tasks are tagged e2e. The repo already has a smoke task (simple_echo), so this isn't a gap per se, but worth noting that these features have no fast-feedback tier.

Area Ratings

Area Status Notes
Frontmatter N/A No skill SKILL.md changes
E2E Tests ✅ OK Four well-structured e2e tasks with shape checks
Skill Body N/A No skill content changes
References & Assets N/A No reference changes
Repo Hygiene ✅ OK No secrets, no cross-skill refs, CODEOWNERS covers parent path

Issues for Manual Review

  • Verify that the bindings.json resource shapes asserted in the check scripts (e.g., app with ActivityName=create_async, process with ActivityName=invoke_async, index with ActivityName=retrieve_async) match the current UiPath SDK binding spec. The check scripts encode specific field names and values that could drift if the SDK changes.
  • Confirm ContextGroundingRetriever is still importable from uipath_langchain.agent.tools and/or uipath_langchain.retrievers — the RAG check accepts both paths.

Conclusion

Clean PR. The four test tasks are well-structured, consistent with existing coded tests, and make good use of shared utilities. The check scripts are thorough — especially the tracing script's AST-based decorator validation and the HITL script's checkpointer assertion.

Three Low items worth addressing:

  1. Remove unused import json in check_hitl_create_task.py:24
  2. Consider \bname anchor in check_process_invoke.py:54 for regex precision
  3. Consider tightening the string-containment import check in check_tracing_redaction.py:49 to a regex for consistency

None are blockers. Approve.


@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from ac9ed4a to bfc8136 Compare April 30, 2026 09:53
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from 15da95f to 4aa145b Compare April 30, 2026 09:53
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from bfc8136 to 737e97b Compare April 30, 2026 09:57
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from 4aa145b to 23b712b Compare April 30, 2026 09:57
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from 737e97b to fc46782 Compare April 30, 2026 11:42
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from 23b712b to 2a98b40 Compare April 30, 2026 11:42
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from fc46782 to 10872ad Compare April 30, 2026 12:04
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from 2a98b40 to d949d53 Compare April 30, 2026 12:04
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from 10872ad to f5c690d Compare May 4, 2026 09:37
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from d949d53 to 9ed1291 Compare May 4, 2026 09:37
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from f5c690d to 891fc53 Compare May 4, 2026 10:09
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from 9ed1291 to fa8bc6b Compare May 4, 2026 10:10
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from 891fc53 to 052970a Compare May 4, 2026 13:27
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from fa8bc6b to a2b7c01 Compare May 4, 2026 13:27
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch 2 times, most recently from 1d918e1 to 57c0f60 Compare May 4, 2026 13:32
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from a2b7c01 to d168789 Compare May 4, 2026 13:32
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from 57c0f60 to 945bcb6 Compare May 4, 2026 15:42
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from d168789 to 59ee4c3 Compare May 4, 2026 15:42
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from 945bcb6 to bc8ac53 Compare May 4, 2026 15:59
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from 86bfe71 to 72a8cc6 Compare May 4, 2026 16:25
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from bc8ac53 to bd7c69b Compare May 4, 2026 16:35
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from 72a8cc6 to 23543b3 Compare May 4, 2026 16:35
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from bd7c69b to 2ea8a8d Compare May 4, 2026 17:24
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from 23543b3 to e3cfe5b Compare May 4, 2026 17:24
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from 2ea8a8d to 9aaa640 Compare May 4, 2026 18:53
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from e3cfe5b to f48ca21 Compare May 4, 2026 19:07
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch 3 times, most recently from b759597 to c77a1bc Compare May 4, 2026 21:26
@radugheo radugheo force-pushed the test/coded-tests-cli-commands branch from 9aaa640 to 95d6283 Compare May 5, 2026 08:26
Base automatically changed from test/coded-tests-cli-commands to main May 5, 2026 08:26
- skill-agent-coded-hitl-create-task (e2e) — LangGraph agent with
  interrupt(CreateTask(app_name="ExpenseReview",
  app_folder_path="Finance", ...)) and a MemorySaver checkpointer.
  Asserts the imports, the call shape, the checkpointer wiring,
  and the `app` binding for ExpenseReview/Finance.
- skill-agent-coded-process-invoke (e2e) — LangGraph agent with
  interrupt(InvokeProcess(name="DataScraper",
  process_folder_path="Workflows", ...)). Asserts the call shape
  and the `process` binding.
- skill-agent-coded-rag-langgraph (e2e) — LangGraph RAG agent with
  ContextGroundingRetriever(index_name="company_docs",
  folder_path="Shared") inside a node. Asserts the import path,
  retriever args, lazy-init, and the `index` binding.
- skill-agent-coded-tracing-redaction (e2e) — Simple Function agent
  whose `main` is decorated with @Traced(name=...,
  input_processor=..., output_processor=...). Asserts all three
  kwargs are present and rejects the conflicting hide_input/
  hide_output combo.
@radugheo radugheo force-pushed the test/coded-tests-capabilities branch from c77a1bc to 6924bbd Compare May 5, 2026 08:34
@radugheo radugheo merged commit d590ddc into main May 5, 2026
3 checks passed
@radugheo radugheo deleted the test/coded-tests-capabilities branch May 5, 2026 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants