test(uipath-agents): coded anti-pattern + bindings sweep tests (PR 5/5)

[radugheo]

Summary

Adds seven coded-agent tests — four anti-pattern smokes that verify the skill catches the most expensive coded mistakes, plus three bindings-sweep tests that close the binding-resource and entrypoint-resolution gaps.

Anti-pattern tests (smoke)

Each task seeds a violating project layout in the prompt; the skill must drive the agent to detect and fix the violation in place.

Task	What it covers
skill-agent-coded-antipattern-build-system	Pre-seeded pyproject.toml with [build-system] (hatchling). Check verifies the section is removed and [project] survives.
skill-agent-coded-antipattern-module-level-llm	Pre-seeded LangGraph main.py with llm = UiPathChat(...) at module level. Check verifies no module-level UiPath* construction remains and the top-level graph variable is preserved.
skill-agent-coded-antipattern-wrong-sdk-import	Pre-seeded from uipath import UiPath (raises ImportError). Check verifies the import was switched to from uipath.platform import UiPath.
skill-agent-coded-antipattern-login-no-tenant	Goal-only auth prompt with environment / org / tenant. Asserts uip login --output json and uip login tenant set "MyTenant" were both run.

Bindings sweep tests (e2e)

Task	What it covers
skill-agents-bindings-queue-app-index	Five binding types in one project — queue, app, index, connection, mcpServer. Asserts one entry per resource with the correct key (bare for connections; bare for the no-folder queue), value block, and metadata (ActivityName, DisplayLabel, UseConnectionService).
skill-agents-bindings-asset-subtypes	Three asset retrieves annotated str / int / bool. Asserts three asset bindings; per-binding metadata.SubType either matches the annotation (stringAsset / integerAsset / booleanAsset) OR is omitted. A wrong SubType fails — that would mis-place the resource at uipath push time.
skill-agents-bindings-multi-entrypoint	entry-points.json with two entrypoints, two SDK calls across two source files. Asserts each binding's EntryPointUniqueId references one of the real entrypoints (or has no entrypoint link at all); fabricated UUIDs fail.

Stacked on #475.

Test plan

• All six sidecar check_*.py scripts dry-run green against synthetic well-formed projects (the seventh task, antipattern-login-no-tenant, has no sidecar — pure command_executed checks).
• Negative cases verified: leaving [build-system] in fails the build-system check; from uipath import UiPath fails the import check; a fabricated entrypoint UUID fails the multi-entrypoint check; SubType="credentialAsset" on a str-annotated retrieve fails the asset-subtypes check.
• All nineteen task YAMLs in coded/ parse and tag lists are consistent.

[github-actions]

Claude finished @radugheo's task in 2m 39s —— View job

---

Summary

Adds seven coder_eval test tasks (four anti-pattern smoke tests, three bindings-sweep e2e tests) and six Python check scripts for the uipath-agents coded-agent skill. Well-structured, consistent with existing tasks, and the check scripts use the shared _shared/ helpers correctly.

Change-by-Change Review

1. Anti-pattern task YAMLs (4 files)

Severity: OK — antipattern_build_system.yaml, antipattern_login_no_tenant.yaml, antipattern_module_level_llm.yaml, antipattern_wrong_sdk_import.yaml

Tags are correct (uipath-agents first, smoke test type, valid facets). task_id follows skill-agent-coded-antipattern-*, consistent with skill-agent-coded-* convention. Each has max_iterations: 1, reasonable success_criteria weights, and minimal initial_prompts that describe the goal and seed the violating project without prescribing steps.

2. Bindings sweep task YAMLs (3 files)

Severity: Medium — bindings_asset_subtypes.yaml, bindings_multi_entrypoint.yaml, bindings_queue_app_index.yaml

Tags are correct (uipath-agents first, e2e test type, valid facets). However, the task_id values deviate from the established convention:

New bindings task_id	Convention would be
skill-agents-bindings-asset-subtypes	skill-agent-coded-bindings-asset-subtypes
skill-agents-bindings-multi-entrypoint	skill-agent-coded-bindings-multi-entrypoint
skill-agents-bindings-queue-app-index	skill-agent-coded-bindings-queue-app-index

All 16 existing tasks in coded/ use skill-agent-coded-*. The new bindings tasks use skill-agents-bindings-* — two deviations: agents instead of agent, and drops coded. If this is intentional (a new naming tier for bindings-specific tests), it should be documented. Otherwise, align with the existing pattern.

Fix this →

3. check_antipattern_build_system.py

Severity: OK

Clean regex-based check: line-anchored [build-system] detection, sanity check that [project] survived. Correct use of re.M flag. No external dependencies.

4. check_antipattern_module_level_llm.py

Severity: OK

Correctly imports from _shared.ast_lazy_init_check for AST-level module-level client detection. Also verifies the graph = export survived the refactor. Good separation of concerns.

5. check_antipattern_wrong_sdk_import.py

Severity: Low — minor regex edge case

The WRONG regex (check_antipattern_wrong_sdk_import.py:21):

WRONG = re.compile(r"^\s*from\s+uipath\s+import\s+(?:[^,\n]*,\s*)*UiPath(?:\s*,|$)", re.M)

The trailing (?:\s*,|$) requires UiPath to be followed by a comma or end-of-line. An import with a trailing comment like from uipath import UiPath # wrong would not match. This is unlikely in practice (the agent would typically strip comments when recreating), and the RIGHT regex catches the missing correct import regardless, so it's a defense-in-depth gap rather than a functional bug. A more robust alternative:

WRONG = re.compile(r"^\s*from\s+uipath\s+import\s+(?:[^,\n]*,\s*)*UiPath\b", re.M)

Using \b (word boundary) covers all trailing contexts — matches the same approach used in the RIGHT regex on line 22.

6. check_bindings_asset_subtypes.py

Severity: OK

Good acceptance criteria: requires exactly 3 asset bindings, accepts SubType either matching the annotation or being omitted, and explicitly fails on wrong SubType. Uses _shared.bindings_assertions correctly.

7. check_bindings_multi_entrypoint.py

Severity: OK

Reads real entrypoint UUIDs from entry-points.json, then validates any EntryPointUniqueId in the bindings references a real one. Correctly accepts absence of the field (agent might leave it off). The fabricated-UUID detection is the load-bearing assertion.

8. check_bindings_queue_app_index.py

Severity: OK

Covers all five resource types with correct key construction rules (bare key for connections, <name>.<folder> for others). Validates metadata fields (ActivityName, DisplayLabel, UseConnectionService). Deduplication check ensures exactly one entry per kind.

What's Missing

• --output json not enforced on tenant set: antipattern_login_no_tenant.yaml:46 — the prompt tells the agent "Use --output json on every uip command you run" but the command_pattern for tenant set doesn't require it. The uip login pattern on line 38 does enforce --output json. This is likely intentional leniency (tenant set is the critical behavior being tested), but noting the asymmetry.
• Nothing else identified.

Area Ratings

Area	Status	Notes
Frontmatter	N/A	No skill changes — tests only
E2E Tests	Issue	task_id naming inconsistency on bindings tasks
Skill Body	N/A	No skill changes
References & Assets	N/A	No skill changes
Repo Hygiene	OK	No secrets, no personal paths, no cross-skill refs, changes scoped to test tasks

Issues for Manual Review

• The index binding (check_bindings_queue_app_index.py:49) asserts ActivityName = "retrieve_async" even though the call site is search_async. The check comment explains this matches the bindings reference ("ALL methods bind via retrieve_async"). Verify this is still the current contract — if the reference changes, this assertion will silently pass incorrect bindings.
• The antipattern_login_no_tenant task seeds only a goal prompt (no project files). Verify the skill activates correctly from a bare auth request without a pre-existing project context.

Conclusion

Clean PR. The tests are well-designed with good positive/negative coverage and correct use of shared helpers. One Medium issue: the three bindings task_id values should align with the established skill-agent-coded-* convention. One Low cosmetic: the WRONG regex in check_antipattern_wrong_sdk_import.py could use \b instead of (?:\s*,|$) for consistency with the RIGHT regex on the same page. Neither blocks merge.

---