Skip to content

feat(hitl-skill): resilience for real-world developer edge cases#477

Open
dushyant-uipath wants to merge 1 commit intomainfrom
feat/hitl-skill-resilience
Open

feat(hitl-skill): resilience for real-world developer edge cases#477
dushyant-uipath wants to merge 1 commit intomainfrom
feat/hitl-skill-resilience

Conversation

@dushyant-uipath
Copy link
Copy Markdown
Collaborator

@dushyant-uipath dushyant-uipath commented Apr 29, 2026

Summary

  • QuickForm as safe default — when user says "just pick one" or is unsure, default to QuickForm and explain why
  • Fallback rules table — explicit guidance for when each path (Existing App, New App) hits a blocker: app not found → offer QuickForm fallback; no dist build → ask to build first or offer QuickForm; auth expired → run `uip login`
  • Data type inference warnings — during schema design, flag number/date/boolean candidates and confirm before writing
  • Vague schema handling — when user says "add some fields", infer from flow context, show explicit proposal, handle no-upstream-nodes case
  • Partial confirmation — if user says "yes but change X", re-show updated schema before writing
  • 2 new critical rules — node ID collision check (scan existing IDs); never report failed validation as done

Motivation

Recommendation: the skill should be resilient to real-world developer behavior — users who don't have an app ready, can't describe their schema clearly, give ambiguous confirmations, or don't know which path to choose.

Test plan

  • User selects "Existing Deployed App" with an app name that doesn't exist → skill offers fallback to QuickForm
  • User selects "New Coded Action App" but says they don't have the source ready → skill offers QuickForm
  • User says "just pick something for me" at Step 3 → skill defaults to QuickForm with explanation
  • User describes a field as "invoice amount" → skill proposes number type and asks to confirm
  • User says "yes but rename the field" after schema proposal → skill re-shows updated schema before writing
  • Flow with no upstream nodes → skill notes "no upstream nodes" and uses output-direction fields only

- QuickForm as safe default when user is unsure of task type
- Fallback rules for blocked paths (app not found, no dist build, auth expired)
- Data type inference warnings for number/date/boolean fields
- Vague schema description handling with explicit proposal-before-write
- Partial confirmation handling (re-show updated schema before writing)
- New critical rules: node ID collision check and validation failure recovery
- Production readiness notes per task type in Step 6
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 29, 2026

Claude finished @dushyant-uipath's task in 2m 24s —— View job


PR Review: feat(hitl-skill): resilience for real-world developer edge cases

  • Read review criteria and project conventions
  • Read existing PR conversation
  • Analyze full diff against origin/main
  • Read full changed file for context
  • Check skill-specific rules (links, cross-refs, CODEOWNERS, tests)
  • Post review findings

Summary

Adds resilience guidance to the HITL skill: QuickForm as safe default when users are unsure, a fallback table for path blockers, data type inference warnings during schema design, handling for vague/partial user confirmations, and two new critical rules (node ID collision, never report failed validation as done).

Change-by-Change Review

1. skills/uipath-human-in-the-loop/SKILL.md — New Critical Rules (lines 39–40)

Severity: OK

Rules 7 and 8 are well-scoped and actionable. Rule 7 (check existing node IDs) prevents a real collision bug. Rule 8 (never report failed validation as done) closes a gap where an agent might treat a validation error as success. Both are prescriptive with clear actions.

2. skills/uipath-human-in-the-loop/SKILL.md — QuickForm default + fallback table (lines 138–153)

Severity: OK

The "just pick one" default (line 138) is a pragmatic addition. The fallback table covers the right blockers: app not found, no dist build, no source path, auth expired. Each row includes a specific response template with a recovery path.

3. skills/uipath-human-in-the-loop/SKILL.md — New "Step 4b" section (lines 164–189)

Severity: High — Step ID collision

The new section is titled ## Step 4b — Schema Design Resilience (QuickForm only) (line 164). However, "Step 4b" is already used in the AppTask flow:

  • Line 144: "Read references/hitl-node-apptask.md for Step 4b details"
  • Line 235: "Step 4b must be completed first — app resolved, configuration retrieved."

An agent following the AppTask path will see "Step 4b" and may navigate to the Schema Design Resilience section instead of the apptask reference doc — or vice versa.

Fix: Rename the new section to avoid collision. Suggestion: ## Step 3b — Schema Design Resilience (QuickForm only) (since it applies during schema design which conceptually follows Step 3's choice) or a non-numbered heading like ## Schema Design Resilience (QuickForm only) placed as a subsection under the QuickForm path. Alternatively, rename the existing AppTask "Step 4b" references (which live in the reference doc, not as a SKILL.md heading) to something like "Step A1" to disambiguate.

Fix this →

4. skills/uipath-human-in-the-loop/SKILL.md — Data type warnings table (lines 170–176)

Severity: Low

Content is correct — date, number, and boolean are all valid QuickForm field types per references/hitl-node-quickform.md:63. The keyword-to-type mapping is reasonable. Minor note: the "is " pattern (with trailing space) in the boolean row is intentional but fragile — consider noting it matches prefix patterns like "is_approved".

5. skills/uipath-human-in-the-loop/SKILL.md — Production readiness note in Step 6 (lines 290–294)

Severity: OK

Useful differentiation between the three paths. The note about appSystemName: null for new coded action apps is a real gotcha worth calling out.

What's Missing

  • No new test tasks for fallback behaviors. The PR adds significant new decision paths (user unsure → QuickForm default, blocker fallbacks, vague schema handling, partial confirmation) but no corresponding test tasks in tests/tasks/uipath-human-in-the-loop/. Existing tests cover the happy paths. (Medium — consistent with most skills not being fully test-compliant, but worth tracking.)
  • No routing from Step 3 table to the new Step 4b section. The QuickForm row in the Step 3 routing table (line 142) sends the agent to references/hitl-node-quickform.md for schema design, but never mentions the new "Step 4b — Schema Design Resilience" section. An agent following the QuickForm path may never encounter the resilience checks. Consider adding a note like: "Apply Step 4b checks during schema design" to the QuickForm routing row.

Area Ratings

Area Status Notes
Frontmatter OK No changes to frontmatter; existing frontmatter valid
E2E Tests Issue No new tasks for the added decision paths (Medium)
Skill Body Issue Step 4b naming collision with existing AppTask references (High)
References & Assets OK No reference files changed; all links resolve
Repo Hygiene OK CODEOWNERS present; no secrets; changes scoped to one skill

Issues for Manual Review

  • Verify that the fallback response templates (line 150–153) match the actual UX the agent should present — these are conversational scripts and domain accuracy can't be verified from the docs alone.
  • Confirm whether Step 4b / Step 4c numbering in the AppTask and Coded Action App references should also be updated for consistency if the new section is renamed.

Conclusion

Good additions that address real-world edge cases. One high-severity issue blocks approval: the Step 4b naming collision between the new schema resilience section and the existing AppTask step references. An agent following either path could navigate to the wrong section. Rename the new section to resolve the ambiguity, and consider adding a routing pointer from the Step 3 QuickForm row so the agent actually discovers the new resilience checks.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants