feat: rename llamaindex template input field to question with description

- rename input field `query` to `question` and add a pydantic Field
  description so the FE renders a helpful hint
- run.sh in testcases exercises each llm provider by uncommenting
  them one at a time

Author: radu-mocanu

packages/uipath-llamaindex/template/README.md

@@ -0,0 +1,73 @@
+# UiPath LlamaIndex Template Agent
+
+A quickstart UiPath LlamaIndex agent. It answers user queries using live tools and supports multiple LLM providers.
+
+> **Docs:** [uipath-llamaindex quick start](https://uipath.github.io/uipath-python/llamaindex/quick_start/) — **Samples:** [uipath-llamaindex/samples](https://github.com/UiPath/uipath-integrations-python/tree/main/packages/uipath-llamaindex/samples)
+
+## What it does
+
+1. **Prepares** the conversation — injects a system prompt and the user question into workflow context
+2. **Runs a ReAct agent step** that autonomously decides which tools to call and in what order
+3. **Postprocesses** — validates and truncates the response if it exceeds the configured max length
+
+### Tools
+
+| Tool               | Description                                      |
+| ------------------ | ------------------------------------------------ |
+| `get_current_time` | Returns the current UTC date and time (ISO 8601) |
+| `get_weather`      | Returns weather data for a city (mock data)      |
+
+### LLM Providers
+
+The template defaults to **Claude Haiku 4.5** via `UiPathChatBedrockConverse`. To switch providers, edit `main.py`:
+
+```python
+# Choose your LLM provider by uncommenting one of the following:
+llm = UiPathChatBedrockConverse(model=BedrockModel.anthropic_claude_haiku_4_5)
+# llm = UiPathOpenAI(model=OpenAIModel.GPT_4_1_MINI_2025_04_14.value)
+# llm = UiPathVertex(model=GeminiModel.gemini_2_5_flash)
+```
+
+## Workflow
+
+```mermaid
+flowchart TD
+    START --> prepare
+    prepare --> react_agent
+    react_agent -->|tool calls| tool_executor
+    tool_executor --> react_agent
+    react_agent -->|final| postprocess
+    postprocess --> END
+```
+
+## Input / Output
+
+```json
+// Input
+{
+  "question": "What's the weather like in London?"
+}
+
+// Output
+{
+  "response": "..."
+}
+```
+
+## Running locally
+
+```bash
+# Run
+uv run uipath run agent --input-file input.json --output-file output.json
+
+# Debug with dynamic node breakpoints
+uv run uipath debug agent --input-file input.json --output-file output.json
+```
+
+## Evaluation
+
+The agent ships with a tool call order evaluator that verifies the ReAct step calls `get_current_time` **before** `get_weather` when given a time-and-weather query, and an LLM judge that checks weather output for semantic similarity.
+
+```bash
+uv run uipath eval
+```

packages/uipath-llamaindex/template/entry-points.json

@@ -4,18 +4,19 @@
     "entryPoints": [
         {
             "filePath": "agent",
-            "uniqueId": "d64050f7-add5-4197-91f2-7b9cf3187751",
+            "uniqueId": "9016cb4a-25b4-44d3-8ace-08c3fea5316e",
             "type": "agent",
             "input": {
                 "type": "object",
                 "properties": {
-                    "query": {
-                        "title": "Query",
+                    "question": {
+                        "description": "Question for the assistant, e.g. 'What's the weather in Paris?'",
+                        "title": "Question",
                         "type": "string"
                     }
                 },
                 "required": [
-                    "query"
+                    "question"
                 ]
             },
             "output": {

packages/uipath-llamaindex/template/evaluations/eval-sets/evaluation-set-default.json

@@ -13,7 +13,7 @@
       "id": "ada5a2c1-976c-470b-964f-eb70a5e61eb4",
       "name": "Weather in Paris",
       "inputs": {
-        "query": "Is it good weather for a walk in Paris?"
+        "question": "Is it good weather for a walk in Paris?"
       },
       "evaluationCriterias": {
         "evaluator-llm-judge-output": {

packages/uipath-llamaindex/template/input.json

@@ -1,3 +1,3 @@
 {
-  "query": "What's the weather like in London?"
+  "question": "What's the weather like in London?"
 }

packages/uipath-llamaindex/template/main.py

@@ -10,6 +10,7 @@
     Workflow,
     step,
 )
+from pydantic import Field
 
 from uipath_llamaindex.llms import BedrockModel, GeminiModel, OpenAIModel, UiPathOpenAI
 from uipath_llamaindex.llms.bedrock import UiPathChatBedrockConverse
@@ -63,7 +64,9 @@ def get_weather(city: str, utc_time: str) -> str:
 
 
 class QueryEvent(StartEvent):
-    query: str
+    question: str = Field(
+        description="Question for the assistant, e.g. 'What's the weather in Paris?'"
+    )
 
 
 class LLMInputEvent(Event):
@@ -87,7 +90,7 @@ class TemplateAgent(Workflow):
     async def prepare(self, ctx: Context, ev: QueryEvent) -> LLMInputEvent:
         await ctx.store.set("messages", [
             ChatMessage(role="system", content=SYSTEM_PROMPT),
-            ChatMessage(role="user", content=ev.query),
+            ChatMessage(role="user", content=ev.question),
         ])
         return LLMInputEvent()
 

packages/uipath-llamaindex/testcases/template-agent/run.sh

@@ -19,49 +19,26 @@ uv run uipath auth --client-id="$CLIENT_ID" --client-secret="$CLIENT_SECRET" --b
 echo "Initializing the project..."
 uv run uipath init
 
-run_agent() {
-    local extra_args="$1"
-    if uv run uipath run agent --file input.json $extra_args 2>&1; then
-        return 0
-    else
-        if uv run uipath run agent --file input.json $extra_args 2>&1 | grep -q "timed out"; then
-            return 1
-        fi
-        # non-timeout error, fail immediately
-        return 2
-    fi
-}
+# Test each LLM provider from main.py by uncommenting them one at a time.
+PROVIDERS=(
+    "UiPathChatBedrockConverse"
+    "UiPathOpenAI"
+    "UiPathVertex"
+)
+
+for provider in "${PROVIDERS[@]}"; do
+    echo "===== Testing provider: $provider ====="
+    sed -i -E 's/^llm = /# llm = /' main.py
+    sed -i -E "s|^# llm = ${provider}\(|llm = ${provider}(|" main.py
 
-try_with_fallback() {
-    local extra_args="$1"
-    echo "Running agent with Bedrock provider..."
-    if uv run uipath run agent --file input.json $extra_args; then
-        return 0
-    fi
-
-    echo "⚠ Bedrock provider timed out or failed, switching to OpenAI fallback..."
-    sed -i 's/^llm = UiPathChatBedrockConverse.*/# &/' main.py
-    sed -i 's/^# llm = UiPathOpenAI/llm = UiPathOpenAI/' main.py
     uv run uipath init
 
-    if uv run uipath run agent --file input.json $extra_args; then
-        return 0
-    fi
-
-    echo "⚠ OpenAI provider also failed, trying Vertex fallback..."
-    sed -i 's/^llm = UiPathOpenAI.*/# &/' main.py
-    sed -i 's/^# llm = UiPathVertex/llm = UiPathVertex/' main.py
-    uv run uipath init
-
-    uv run uipath run agent --file input.json $extra_args
-}
-
-echo "Running agent..."
-try_with_fallback ""
+    echo "Running agent with $provider..."
+    uv run uipath run agent --input-file input.json --output-file output.json
 
-echo "Running agent again with empty UIPATH_JOB_KEY..."
-export UIPATH_JOB_KEY=""
-try_with_fallback "--trace-file .uipath/traces.jsonl" >> local_run_output.log
+    echo "Running agent again with empty UIPATH_JOB_KEY ($provider)..."
+    UIPATH_JOB_KEY="" uv run uipath run agent --trace-file .uipath/traces.jsonl --input-file input.json --output-file output.json >> local_run_output.log
 
-echo "Running evaluation..."
-uv run uipath eval --no-report --output-file eval_output.json
+    echo "Running evaluation with $provider..."
+    uv run uipath eval --no-report --output-file "eval_output_${provider}.json"
+done