OTA-1927: Eval cluster update prompts

Add comprehensive MCP test scenarios to evaluation dataset for validating
OpenShift cluster update workflow AI responses. These scenarios establish
quality benchmarks for LLM outputs across different update phases.

Test Scenarios Added (conv_798-802):
- Precheck: Pre-upgrade validation and readiness assessment
  Comprehensive analysis of cluster health, available updates, and
  upgrade blockers before initiating updates

- Precheck-Specific: Targeted upgrade path validation
  Validates specific version availability and upgrade feasibility
  for planned update targets

- No-Updates: Cluster health assessment at latest version
  Health monitoring and operational status when no updates are
  available in current channel

- Progress: Real-time upgrade progress monitoring
  Tracks upgrade progress with component status, timeline analysis,
  and ETA calculations during active updates

- Troubleshoot: Upgrade failure diagnosis and remediation
  Root cause analysis and conservative troubleshooting guidance
  for failed or stuck upgrade scenarios

Each scenario includes:
- Complete analysis prompts with constraints and requirements
- Full ClusterVersion YAML data as attachments
- Full ClusterOperator YAML data as attachments
- Expected responses with Summary and TL;DR sections
- Real cluster data from production-like scenarios

These scenarios mirror the CONSOLE-5118 OLS integration workflow phases
and provide the evaluation baseline for cluster update AI assistance.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Fabricio Aguiar <fabricio.aguiar@gmail.com>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED

Author: fao89

eval/README.md

@@ -50,17 +50,64 @@ lightspeed-eval --system-config system.yaml --eval-data eval_data_short.yaml --o
 
 # Full evaluation (797 questions)
 lightspeed-eval --system-config system.yaml --eval-data eval_data.yaml --output-dir ./results
+
+# Cluster-updates evaluation (17 conversations, 18 test turns) - uses optimized config
+lightspeed-eval --system-config system_cluster_updates.yaml \
+                --eval-data eval_data_cluster_updates.yaml \
+                --output-dir ./results
+
+# Run specific cluster-updates test category (e.g., critical tests)
+lightspeed-eval --system-config system_cluster_updates.yaml \
+                --eval-data eval_data_cluster_updates.yaml \
+                --tags cluster-updates-critical \
+                --output-dir ./results
 ```
 
 ## What's Included
 
 ### Datasets
-- **`eval_data_short.yaml`**: 10 conversations
-- **`eval_data.yaml`**: 797 conversations
-
-### Configuration
-- **`system.yaml`**: Pre-configured for OLS at `localhost:8080`
-- **Default metrics**: answer correctness
+- **`eval_data_short.yaml`**: 10 conversations (quick smoke test)
+- **`eval_data.yaml`**: 797 general OpenShift knowledge questions (conv_001-797)
+- **`eval_data_cluster_updates.yaml`**: 17 cluster-updates test conversations (conv_798-814, 18 test turns)
+
+### Test Categories (by tag)
+- **cluster-updates-scenarios**: Comprehensive health assessment with extensive constraints (conv_798-802, 5 conversations)
+- **cluster-updates-critical**: Condition status interpretation - MUST pass 100% (conv_803)
+- **cluster-updates-format**: Output format compliance (Summary + TL;DR) (conv_804)
+- **cluster-updates-blockers**: Admin-ack gates and upgrade blockers (conv_805)
+- **cluster-updates-risks**: Conditional update risk analysis (conv_806)
+- **cluster-updates-path**: Upgrade path validation (conv_807)
+- **cluster-updates-troubleshoot**: Failure diagnosis with error extraction (conv_808)
+- **cluster-updates-conversation**: Multi-turn conversations (conv_809, 2 turns)
+- **cluster-updates-no-updates**: Handling clusters at latest version (conv_810)
+- **cluster-updates-channels**: Update channel understanding (conv_811)
+- **cluster-updates-mcp**: MachineConfigPool upgrade behavior (conv_812)
+- **cluster-updates-pdb**: PodDisruptionBudget impact on upgrades (conv_813)
+- **cluster-updates-eus**: EUS (Extended Update Support) upgrades (conv_814)
+
+### Configuration Files
+
+Two configuration files are available depending on your use case:
+
+#### `system.yaml` - Default Configuration
+- **Use for:** General OpenShift knowledge evaluation (conv_001-797)
+- **API Base:** `http://localhost:8080` (local development)
+- **Max Tokens:** 512 (standard responses)
+- **API Provider:** `openai`
+- **Metrics:** All standard metrics available (Ragas, DeepEval, custom)
+
+#### `system_cluster_updates.yaml` - Cluster-Updates Optimized
+- **Use for:** Cluster-updates evaluation (conv_798-814)
+- **API Base:** `https://localhost:8080` (same as default)
+- **Max Tokens:** 2048 (detailed cluster analysis - 4x larger for complex responses)
+- **API Provider:** `myOpenai` (cluster-specific configuration)
+- **Output Directory:** `./results` (organized test output)
+- **Available Metrics:**
+  - `custom:answer_correctness` - Basic correctness evaluation
+  - `geval:condition_status_accuracy` - Kubernetes condition interpretation (threshold: 1.0 - CRITICAL!)
+  - `geval:output_format_compliance` - Response format validation (threshold: 0.95)
+  - `geval:technical_accuracy` - OpenShift/Kubernetes domain knowledge (threshold: 0.85)
+  - `geval:actionable_guidance` - Specific remediation steps (threshold: 0.7)
 
 
 ## Results
@@ -72,4 +119,4 @@ Results are saved in output directories:
 
 
 ## Data & Eval system setup
-Refer [Lightspeed Evaluation tool](https://github.com/lightspeed-core/lightspeed-evaluation#readme)
+Refer [Lightspeed Evaluation tool](https://github.com/lightspeed-core/lightspeed-evaluation#readme)