Skip to content

Bonus/trend analysis#8

Merged
Ayush04-C merged 4 commits into
mainfrom
bonus/trend-analysis
Jul 4, 2026
Merged

Bonus/trend analysis#8
Ayush04-C merged 4 commits into
mainfrom
bonus/trend-analysis

Conversation

@Ayush04-C

Copy link
Copy Markdown
Owner

Bonus Feature: Historical Trend Analysis & Regression Detection

Overview

This Pull Request implements the Trend Analysis bonus challenge for the JioPC Automated Testing Agent. It introduces a persistent historical state mechanism that compares current validation runs against previous runs to intelligently surface application regressions.

By tracking state across executions, the agent shifts from simply stating "What is broken?" to explicitly answering "What broke since the last patch?"

Implementation Details

  • trend_analyser.py Module: Created a robust, standalone module that manages a rolling history of the last 5 test runs in ~/.local/share/jiopc/agent/history.json.
  • Intelligent Classification: Automatically categorizes state changes between the current run and the immediate predecessor:
    • 🔴 REGRESSIONS: Tests that transitioned from PASS to FAIL (Immediate action required).
    • 🟢 RECOVERIES: Tests that transitioned from failing states to PASS (Validation of fixes).
    • 🆕 NEW FAILURES: Failing tests that were not present in the previous run.
    • ⚠️ CONSISTENT FAILURES: Tests failing persistently for 3 or more consecutive runs.
  • Bot Protection Awareness: Explicitly designed to ignore BLOCKED states (Jio Cloudflare challenges) in regression calculations, maintaining compliance with the hackathon's headless validation constraints.
  • Native Integration: Directly hooked into the core jiopc_agent.py orchestrator. Trend analysis runs automatically and prints to stdout whenever the --analyse flag is passed.
  • CI/CD Propagation: Updated the GitHub Actions workflow (ci.yml) to automatically append the Trend Analysis report to the automated PR comment, giving reviewers immediate regression visibility.

Persistence & Architecture

The history file is stored locally in the agent's application data directory (~/.local/share/jiopc/agent/). Because JioPC relies on NFS-mounted user home directories, this file will safely persist across VM reassignments, making it perfectly suited for JioPC's floating VM infrastructure. All file writes are executed atomically to prevent JSON corruption during unexpected termination.

Testing Performed

  • Initialized agent to successfully establish a baseline history file.
  • Simulated a failure (changed URL to force a 404) to successfully trigger a 🔴 REGRESSION alert.
  • Reverted the simulated failure to successfully trigger a 🟢 RECOVERY alert.
  • Verified that Pathlib PosixPath JSON serialization errors have been patched.
  • Packaged, built, and executed via the system .deb binary to ensure compatibility.

Recommendation

Merge to main. This completes the final bonus requirement for the hackathon, providing judges with a comprehensive, state-aware agentic testing framework.

@Ayush04-C
Ayush04-C merged commit b4db3c7 into main Jul 4, 2026
1 check passed
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

✅ JioPC Testing Agent — PASS

Metric Count
Total Tests 8
✅ Passed 3
❌ Failed 0
🔒 Blocked (bot protection) 5

Note: BLOCKED results are expected for Jio URLs behind
Cloudflare protection and do not indicate failures.


🤖 LLM Analysis

INFO: HTTP Request: POST https://bharatcode.ai/api/model/v1/chat/completions "HTTP/1.1 200 OK"
Analysing log with bharatcode:qwen36-35b-q6-256k-vision...

============================================================
LLM ANALYSIS RESULT
============================================================


## EXECUTIVE SUMMARY
The automated testing agent executed a total of 8 tests across the OS image, resulting in 3 passes and 5 blocked responses, with zero failures, degradations, or missing/misplaced components. Based on these results, the OS Image patch is safe to promote to production, as all core functionality is stable and the blocked responses align with expected security behaviors rather than system defects.

## FAILURES AND ANOMALIES
**Component A (Web App URL Reachability & UI)**
Expected Bot Protection:
- JioSaavn: Blocked by bot detection page.
- JioHotstar: Blocked by bot detection page.
- JioCloud: Blocked by bot detection page.
- YouTube: Blocked by bot detection page.
- GitHub: Blocked by bot detection page.
No failures detected.

**Component B (Native App Launch & Process)**
No failures detected.

**Component C (Desktop Folder & Start Menu)**
No failures detected.

## PATTERNS AND CORRELATIONS
All blocked results in Component A returned the identical "Bot detection page detected" detail, indicating a consistent infrastructure-level response rather than isolated application failures. No other cross-cutting patterns, timing anomalies, or multi-component correlations were detected.

## RECOMMENDATION
PROMOTE
The patch is safe to promote as there are zero failures, no degraded or misplaced components, and all blocked results represent expected bot protection behavior rather than system defects.

Triggered by commit ccf00f9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant