diff --git a/architecture/Workflow.drawio.png b/architecture/Workflow.drawio.png new file mode 100644 index 0000000..67eb159 Binary files /dev/null and b/architecture/Workflow.drawio.png differ diff --git a/architecture/architecture.drawio.png b/architecture/architecture.drawio.png new file mode 100644 index 0000000..e93ad2b Binary files /dev/null and b/architecture/architecture.drawio.png differ diff --git a/architecture/architecture.md b/architecture/architecture.md new file mode 100644 index 0000000..473ed0f --- /dev/null +++ b/architecture/architecture.md @@ -0,0 +1,293 @@ +# JioPC Testing Agent - System Architecture + +## 1. Overview + +The JioPC Testing Agent is a modular validation framework designed to automate quality assurance for JioPC operating system images. Instead of relying on manual verification after every image update, the agent executes a configurable suite of validation tests covering web applications, native desktop applications, and desktop launcher integrity. + +The entire system is configuration-driven through a YAML file, allowing new applications or validation rules to be added without modifying the source code. Every execution generates a structured JSON Lines (JSONL) log that serves as the single source of truth for analysis, historical tracking, and automated reporting. + +The architecture follows a modular pipeline where each validation component is isolated, making the system easy to maintain, test, and extend. + +--- + +# 2. High-Level Architecture + +The system consists of six primary layers: + +``` + User / CI Pipeline + │ + ▼ + Runner (CLI Entry) + │ + ┌───────────────┼────────────────┐ + ▼ ▼ ▼ + Web Tests Native Tests Desktop Audit + │ │ │ + └───────────────┼────────────────┘ + ▼ + Shared JSON Logger + │ + ┌─────────────┴─────────────┐ + ▼ ▼ + Trend Analysis LLM Analysis + │ │ + └─────────────┬─────────────┘ + ▼ + Email Notification + │ + ▼ + Final Validation Report +``` + +--- + +# 3. Architectural Principles + +The project was designed around several guiding principles. + +### Modular Components + +Each validation domain is implemented independently. + +- Part A validates web applications. +- Part B validates native applications. +- Part C validates desktop launcher integrity. + +Each module exposes exactly one public entry point, allowing the Runner Core to orchestrate execution without knowing the implementation details. + +--- + +### Configuration Driven + +Application definitions are never hardcoded. + +Instead, every validation target is described inside a YAML configuration file. + +This allows QA engineers to: + +- add new applications +- change timeouts +- update selectors +- modify launch commands + +without editing Python source code. + +--- + +### Shared Logging Pipeline + +Rather than allowing every module to write its own logs, all components communicate through a shared Logger. + +This guarantees: + +- consistent log format +- unified timestamps +- centralized summaries +- easy downstream processing + +The generated JSONL file becomes the single source of truth for the remainder of the pipeline. + +--- + +### Post-Processing Layer + +Once testing finishes, additional processing modules consume the generated log. + +These modules include: + +- Trend Analyzer +- LLM Analyzer +- SMTP Mailer + +Since they operate on the log instead of the testing modules directly, new analysis features can be added without changing the validation code. + +--- + +# 4. Component Responsibilities + +## Runner Core + +Responsible for: + +- CLI argument parsing +- configuration loading +- module orchestration +- failure aggregation +- summary generation +- optional analysis execution + +The Runner never performs validation itself. Its responsibility is orchestration. + +--- + +## Part A — Web Validation + +Uses Playwright with a headless Chromium browser to validate: + +- page availability +- HTTP status +- page loading +- expected DOM elements +- bot-detection pages + +Results are recorded as: + +- PASS +- FAIL +- BLOCKED + +--- + +## Part B — Native Application Validation + +Launches native desktop applications and monitors their health using psutil. + +Validation includes: + +- executable availability +- successful startup +- unexpected crashes +- startup timeout +- graceful cleanup + +Results are recorded as: + +- PASS +- FAIL + +--- + +## Part C — Desktop Presence Validation + +Audits Linux desktop launcher metadata. + +Checks include: + +- launcher existence +- desktop file parsing +- application type +- category validation + +Results are recorded as: + +- PASS +- MISSING +- MISPLACED + +--- + +## Shared Logger + +Provides a unified logging interface used by every validation component. + +Responsibilities include: + +- writing JSONL records +- maintaining counters +- component statistics +- execution summary generation + +The logger is intentionally isolated so future modules can consume a standardized log format. + +--- + +## Trend Analyzer + +Compares the current execution against previous runs stored inside the history database. + +It detects: + +- regressions +- recoveries +- recurring failures +- execution trends + +This enables engineers to identify problems introduced by recent image changes. + +--- + +## LLM Analysis + +Reads the completed JSONL log together with a structured prompt. + +The language model converts raw validation output into: + +- human-readable summaries +- likely root causes +- deployment recommendation +- production readiness assessment + +--- + +## SMTP Mailer + +Distributes the generated validation report to configured recipients. + +Separating notification logic from validation logic keeps the testing pipeline independent of delivery mechanisms. + +--- + +# 5. Data Storage + +The agent stores all execution artifacts inside the user's local data directory. + +``` +~/.local/share/jiopc/agent/ +``` + +Typical contents include: + +- JSONL execution logs +- historical execution database +- generated reports +- archived results + +This satisfies the hackathon requirement that no system files are modified during execution. + +--- + +# 6. Configuration Architecture + +The YAML configuration file defines every validation target. + +Major sections include: + +- agent +- web_apps +- native_apps +- desktop_presence +- email + +This design keeps the validation engine generic while allowing project-specific behavior to remain inside configuration. + +--- + +# 7. Extensibility + +The architecture was intentionally designed for future expansion. + +New validation modules can be introduced by: + +1. implementing a new runner function +2. registering it inside the Runner dispatch table +3. extending the YAML configuration + +No existing validation modules require modification. + +This minimizes merge conflicts and encourages independent development. + +--- + +# 8. Design Decisions + +Several architectural decisions were made to improve maintainability. + +- Modular execution instead of one monolithic script. +- Configuration-driven validation instead of hardcoded logic. +- Shared logger instead of independent log files. +- JSONL storage for easy machine parsing. +- Post-processing architecture separating testing from reporting. +- Provider-independent LLM integration using OpenAI-compatible APIs. + +Together, these decisions produce a validation framework that is portable, maintainable, and suitable for continuous integration workflows. \ No newline at end of file diff --git a/architecture/workflow.md b/architecture/workflow.md new file mode 100644 index 0000000..1e30df1 --- /dev/null +++ b/architecture/workflow.md @@ -0,0 +1,305 @@ +# JioPC Testing Agent - Workflow + +## 1. Workflow Overview + +The JioPC Testing Agent follows a sequential execution pipeline beginning with user invocation and ending with automated reporting. Each stage of the workflow has a clearly defined responsibility and communicates with the next stage through structured data rather than direct coupling. + +The workflow is designed to ensure that every execution follows the same predictable lifecycle regardless of which validation modules are enabled. + +--- + +# 2. End-to-End Execution Flow + +``` +User / CI Pipeline + │ + ▼ +Execute jiopc-agent + │ + ▼ +Parse CLI Arguments + │ + ▼ +Load YAML Configuration + │ + ▼ +Initialize Logger + │ + ▼ +Determine Validation Parts + │ + ▼ +Execute Validation Modules +(A / B / C) + │ + ▼ +Collect Results + │ + ▼ +Write Execution Summary + │ + ▼ +(Optional) +Run Trend Analysis + │ + ▼ +(Optional) +Run LLM Analysis + │ + ▼ +(Optional) +Send Email Report + │ + ▼ +Exit +``` + +--- + +# 3. Detailed Workflow + +## Step 1 — User Invocation + +Execution begins when the user launches the testing agent from the terminal. + +Example: + +```bash +jiopc-agent --config jiopc-agent.yaml --analyse +``` + +The command-line interface allows the user to: + +- specify a custom configuration file +- execute only selected validation modules +- enable or disable LLM analysis + +At this point no validation has started. + +--- + +## Step 2 — Configuration Loading + +The Runner Core loads the YAML configuration file. + +The configuration defines: + +- agent settings +- web applications +- native applications +- desktop launchers +- notification settings + +If the configuration cannot be loaded or parsed, execution stops immediately. + +--- + +## Step 3 — Logger Initialization + +A shared Logger instance is created before any validation begins. + +The logger: + +- creates a new JSONL log file +- initializes execution counters +- prepares component statistics +- timestamps every future event + +All validation modules receive this same logger instance. + +--- + +## Step 4 — Validation Scheduling + +The Runner determines which validation modules should execute. + +This depends on: + +- the `parts_order` defined in the YAML configuration +- optional `--part` command-line arguments + +Each selected module is executed independently while sharing the same configuration and logger. + +--- + +## Step 5 — Web Application Validation (Part A) + +The Web Tester validates every configured web application. + +For each application it performs: + +1. Launch headless Chromium +2. Navigate to target URL +3. Wait for page load +4. Detect bot-protection pages +5. Verify required DOM elements +6. Record the result + +Possible outcomes include: + +- PASS +- FAIL +- BLOCKED + +Each result is immediately written to the JSONL log. + +--- + +## Step 6 — Native Application Validation (Part B) + +The Native App Health module validates installed desktop applications. + +For every configured application it: + +1. launches the executable +2. monitors process startup +3. waits for successful initialization +4. detects crashes or startup failures +5. terminates all spawned processes + +Possible outcomes include: + +- PASS +- FAIL + +Each result is recorded through the shared logger. + +--- + +## Step 7 — Desktop Presence Validation (Part C) + +The Desktop Auditor verifies Linux desktop integration. + +For every configured launcher it: + +1. searches standard application directories +2. locates the corresponding `.desktop` file +3. parses launcher metadata +4. validates application type +5. validates menu category + +Possible outcomes include: + +- PASS +- MISSING +- MISPLACED + +Results are appended to the execution log. + +--- + +## Step 8 — Summary Generation + +Once all validation modules finish execution, the Logger writes a final summary. + +The summary contains: + +- total validations +- overall pass count +- failure count +- blocked count +- missing launchers +- misplaced launchers +- component-wise statistics + +This marks the completion of the validation phase. + +--- + +## Step 9 — Trend Analysis (Optional) + +If enabled, the Trend Analyzer compares the current execution against historical validation data. + +It identifies: + +- newly introduced failures +- recovered applications +- recurring failures +- long-term execution trends + +This information provides additional context beyond the current execution. + +--- + +## Step 10 — LLM Analysis (Optional) + +When the `--analyse` flag is provided, the completed JSONL log is processed by the LLM Analysis module. + +The module: + +1. loads the prompt template +2. reads the execution log +3. injects the log into the prompt +4. calls the configured language model +5. generates a human-readable report + +The report summarizes: + +- detected issues +- likely causes +- deployment recommendation +- production readiness + +--- + +## Step 11 — Email Notification (Optional) + +If email notifications are configured, the generated report is delivered to the configured recipients. + +This enables automated reporting after every execution without requiring manual log inspection. + +--- + +## Step 12 — Program Termination + +The application exits after all requested stages have completed. + +Exit codes follow standard UNIX conventions: + +| Exit Code | Meaning | +|-----------|---------| +| 0 | Validation completed successfully | +| 1 | One or more validation failures occurred | + +--- + +# 4. Information Flow + +Throughout execution, information flows in one direction. + +``` +Configuration + │ + ▼ +Runner Core + │ + ▼ +Validation Modules + │ + ▼ +Shared Logger + │ + ▼ +JSONL Log + │ + ├────────► Trend Analysis + │ + ├────────► LLM Analysis + │ + └────────► Email Notification +``` + +This unidirectional flow minimizes coupling between modules and makes the system easier to maintain and extend. + +--- + +# 5. Workflow Characteristics + +The execution workflow was designed around several key principles. + +- **Sequential orchestration** ensures deterministic execution. +- **Independent validation modules** prevent failures in one component from affecting others. +- **Centralized logging** provides a single source of truth. +- **Configuration-driven execution** enables flexible test selection. +- **Optional post-processing** keeps validation independent from reporting. +- **Graceful error handling** allows execution to continue whenever possible, ensuring maximum diagnostic information is collected during each run. \ No newline at end of file