Merge pull request #201 from microsoft/v2

feat: introduce v2

Author: Roopan-Microsoft

src/processor/.dockerignore .dockerignoresrc/processor/.dockerignore renamed to .dockerignore

@@ -1,3 +1,37 @@
+# Include any files or directories that you don't want to be copied to your
+# container here (e.g., local build artifacts, temporary files, etc.).
+#
+# For more help, visit the .dockerignore file reference guide at
+# https://docs.docker.com/engine/reference/builder/#dockerignore-file
+
+**/.DS_Store
+**/__pycache__
+**/.venv
+**/.classpath
+**/.dockerignore
+**/.env
+**/.git
+**/.gitignore
+**/.project
+**/.settings
+**/.toolstarget
+**/.vs
+**/.vscode
+**/*.*proj.user
+**/*.dbmdl
+**/*.jfm
+**/bin
+**/charts
+**/docker-compose*
+**/compose*
+**/Dockerfile*
+**/node_modules
+**/npm-debug.log
+**/obj
+**/secrets.dev.yaml
+**/values.dev.yaml
+LICENSE
+README.md
 # UV and Python cache directories
 **/__pycache__/
 **/*.py[cod]

.flake8

@@ -2,4 +2,4 @@
 max-line-length = 88
 extend-ignore = E501
 exclude = .venv, frontend
-ignore = E722,E203, W503, G004, G200, F,E711
+ignore = E722,E203, W503, G004, G200, F,E711,E704

.github/workflows/docker-build-and-push.yml

@@ -71,7 +71,7 @@ jobs:
           DATE="${{ steps.date.outputs.date }}"
           GITHUB_RUN_NUMBER="${{ github.run_number }}"
           if [[ "$BRANCH" == "main" ]]; then
-            BASE_TAG="latest"
+            BASE_TAG="latest_v2"
           elif [[ "$BRANCH" == "dev" ]]; then
             BASE_TAG="dev"
           elif [[ "$BRANCH" == "demo" ]]; then

.github/workflows/job-deploy.yml

@@ -427,17 +427,17 @@ jobs:
             
             # Determine image tag based on branch
             if [[ "$BRANCH_NAME" == "main" ]]; then
-              IMAGE_TAG="latest"
-              echo "Using main branch - image tag: latest"
+              IMAGE_TAG="latest_v2"
+              echo "Using main branch - image tag: latest_v2"
             elif [[ "$BRANCH_NAME" == "dev" ]]; then
               IMAGE_TAG="dev"
               echo "Using dev branch - image tag: dev"
             elif [[ "$BRANCH_NAME" == "demo" ]]; then
               IMAGE_TAG="demo"
               echo "Using demo branch - image tag: demo"
             else
-              IMAGE_TAG="latest"
-              echo "Using default for branch '$BRANCH_NAME' - image tag: latest"
+              IMAGE_TAG="latest_v2"
+              echo "Using default for branch '$BRANCH_NAME' - image tag: latest_v2"
             fi
             
             echo "Using existing Docker image tag: $IMAGE_TAG"

.vscode/settings.json

@@ -0,0 +1,9 @@
+{
+    "chat.tools.terminal.autoApprove": {
+        "/^cd H:\\\\Works\\\\Code-Migration\\\\Container-Migration-Solution-Accelerator\\\\src\\\\backend-api ; python -m ruff check src/ --fix 2>&1$/": {
+            "approve": true,
+            "matchCommandLine": true
+        },
+        "npx eslint": true
+    }
+}

README.md

@@ -2,7 +2,7 @@
 
 ## What is the Container Migration Solution Accelerator?
 
-This solution accelerator is an open-source GitHub Repository designed to streamline the migration of Kubernetes workloads from various container platforms (EKS, GKE, etc.) to Azure Kubernetes Service (AKS). It automates the analysis, design, configuration transformation, and documentation generation processes to enhance the speed and accuracy of container platform migrations. The solution is built using Azure OpenAI Service, Semantic Kernel Process Framework, Model Context Protocol (MCP) servers, and Azure services integration.
+This solution accelerator is an open-source GitHub Repository designed to streamline the migration of Kubernetes workloads from various container platforms (EKS, GKE, etc.) to Azure Kubernetes Service (AKS). It automates the analysis, design, configuration transformation, and documentation generation processes to enhance the speed and accuracy of container platform migrations. The solution is built using Azure OpenAI Service, Microsoft Agent Framework (workflow + group chat orchestration), Model Context Protocol (MCP) servers, and Azure services integration.
 
 ## What can the Container Migration Solution Accelerator do?
 
@@ -40,7 +40,7 @@ The solution was evaluated using multiple approaches:
 
 3. **Migration Coverage**: Testing across different source platforms (EKS, GKE) and various workload types (stateless applications, databases, monitoring systems, etc.).
 
-4. **Process Framework Reliability**: Evaluation of the Semantic Kernel Process Framework's error handling, retry mechanisms, and failure recovery capabilities.
+4. **Process Framework Reliability**: Evaluation of the Agent Framework workflow's error handling, retry mechanisms, and failure recovery capabilities.
 
 5. **Documentation Quality**: Assessment of generated documentation for completeness, accuracy, and actionability.
 

TRANSPARENCY_FAQ.md

@@ -1,110 +1,84 @@
 # Agentic Architecture - Container Migration Solution Accelerator
 
-Based on your actual implementation, here's the comprehensive agentic architecture that mirrors the style of your reference image:
+High-level view of how the 4-step orchestration works (executors, group chat orchestrators, and tools).
 
 ## Architecture Overview
 
 ```mermaid
-graph TB
-    subgraph "Entry Layer"
-        WEB[Web App/Queue]
-        SERVICE[Migration Service]
+flowchart LR
+    %% Top-level orchestration + telemetry
+    TELEM["Agent & Process Status<br/>(telemetry)"]
+    COSMOS[("Cosmos DB<br/>telemetry/state")]
+    PROC["Process Orchestration<br/>Agent Framework WorkflowBuilder"]
+
+    TELEM --> COSMOS
+    PROC --- TELEM
+
+    %% Step lanes
+    subgraph S1["Step 1: Analysis"]
+        direction TB
+        S1EXEC["Analysis Executor"] --> S1ORCH["Analysis Chat Orchestrator<br/>(GroupChatOrchestrator)"] --> S1AGENTS["Agents:<br/>Chief Architect<br/>AKS Expert<br/>Platform experts (EKS/GKE/OpenShift/Rancher/Tanzu/OnPremK8s)"]
     end
 
-    subgraph "Process Engine"
-        PROC[Process Orchestrator<br/>Semantic Kernel]
+    subgraph S2["Step 2: Design"]
+        direction TB
+        S2EXEC["Design Executor"] --> S2ORCH["Design Chat Orchestrator<br/>(GroupChatOrchestrator)"] --> S2AGENTS["Agents:<br/>Chief Architect<br/>AKS Expert<br/>Platform experts (EKS/GKE/OpenShift/Rancher/Tanzu/OnPremK8s)"]
     end
 
-    subgraph "Migration Steps"
-        ANALYSIS[Analysis Step<br/>Platform Discovery]
-        DESIGN[Design Step<br/>Azure Architecture]
-        YAML[YAML Step<br/>Configuration Transform]
-        DOCS[Documentation Step<br/>Report Generation]
+    subgraph S3["Step 3: YAML Conversion"]
+        direction TB
+        S3EXEC["Convert Executor"] --> S3ORCH["YAML Chat Orchestrator<br/>(GroupChatOrchestrator)"] --> S3AGENTS["Agents:<br/>YAML Expert<br/>Azure Architect<br/>AKS Expert<br/>QA Engineer<br/>Chief Architect"]
     end
 
-    subgraph "AI Agents (7 Specialists)"
-        AGENTS[Multi-Agent System<br/>• Technical Architect<br/>• Azure Expert<br/>• EKS/GKE Experts<br/>• QA Engineer<br/>• Technical Writer<br/>• YAML Expert]
+    subgraph S4["Step 4: Documentation"]
+        direction TB
+        S4EXEC["Documentation Executor"] --> S4ORCH["Documentation Chat Orchestrator<br/>(GroupChatOrchestrator)"] --> S4AGENTS["Agents:<br/>Technical Writer<br/>Azure Architect<br/>AKS Expert<br/>Chief Architect<br/>Platform experts (EKS/GKE/OpenShift/Rancher/Tanzu/OnPremK8s)"]
     end
 
-    subgraph "Tool Layer"
-        MCP[MCP Servers<br/>• Blob Storage<br/>• Microsoft Docs<br/>• DateTime Utils]
-    end
-
-    subgraph "Storage Layer"
-        STORAGE[Azure Services<br/>• Blob Storage<br/>• Cosmos DB<br/>• OpenAI GPT o3]
-    end
+    PROC --> S1
+    S1 -->|Analysis Result| S2
+    S2 -->|Design Result| S3
+    S3 -->|YAML Converting Result| S4
 
-    %% Main Flow
-    WEB --> SERVICE
-    SERVICE --> PROC
-    PROC --> ANALYSIS
-    ANALYSIS --> DESIGN
-    DESIGN --> YAML
-    YAML --> DOCS
-
-    %% AI Integration
-    ANALYSIS -.-> AGENTS
-    DESIGN -.-> AGENTS
-    YAML -.-> AGENTS
-    DOCS -.-> AGENTS
-
-    %% Tool Access
-    AGENTS -.-> MCP
-    MCP -.-> STORAGE
-
-    %% Styling for better readability
-    classDef entryLayer fill:#e3f2fd,stroke:#1976d2,stroke-width:3px,color:#000
-    classDef processLayer fill:#fff3e0,stroke:#f57c00,stroke-width:3px,color:#000
-    classDef stepLayer fill:#f3e5f5,stroke:#7b1fa2,stroke-width:3px,color:#000
-    classDef agentLayer fill:#e8f5e8,stroke:#388e3c,stroke-width:3px,color:#000
-    classDef toolLayer fill:#fce4ec,stroke:#c2185b,stroke-width:3px,color:#000
-    classDef storageLayer fill:#e1f5fe,stroke:#0288d1,stroke-width:3px,color:#000
-
-    class WEB,SERVICE entryLayer
-    class PROC processLayer
-    class ANALYSIS,DESIGN,YAML,DOCS stepLayer
-    class AGENTS agentLayer
-    class MCP toolLayer
-    class STORAGE storageLayer
 ```
 
 ## Agent Specialization by Phase
 
 ### Analysis Phase Agents
 
-- **Technical Architect**: Leads overall analysis strategy and coordination
-- **EKS Expert**: Identifies AWS EKS-specific patterns and configurations
-- **GKE Expert**: Identifies Google GKE-specific patterns and configurations
+- **Chief Architect**: Leads overall analysis strategy and coordination
+- **AKS Expert**: Reviews for AKS/Azure migration readiness
+- **Platform experts**: Registry-loaded participants (EKS/GKE/OpenShift/Rancher/Tanzu/OnPremK8s); coordinator keeps non-matching experts quiet
 
 ### Design Phase Agents
 
-- **Technical Architect**: Defines migration architecture patterns
-- **Azure Expert**: Designs Azure service mappings and optimizations
-- **EKS Expert**: Provides source platform context for AWS workloads
-- **GKE Expert**: Provides source platform context for GCP workloads
+- **Chief Architect**: Defines migration architecture patterns and reconciles trade-offs
+- **AKS Expert**: Ensures AKS-specific conventions and constraints are applied
+- **Platform experts**: Provide source-platform context and constraints for the detected platform
 
 ### YAML Conversion Phase Agents
 
 - **YAML Expert**: Performs configuration transformations and syntax optimization
-- **Azure Expert**: Ensures Azure service integration and compliance
+- **Azure Architect**: Ensures Azure service integration and compliance
+- **AKS Expert**: Ensures converted manifests align with AKS expectations
 - **QA Engineer**: Validates converted configurations and tests
-- **Technical Writer**: Documents conversion decisions and generates reports
+- **Chief Architect**: Provides overall review and integration
 
 ### Documentation Phase Agents
 
-- **Technical Architect**: Provides architectural documentation and migration summary
-- **Azure Expert**: Documents Azure-specific configurations and optimizations
-- **EKS/GKE Experts**: Document source platform analysis and transformation logic
-- **QA Engineer**: Provides validation reports and testing documentation
 - **Technical Writer**: Creates comprehensive migration documentation
+- **Azure Architect**: Documents Azure-specific configurations and optimizations
+- **AKS Expert**: Documents AKS-focused implementation guidance and caveats
+- **Chief Architect**: Provides architectural documentation and migration summary
+- **Platform experts**: Document source platform analysis and transformation logic
 
 ## Data Flow Architecture
 
 ### Input Processing
 
-1. **Queue Service** receives migration requests from web app or direct API
-2. **Migration Service** processes queue messages and initiates migration process
-3. **Process Orchestrator** manages step-by-step execution with event routing
+1. **Web app** creates a migration request
+2. **Queue worker service** receives the migration request from **Azure Storage Queue**
+3. **Migration Processor** runs the end-to-end workflow (analysis → design → yaml → documentation)
 
 ### Step Execution Pattern
 
@@ -121,11 +95,13 @@ Each step follows this pattern:
 
 ### MCP Server Integration
 
-All agents have access to Model Context Protocol (MCP) servers via Semantic Kernel plugin:
+All agents have access to Model Context Protocol (MCP) servers via Microsoft Agent Framework tool abstractions:
 
 - **Blob Operations**: File reading/writing to Azure Blob Storage
 - **Microsoft Docs**: Azure documentation lookup and best practices
 - **DateTime Utilities**: Timestamp generation and time-based operations
+- **Fetch**: URL fetching for validation (e.g., verifying references)
+- **YAML Inventory**: Enumerate converted YAML objects for runbooks
 
 ## Key Architectural Principles
 
@@ -140,22 +116,27 @@ Each step has a focused objective:
 
 ### Event-Driven Orchestration
 
-Steps communicate through Semantic Kernel events:
-
-- `StartMigration` → Analysis Step
-- `AnalysisCompleted` → Design Step
-- `DesignCompleted` → YAML Step
-- `YamlCompleted` → Documentation Step
+Steps are executed as a directed workflow (with start node and edges) using the Agent Framework workflow engine.
+The processor emits workflow/executor events for observability and telemetry.
 
 ### Multi-Agent Collaboration
 
-Within each step, specialized agents collaborate through GroupChat orchestration:
+Within each step, specialized agents collaborate through group chat orchestration:
 
 - Structured conversation patterns
 - Domain expertise contribution
 - Consensus building on decisions
 - Quality validation and review
 
+### Evaluation and Quality Checks
+
+The processor uses multiple quality signals to reduce regressions and increase reliability:
+
+- **Typed step outputs**: workflow executors and orchestrators exchange typed models per step (analysis → design → yaml → documentation).
+- **QA sign-offs**: the QA agent focuses on validation steps and flags missing/unsafe transformations.
+- **Tool-backed validation**: steps can call validation tools via MCP (e.g., Mermaid validation, YAML inventory grounding, docs lookups).
+- **Unit tests**: processor unit tests live under [src/processor/src/tests/unit/](../src/processor/src/tests/unit/).
+
 ### Tool-Enabled Intelligence
 
 Agents access external capabilities through MCP servers:
@@ -176,41 +157,27 @@ Comprehensive tracking throughout the process:
 ## File Location Mapping
 
 ```text
-src/
-├── main_service.py                    # Queue Service Entry Point
-├── services/migration_service.py      # Migration Orchestration
-├── libs/processes/
-│   └── aks_migration_process.py       # Process Framework Definition
-├── libs/steps/
-│   ├── analysis_step.py               # Analysis Step Implementation
-│   ├── design_step.py                 # Design Step Implementation
-│   ├── yaml_step.py                   # YAML Step Implementation
-│   └── documentation_step.py          # Documentation Step Implementation
-├── libs/steps/orchestration/
-│   ├── analysis_orchestration.py      # Analysis Agent Orchestration
-│   ├── design_orchestration.py        # Design Agent Orchestration
-│   ├── yaml_orchestration.py          # YAML Agent Orchestration
-│   └── documentation_orchestration.py # Documentation Agent Orchestration
-├── agents/
-│   ├── technical_architect/agent_info.py
-│   ├── azure_expert/agent_info.py
-│   ├── eks_expert/agent_info.py
-│   ├── gke_expert/agent_info.py
-│   ├── qa_engineer/agent_info.py
-│   ├── technical_writer/agent_info.py
-│   └── yaml_expert/agent_info.py
-└── plugins/mcp_server/
-    ├── MCPBlobIOPlugin.py             # Azure Blob Storage MCP Server
-    ├── MCPMicrosoftDocs.py            # Microsoft Docs MCP Server
-    └── MCPDatetimePlugin.py           # DateTime Utilities MCP Server
+src/processor/src/
+├── main_service.py                             # Queue worker entry point
+├── services/queue_service.py                   # Azure Storage Queue consumer
+├── services/control_api.py                     # Control API (health/kill)
+├── services/process_control.py                 # Process control store/manager
+├── steps/migration_processor.py                # WorkflowBuilder + step chaining
+├── steps/analysis/workflow/analysis_executor.py
+├── steps/design/workflow/design_executor.py
+├── steps/convert/workflow/yaml_convert_executor.py
+└── steps/documentation/
+    ├── orchestration/documentation_orchestrator.py
+    ├── workflow/documentation_executor.py
+    └── agents/                                  # Agent prompt files
 ```
 
 ## Summary
 
 This architecture implements a sophisticated agentic system that combines:
 
-- **Semantic Kernel Process Framework** for structured workflow execution
-- **Multi-Agent GroupChat Orchestration** for domain expertise collaboration
+- **Microsoft Agent Framework Workflow** for structured workflow execution
+- **Multi-Agent Group Chat Orchestration** for domain expertise collaboration
 - **Model Context Protocol (MCP)** for tool integration and external system access
 - **Azure Cloud Services** for scalable storage and data management
 - **Event-Driven Architecture** for loose coupling and reliability