fix: downmerge from main to demo

[Avijit-Microsoft]

Purpose

This pull request makes significant improvements and standardizations to the project's GitHub Actions workflows and development container configuration. The main focus is on unifying and clarifying environment variable naming conventions, enhancing workflow scheduling, and improving Azure-related automation. Additionally, it updates the devcontainer to better support Bicep deployments.

Key changes include:

Workflow and Variable Naming Standardization

• Refactored environment variable names across multiple workflows (such as deploy-v2.yml, deploy-orchestrator.yml) to use clearer and more consistent names (e.g., USE_CASE, AZURE_ENV_EXISTING_LOG_ANALYTICS_WORKSPACE_RID, AZURE_EXISTING_AIPROJECT_RESOURCE_ID) instead of legacy names (e.g., AZURE_ENV_USE_CASE, AZURE_ENV_LOG_ANALYTICS_WORKSPACE_ID). This affects workflow inputs, job environments, and validation logic. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]

Workflow Enhancements and Scheduling

• Added scheduled triggers to workflows for regular automated validation and deployment (e.g., running at specific times or days), and improved workflow dispatch options for flexibility. [1] [2]
• Removed the outdated azure-dev-validation.yml workflow in favor of a new, more flexible azd-template-validation.yml with improved environment variable handling and output reporting. [1] [2]

Azure Deployment Automation

• Improved Azure deployment workflows (azure-dev.yml) by updating environment variable usage, switching to more secure secrets, and streamlining the provisioning and deployment steps. This includes dynamic naming of Azure environments and enhanced login procedures.

Devcontainer Improvements

• Updated the Azure CLI devcontainer feature to explicitly install Bicep and set both its version and the CLI version to latest, ensuring compatibility with Bicep-based infrastructure as code.

Minor Fixes and Cleanups

• Removed unused or unsupported Azure regions (e.g., francecentral) from deployment options.
• Fixed parameter usage in Bicep deployment scripts and removed unused contentUnderstandingLocation parameter.
• Improved workflow triggers to only run Bicep deployments on relevant file changes.

These changes collectively improve maintainability, clarity, and automation reliability for the project's CI/CD and developer tooling.

Does this introduce a breaking change?

• [x ] Yes
• No

Golden Path Validation

• [ x] I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• [ x] I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

Verify that the following are valid

• ...

Other Information

[github-actions]

Coverage Report •

File	Stmts	Miss	Cover	Missing
src/api/api
history_routes.py	256	39	84%	42, 159, 163, 241–243, 248–252, 268, 272, 400, 404, 407, 410, 430, 433–436, 441–445, 462, 466, 477, 481, 495–497, 503–507
src/api/common/config
config.py	26	0	100%
src/api/common/database
cosmosdb_service.py	107	2	98%	31, 177
sqldb_service.py	187	22	88%	18–26, 28–29, 62–63, 65–70, 195, 294, 334
src/api/helpers
azure_credential_utils.py	15	1	93%	52
src/api/services
chat_service.py	172	12	93%	59–60, 223–226, 230, 233, 244–245, 265–266
history_service.py	206	24	88%	93, 224–225, 227, 264–266, 282, 288–290, 307, 323–324, 326, 342, 368–369, 371, 387, 407–408, 410, 429
TOTAL	1314	152	88%

Tests	Skipped	Failures	Errors	Time
163	0 💤	0 ❌	0 🔥	5.847s ⏱️

[Copilot]

Pull request overview

This PR downmerges updates from main into demo, bringing substantial CI/CD, infra scripting, backend, and frontend changes aimed at standardizing environment variables, improving Azure automation (including Content Understanding and AI Foundry usage), and refactoring the frontend state management to Redux.

Changes:

• Refactors the frontend to Redux Toolkit, adds new shared utilities (HTTP client, message/chart parsing), and updates citation/chat streaming handling.
• Updates backend chat/history services and tests (notably streaming output now emits (role, content) deltas and credentials handling for async SDKs).
• Revises infra scripts/workflows/docs for new env var naming and Content Understanding GA API requirements, plus new scheduled validation workflows.

Reviewed changes

Copilot reviewed 135 out of 148 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/e2e-test/tests/test_telecom_smoke_tc.py	Minor e2e test cleanup (remove unused assignment / unused page object).
tests/e2e-test/pages/loginPage.py	Ensure page object base class is initialized via super().__init__.
tests/e2e-test/pages/KMGenericPage.py	Ensure page object base class is initialized via super().__init__.
tests/e2e-test/pages/HomePage.py	Ensure page object base class is initialized via super().__init__.
tests/e2e-test/config/constants.py	Adds explicit __all__ exports for constants/test data.
tests/e2e-test/base/base.py	Simplifies base page helpers and request validation setup.
src/tests/api/services/test_history_service.py	Updates unit tests for new credential helper and removed OpenAI deployment config.
src/tests/api/services/test_chat_service.py	Updates tests for new streaming tuple semantics and citation/tool-message behavior.
src/tests/api/helpers/test_chat_helper.py	Removes tests for removed chart helper functionality.
src/tests/api/helpers/test_azure_openai_helper.py	Removes tests for removed Azure OpenAI helper module.
src/tests/api/common/logging/test_event_utils.py	Cleans up unused mock imports.
src/tests/api/common/config/test_config.py	Removes assertions for deleted Azure OpenAI config fields.
src/tests/api/auth/test_auth_utils.py	Removes unused MagicMock import; formatting normalization.
src/start.sh	Improves local start flow (portable sed, cleanup/traps, port management).
src/start.cmd	Improves local start flow on Windows (env var fixes, port management, cleanup).
src/App/WebApp.Dockerfile	Updates Node base image and adds nginx config + API proxy scaffolding.
src/App/src/utils/messageUtils.ts	Adds helpers for chart-query detection and robust citation parsing.
src/App/src/utils/chartUtils.ts	Adds helpers for parsing/validating chart responses and sentiment colors.
src/App/src/utils/apiUtils.ts	Adds retry, request caching, debounce/throttle, and error response helpers.
src/App/src/types/AppTypes.ts	Updates streaming chunk types (delta) and removes unused types/imports.
src/App/src/state/useAppContext.tsx	Removes legacy React context hook.
src/App/src/state/store.ts	Adds Redux store configuration and RootState/AppDispatch types.
src/App/src/state/slices/dashboardSlice.ts	Adds Redux slice for dashboard filters/charts state.
src/App/src/state/slices/citationSlice.ts	Adds Redux slice for citation panel state.
src/App/src/state/slices/chatSlice.ts	Adds Redux slice for chat message/streaming state.
src/App/src/state/slices/chatHistorySlice.ts	Adds Redux slice + thunks for chat history CRUD and pagination.
src/App/src/state/slices/appSlice.ts	Adds Redux slice for app-level config and conversation selection.
src/App/src/state/hooks.ts	Adds typed Redux hooks (useAppDispatch, useAppSelector).
src/App/src/state/AppReducer.tsx	Removes legacy reducer-based state management.
src/App/src/state/AppProvider.tsx	Removes legacy context provider.
src/App/src/state/ActionConstants.tsx	Removes legacy action constants.
src/App/src/logo.svg	Removes unused CRA logo asset.
src/App/src/index.tsx	Switches root provider from custom context to Redux Provider.
src/App/src/hooks/useAutoScroll.ts	Adds reusable auto-scroll hook for chat UI.
src/App/src/configs/Utils.tsx	Avoids in-place sort mutation and adjusts error parsing behavior.
src/App/src/components/Citations/Citations.tsx	Refactors citations component to Redux and improves keying/callbacks.
src/App/src/components/Citations/AnswerParser.tsx	Simplifies answer parsing (removes commented/unused logic).
src/App/src/components/CitationPanel/CitationPanel.tsx	Refactors citation panel to Redux and removes raw HTML rendering.
src/App/src/components/CitationPanel/CitationPanel.css	Cleans up CSS comments/formatting and keeps styling consistent.
src/App/src/components/ChatHistoryPanel/ChatHistoryPanel.tsx	Refactors to Redux selectors and small a11y/string cleanup.
src/App/src/components/ChatHistoryListItemGroups/ChatHistoryListItemGroups.tsx	Refactors to Redux selectors; improves intersection observer cleanup.
src/App/src/components/ChatHistoryListItemCell/ChatHistoryListItemCell.tsx	Refactors history item actions to Redux thunks; improves deletion flow.
src/App/src/components/ChatHistory/ChatHistory.tsx	Removes unused “Coming soon” ChatHistory component.
src/App/src/components/ChatHistory/ChatHistory.css	Removes unused ChatHistory styles.
src/App/src/components/ChatChart/ChatChart.tsx	Small refactor to stabilize canvas/parent references.
src/App/src/components/Chat/ChatMessageItem.tsx	Adds extracted chat message renderer (markdown/chart/citations).
src/App/src/components/Chat/Chat.css	Removes large blocks of commented CSS.
src/App/src/components/ChartFilter/ChartFilter.tsx	Refactors filter updates to Redux and improves menu rendering callbacks.
src/App/src/chartComponents/WordCloudChart.tsx	Improves redraw trigger logic and ResizeObserver element handling.
src/App/src/chartComponents/HorizontalBarChart.tsx	Fixes effect dependency list to include container ID.
src/App/src/chartComponents/DonutChart.tsx	Safens ref usage and cleanup for d3 selection.
src/App/src/Assets/Reset-icon.svg	Removes unused asset.
src/App/src/App.test.tsx	Replaces placeholder CRA test with reducer unit test.
src/App/src/App.css	Adds layout CSS improvements and new citation panel wrapper styling.
src/App/src/api/httpClient.ts	Adds fetch-based HTTP client with interceptors, params, and timeouts.
src/App/package.json	Adds Redux deps, removes axios, updates deps/overrides.
src/App/nginx.conf	Adds nginx config with SPA routing + optional API proxy include.
src/App/env.sh	Adds WAF-mode nginx proxy generation + keeps APP_* substitution behavior.
src/api/services/history_service.py	Updates Cosmos credential construction for async SDK usage.
src/api/services/chat_service.py	Refactors streaming to yield (role, content) deltas and tool citations.
src/api/requirements.txt	Updates Python dependency versions.
src/api/helpers/chat_helper.py	Removes chart helper code tied to Azure OpenAI client.
src/api/helpers/azure_openai_helper.py	Removes Azure OpenAI client helper module.
src/api/helpers/azure_credential_utils.py	Adds build_async_azure_credential helper for async SDK clients.
src/api/common/database/sqldb_service.py	Fixes missing return and narrows overly-broad exception handling.
src/api/common/database/cosmosdb_service.py	Fixes indentation/return for delete_messages.
src/api/common/config/config.py	Removes Azure OpenAI configuration fields.
src/api/api/history_routes.py	Adds retry to mitigate transient Cosmos failures + preserves HTTPException codes.
src/api/.env.sample	Removes Azure OpenAI env entries now unused by backend.
README.md	Updates deployment notes and adjusts referenced hosting service text.
next-steps.md	Updates infra description/troubleshooting from Container Apps to App Service.
infra/scripts/run_create_index_scripts.sh	Updates parameters/role assignment and passes model args to CU scripts.
infra/scripts/run_create_agents_scripts.sh	Strips CRLF from env values for cross-platform robustness.
infra/scripts/quota_check_params.sh	Updates default embedding model + supported regions list.
infra/scripts/index_scripts/requirements.txt	Updates script requirements (e.g., pypdf) and removes inference SDK.
infra/scripts/index_scripts/content_understanding_client.py	Updates CU client endpoints + adds defaults support + file retrieval changes.
infra/scripts/index_scripts/04_cu_process_custom_data.py	Migrates embeddings from inference SDK to OpenAI client; updates analyzer IDs.
infra/scripts/index_scripts/03_cu_process_data_text.py	Migrates embeddings from inference SDK to OpenAI client; updates analyzer IDs.
infra/scripts/index_scripts/02_create_cu_template_text.py	Updates analyzer ID/template and sets defaults + models during creation.
infra/scripts/index_scripts/02_create_cu_template_audio.py	Updates analyzer ID/template and sets defaults + models during creation.
infra/scripts/fabric_scripts/notebooks/speech_to_text/process_data_stt.ipynb	Updates embedding model used in the notebook.
infra/scripts/fabric_scripts/create_fabric_items.py	Cleans imports, improves variable naming, narrows exception catches.
infra/scripts/checkquota_km.sh	Updates embedding model SKU name for quota checks.
infra/scripts/agent_scripts/requirements.txt	Bumps aiohttp version.
infra/modules/fetch-container-image.bicep	Removes unused Container Apps helper module.
infra/main.waf.parameters.json	Standardizes env var names and removes unused CU location parameter.
infra/main.parameters.json	Standardizes env var names and removes unused CU location parameter.
infra/data/ckm_analyzer_config_json.json	Updates analyzer template to use prebuilt base analyzer.
infra/data/ckm_analyzer_config_audio.json	Updates analyzer template to use prebuilt base analyzer.
infra/abbreviations.json	Removes abbreviations file (no longer referenced).
documents/re-use-log-analytics.md	Updates env var name for reusing Log Analytics workspace RID.
documents/re-use-foundry-project.md	Updates env var name + adds region requirements guidance.
documents/QuotaCheck.md	Updates models/regions examples to new embedding model + region list.
documents/LocalDevelopmentSetup.md	Adds “automated script” instructions + updates env var naming.
documents/DeploymentGuide.md	Updates prerequisites and hosting/service references.
documents/CustomizingAzdParameters.md	Updates parameter names and descriptions for new env var scheme.
documents/CustomizeData.md	Updates process script parameter list.
documents/create_new_app_registration.md	Fixes UI instruction for redirect URI workflow step.
documents/AzureGPTQuotaSettings.md	Updates embedding model name.
documents/AVMPostDeploymentGuide.md	Adds Bicep CLI prereq + updates process script parameter list.
docs/workshop/docs/workshop/support-docs/quota_check.md	Updates embedding model name in examples.
docs/workshop/docs/workshop/support-docs/AzureGPTQuotaSettings.md	Updates embedding model name.
docs/workshop/docs/workshop/Challenge-5/python/utility.py	Narrows broad excepts; removes unused initializations.
docs/workshop/docs/workshop/Challenge-3-and-4/knowledge_mining_api.ipynb	Updates embedding model name in sample config.
azure.yaml	Pins azd version range to exclude 1.23.9.
azure_custom.yaml	Pins azd version range to exclude 1.23.9.
.github/workflows/validate-bicep-params.yml	Adds scheduled/PR validation workflow for Bicep params.
.github/workflows/test.yml	Adds junitxml output + PR coverage comment step and permissions.
.github/workflows/job-test-automation.yml	Standardizes USE_CASE input/env naming across automation.
.github/workflows/job-cleanup-resources.yml	Renames AI service location input/env for consistency.
.github/workflows/docker-build.yml	Limits workflow triggers to relevant path changes.
.github/workflows/deploy-orchestrator.yml	Standardizes variable names and updates downstream workflow wiring.
.github/workflows/deploy-KMGeneric.yml	Removes unused bicep parameter and updates deployment invocation.
.github/workflows/bicep_deploy.yml	Adds path filters to bicep deploy workflow.
.github/workflows/azure-dev.yml	Refactors Azure dev deploy workflow (auth/login + dynamic env naming).
.github/workflows/azure-dev-validation.yml	Removes legacy template validation workflow.
.github/workflows/azd-template-validation.yml	Adds new scheduled azd template validation workflow.
.devcontainer/devcontainer.json	Updates Azure CLI feature config to install latest CLI + Bicep.

Comments suppressed due to low confidence (2)

infra/scripts/index_scripts/content_understanding_client.py:281

• get_file_from_analyze_operation() still asserts Content-Type == image/jpeg even though it now retrieves from a /files/{file_path} endpoint and the method name/docstring indicate arbitrary files. This assertion will raise for any non-JPEG file (or if the service returns a different image type). Consider removing the assertion or validating against an allowed set based on the expected file types.
  src/api/services/history_service.py:53
• init_cosmosdb_client() now constructs an async Azure Identity credential (build_async_azure_credential) and passes it into azure.cosmos.aio.CosmosClient, but the credential is never closed. Since azure.identity.aio credentials own an aiohttp session, this can produce unclosed-session warnings and leak resources over time. Consider updating the Cosmos client wrapper to close the credential (or refactor to create/async with the credential per operation).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.