Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
69e44fd
feat: Add custom data processing script and related infrastructure
Pavan-Microsoft Dec 10, 2025
2113a0e
feat: Add optimized SQL insert script generation for DataFrame proces…
Pavan-Microsoft Dec 10, 2025
1b9e5b9
refactor: Clean up code formatting and remove unused AUDIO_DIRECTORY …
Pavan-Microsoft Dec 10, 2025
df45bfc
Refactor run_create_index_scripts.sh: Enhance Azure authentication, r…
Pavan-Microsoft Dec 11, 2025
ac75442
feat: Add script to assign SQL roles to Azure Managed Identities; rem…
Pavan-Microsoft Dec 11, 2025
47ebc0f
feat: Add virtual environment activation messages for Windows and Uni…
Pavan-Microsoft Dec 11, 2025
cc2abf5
feat: Update virtual environment path and activation handling for cro…
Pavan-Microsoft Dec 11, 2025
a9fe3dd
feat: Enhance script to support dynamic Python command detection and …
Pavan-Microsoft Dec 11, 2025
98f06b0
feat: Add informative messages for running sample data processing com…
Pavan-Microsoft Dec 11, 2025
5d86e4e
feat: Refactor SQL connection handling to use Azure CLI token and imp…
Pavan-Microsoft Dec 11, 2025
203a86b
Refactor infrastructure scripts and Bicep templates to remove Key Vau…
Pavan-Microsoft Dec 11, 2025
51c5dd0
Add process_custom_data.sh script for enabling public access and proc…
Pavan-Microsoft Dec 11, 2025
b96eab7
refactor: Simplify user creation logic in assign_sql_roles function
Pavan-Microsoft Dec 11, 2025
466ec86
feat: Update SQL Server parameter to use fully qualified domain name …
Pavan-Microsoft Dec 11, 2025
06cd33c
fix: Correct variable naming for Azure AI API version in data process…
Pavan-Microsoft Dec 11, 2025
45b70f3
feat: Add Content Understanding analyzer creation scripts to process_…
Pavan-Microsoft Dec 11, 2025
cf0ca27
feat: Update Azure OpenAI API version to 2024-12-01-preview in conten…
Pavan-Microsoft Dec 11, 2025
81995a5
feat: Update scripts and documentation for processing custom and samp…
Pavan-Microsoft Dec 12, 2025
00669d7
feat: Update scripts and documentation to support new parameters for …
Pavan-Microsoft Dec 12, 2025
f1b0781
feat: Update devcontainer configuration and setup script to include M…
Pavan-Microsoft Dec 12, 2025
2fedc88
Refactor code structure for improved readability and maintainability
Pavan-Microsoft Dec 12, 2025
96fabfa
feat: Add Python virtual environment setup and requirements installat…
Pavan-Microsoft Dec 12, 2025
a604f09
feat: Add dependency management for cognitive service deployments and…
Pavan-Microsoft Dec 14, 2025
7e72f30
feat: Refactor SQL Server and Cognitive Services modules to enhance p…
Pavan-Microsoft Dec 15, 2025
697b6ce
feat: Remove unused private DNS zone for vaultcore from main.bicep
Pavan-Microsoft Dec 15, 2025
a683a56
feat: Remove secrets export configuration from AI services module
Pavan-Microsoft Dec 15, 2025
04e541a
feat: Update AzureCliCredential to include process timeout for improv…
Pavan-Microsoft Dec 15, 2025
da41343
Merge branch 'dev' of https://github.com/microsoft/Conversation-Knowl…
Pavan-Microsoft Dec 15, 2025
6465691
feat: Remove tags parameter from main.parameters.json and main.waf.pa…
Pavan-Microsoft Dec 15, 2025
7e51dd3
feat: Enhance install script for ODBC Driver installation on Debian/U…
Pavan-Microsoft Dec 15, 2025
b25b0f4
Revert "feat: Enhance install script for ODBC Driver installation on …
Pavan-Microsoft Dec 15, 2025
782d633
feat: Update ODBC Driver references to version 18 across documentatio…
Pavan-Microsoft Dec 15, 2025
86ffcf0
feat: Remove branch parameter from azd init command in install script
Pavan-Microsoft Dec 15, 2025
2a8f324
feat: Update mssql-odbc-driver version to 18 in devcontainer configur…
Pavan-Microsoft Dec 15, 2025
cf1acb3
fix: Remove unnecessary blank lines in get_db_connection function for…
Pavan-Microsoft Dec 15, 2025
ea9960e
fix: Refactor data processing with manual scripts to resolve deployme…
Avijit-Microsoft Dec 15, 2025
8e47674
Refactor scripts for improved logging and error handling; remove redu…
Pavan-Microsoft Dec 16, 2025
93549fe
fix: Add error handling for Python requirements installation in scripts
Pavan-Microsoft Dec 16, 2025
1b0be78
refactor: scripts for improved logging and error handling; remove red…
Avijit-Microsoft Dec 16, 2025
66019a1
fix: Update deployment guide to include virtual environment activatio…
Pavan-Microsoft Dec 16, 2025
66574d9
fix: Update deployment guide to include virtual environment activatio…
Avijit-Microsoft Dec 16, 2025
90c1734
update readme
malrose07 Dec 17, 2025
27cee26
fix: Update Windows activation instructions for virtual environment i…
Pavan-Microsoft Dec 17, 2025
e74a4e4
docs: Update Windows activation instructions for virtual environment …
Avijit-Microsoft Dec 17, 2025
baa8ca3
add IT helpdesk use case
malrose07 Dec 17, 2025
c8e431f
update process data
malrose07 Dec 17, 2025
f1d4c1b
update sample data
malrose07 Dec 18, 2025
e6b79ae
update architecture
malrose07 Dec 18, 2025
eb36590
refactor: Update use case terminology and improve date handling in sc…
Pavan-Microsoft Dec 18, 2025
b27a7da
fix: Update use case terminology and improve date handling in scripts
Avijit-Microsoft Dec 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@
"features": {
"ghcr.io/azure/azure-dev/azd:latest": {},
"ghcr.io/devcontainers/features/azure-cli:1": {},
"ghcr.io/devcontainers/features/docker-in-docker:2": {}
"ghcr.io/devcontainers/features/docker-in-docker:2": {},
"ghcr.io/jlaundry/devcontainer-features/mssql-odbc-driver:1": {
"version": "18"
}
},
"customizations": {
"vscode": {
Expand Down
3 changes: 2 additions & 1 deletion .devcontainer/setup_env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@ git pull
# provide execute permission to quotacheck script
sudo chmod +x ./infra/scripts/checkquota_km.sh
sudo chmod +x ./infra/scripts/quota_check_params.sh
sudo chmod +x ./infra/scripts/run_process_data_scripts.sh
sudo chmod +x ./infra/scripts/process_sample_data.sh
sudo chmod +x ./infra/scripts/process_custom_data.sh
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,18 @@ Employees can engage directly with conversational data using natural language, e
Clear, contextual insights empower employees to take meaningful action based on data-driven evidence.


</details>

### Use Case
<details>
<summary>Click to learn more about what use cases this solution provides</summary>
<br/>

| **Use case** | **Persona** | **Challenges** | **Summary/approach** |
|---|---|---|---|
| Contact Center Customer Support | Analyst | Difficulty in extracting actionable insights from large, complex datasets due to limited context or practical considerations. Limited ability to engage with data interactively, making it challenging to find the right questions to dig deeper.| Contextualized insights from mined data that enables employees to solve problems and take action. Interactive data that allow employees to ask questions and receive timely responses, providing better understanding and problem-solving.|
IT Helpdesk | IT Helpdesk Analyst | Manually reviewing IT Helpdesk calls to identify recurring issues is time-consuming and inefficient. Creating graphs, analyzing performance problems, and drafting FAQs is often a slow process, leaving gaps in self-service support. | Address these challenges by leveraging AI to gain insights from call data, generating visual summaries, uncovering common issues, and producing FAQ content, transforming a labor-intensive review process into a fast, accurate, and actionable workflow. |

</details>

<br /><br />
Expand Down
5 changes: 5 additions & 0 deletions azure.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,18 @@ hooks:
run: |
Write-Host "Web app URL: "
Write-Host "$env:WEB_APP_URL" -ForegroundColor Cyan
Write-Host "`nRun the following command in your Bash terminal. It will grant the necessary permissions between resources and your user account, and also process and load the sample data into the application."
Write-Host "bash ./infra/scripts/process_sample_data.sh" -ForegroundColor Cyan
shell: pwsh
continueOnError: false
interactive: true
posix:
run: |
echo "Web app URL: "
echo $WEB_APP_URL
echo ""
echo "Run the following command in your Bash terminal. It will grant the necessary permissions between resources and your user account, and also process and load the sample data into the application."
echo "bash ./infra/scripts/process_sample_data.sh"
shell: sh
continueOnError: false
interactive: true
25 changes: 17 additions & 8 deletions documents/CustomizeData.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,24 @@ If you would like to update the solution to leverage your own data please follow
> **Note for WAF-aligned deployments:** If your deployment uses private networking, you'll need to log into a VM within the virtual network to upload files. See [VM login instructions](#how-to-login-to-vm-using-azure-bastion) below.

3. If you have audio files, upload them to `custom_audiodata` folder. If you have call transcript files, upload them to `custom_transcripts` folder.
4. Navigate to the terminal and run the `run_process_data_scripts.sh` to process the new data into the solution with the following commands.
```shell
cd infra/scripts

az login

bash run_process_data_scripts.sh resourcegroupname_param
4. Navigate to the terminal and run the `process_custom_data.sh` script to process the new data into the solution with the following commands:

```bash
bash ./infra/scripts/process_custom_data.sh
```

If you don't have `azd env` then you need to pass parameters along with the command. Parameters are grouped by service for clarity. The command will look like the following:

```bash
bash ./infra/scripts/process_custom_data.sh \
<Resource-Group-Name> <Azure-Subscription-ID> \
<Storage-Account-Name> <Storage-Container-Name> \
<SQL-Server-Name> <SQL-Database-Name> <Backend-User-MID-Client-ID> <Backend-User-MID-Display-Name> \
<AI-Search-Name> <Search-Endpoint> \
<AI-Foundry-Resource-ID> <CU-Foundry-Resource-ID> \
<OpenAI-Endpoint> <Embedding-Model> <Deployment-Model> \
<CU-Endpoint> <AI-Agent-Endpoint> <CU-API-Version>
```
a. resourcegroupname_param - the name of the resource group.

## How to Login to VM Using Azure Bastion

Expand Down
66 changes: 60 additions & 6 deletions documents/DeploymentGuide.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ To deploy this solution, ensure you have access to an [Azure subscription](https

Check the [Azure Products by Region](https://azure.microsoft.com/en-us/explore/global-infrastructure/products-by-region/?products=all&regions=all) page and select a **region** where the following services are available:

- [Azure AI Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry)
- [Azure AI Content Understanding Service](https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/)
- [Microsoft Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry)
- [Azure Content Understanding Service](https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/)
- [Azure OpenAI Service](https://learn.microsoft.com/en-us/azure/ai-services/openai/)
- [GPT Model Capacity](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models)
- [Azure AI Search](https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search)
- [Foundry IQ](https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search)
- [Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/sql-database-paas-overview)
- [Azure Cosmos DB](https://learn.microsoft.com/en-us/azure/cosmos-db/introduction)
- [Azure Container Apps](https://learn.microsoft.com/en-us/azure/container-apps/)
Expand Down Expand Up @@ -150,6 +150,7 @@ If you're not using one of the above options for opening the project, then you'l
- [Python 3.9+](https://www.python.org/downloads/)
- [Docker Desktop](https://www.docker.com/products/docker-desktop/)
- [Git](https://git-scm.com/downloads)
- [Microsoft ODBC Driver 18](https://learn.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server?view=sql-server-ver16) for SQL Server.

2. Clone the repository or download the project code via command-line:

Expand All @@ -176,6 +177,7 @@ When you start the deployment, most parameters will have **default values**, but
| **Azure Region** | The region where resources will be created. | *(empty)* |
| **Environment Name** | A **3–20 character alphanumeric value** used to generate a unique ID to prefix the resources. | env\_name |
| **Azure AI Content Understanding Location** | Region for content understanding resources. | swedencentral |
| **Use Case** | Industry use case: **Contact-center** or **IT_helpdesk**. | (empty) |
| **Secondary Location** | A **less busy** region for **Azure SQL and Azure Cosmos DB**, useful in case of availability constraints. | eastus2 |
| **Deployment Type** | Select from a drop-down list (allowed: `Standard`, `GlobalStandard`). | GlobalStandard |
| **GPT Model** | Choose from **gpt-4, gpt-4o, gpt-4o-mini**. | gpt-4o-mini |
Expand Down Expand Up @@ -244,12 +246,64 @@ Once you've opened the project in [Codespaces](#github-codespaces), [Dev Contain

3. Provide an `azd` environment name (e.g., "ckmapp").
4. Select a subscription from your Azure account and choose a location that has quota for all the resources.
-- This deployment will take *7-10 minutes* to provision the resources in your account and set up the solution with sample data.
5. Choose the use case:
- **Contact-center**
- **IT_helpdesk**

- This deployment generally takes **7-10 minutes** to provision the resources in your account and set up the solution.
- If you encounter an error or timeout during deployment, changing the location may help, as there could be availability constraints for the resources.

5. Once the deployment has completed successfully, open the [Azure Portal](https://portal.azure.com/), go to the deployed resource group, find the App Service, and get the app URL from `Default domain`.
5. Once the deployment has completed successfully, copy the bash command from terminal: (ex: `bash ./infra/scripts/process_sample_data.sh`) for later use.

6. Create and activate a virtual environment in bash terminal:

```shell
python -m venv .venv
```

**For Windows (Bash):**
```shell
source .venv/Scripts/activate
```

**For Linux/VS Code Web (Bash):**
```shell
source .venv/bin/activate
```

7. Login to Azure:

```shell
az login
```

Alternatively, login to Azure using a device code (recommended when using VS Code Web):

```shell
az login --use-device-code
```
8. Run the bash script from the output of the azd deployment. The script will look like the following:

```bash
bash ./infra/scripts/process_sample_data.sh
```

If you don't have `azd env` then you need to pass parameters along with the command. Parameters are grouped by service for clarity. The command will look like the following:

```bash
bash ./infra/scripts/process_sample_data.sh \
<Resource-Group-Name> <Azure-Subscription-ID> \
<Storage-Account-Name> <Storage-Container-Name> \
<SQL-Server-Name> <SQL-Database-Name> <Backend-User-MID-Client-ID> <Backend-User-MID-Display-Name> \
<AI-Search-Name> <Search-Endpoint> \
<AI-Foundry-Resource-ID> <CU-Foundry-Resource-ID> \
<OpenAI-Endpoint> <Embedding-Model> <Deployment-Model> \
<CU-Endpoint> <AI-Agent-Endpoint> <CU-API-Version> <Use-Case>
```

9. Once the deployment has completed successfully, open the [Azure Portal](https://portal.azure.com/), go to the deployed resource group, find the App Service, and get the app URL from `Default domain`.

6. If you are done trying out the application, you can delete the resources by running `azd down`.
10. You can now delete the resources by running `azd down`, if you are done trying out the application.
> **Note:** If you deployed with `enableRedundancy=true` and Log Analytics workspace replication is enabled, you must first disable replication before running `azd down` else resource group delete will fail. Follow the steps in [Handling Log Analytics Workspace Deletion with Replication Enabled](./LogAnalyticsReplicationDisable.md), wait until replication returns `false`, then run `azd down`.

### 🛠️ Troubleshooting
Expand Down
Binary file modified documents/Images/ReadMe/solution-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions documents/LocalDebuggingSetup.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ Install these tools before you start:
- [Azure Tools](https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-node-azure-pack)
- [Bicep](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-bicep)
- [Python](https://marketplace.visualstudio.com/items?itemName=ms-python.python)
- [Python 3.11](https://www.python.org/downloads/). **Important:** Check "Add Python to PATH" during installation.
- [Python 3.9+](https://www.python.org/downloads/). **Important:** Check "Add Python to PATH" during installation.
- [PowerShell 7.0+](https://github.com/PowerShell/PowerShell#get-powershell).
- [Node.js (LTS)](https://nodejs.org/en).
- [Git](https://git-scm.com/downloads).
- [Azure Developer CLI (azd) v1.18.0+](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/install-azd).
- [Microsoft ODBC Driver 17](https://learn.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server?view=sql-server-ver16) for SQL Server.
- [Microsoft ODBC Driver 18](https://learn.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server?view=sql-server-ver16) for SQL Server.


## Setup Steps
Expand Down
8 changes: 8 additions & 0 deletions documents/SampleQuestions.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ To help you get started, here are some **Sample Prompts** you can ask in the app
> Note: To avoid rate limit errors, pause for 2–3 seconds after a response before submitting the next question. <br>
Average response time is 8–14 seconds.

For Contact Center (telecom) use case:
1. Ask the following questions:
- Total number of calls by date for last 7 days.
- To view the response data as a graph, just prompt "Generate Chart".
Expand All @@ -16,5 +17,12 @@ Average response time is 8–14 seconds.

![GenerateDraft](Images/Samplequestions1.png)

For IT Helpdesk use case:
1. Ask the following questions:
- Please provide the total number of calls by date for last 7 days
- Generate a bar chart showing the number of helpdesk calls per day for the last week.
- Provide a summary of performance issues users reported this week.
- Turn these key topics into a structured FAQ.


This structured approach helps users quickly extract actionable insights from client conversations to help users understand priorities, trends, and opportunities for better engagement.
Binary file added infra/data/IT_helpdesk/call_transcripts.zip
Binary file not shown.
Loading
Loading