Skip to content

Commit 29714c5

Browse files
JulianePadraoJulianePadrao
committed
Add Lab 16 and Lab 17: Diagnose slow queries and design machine learning solutions
1 parent 72c55f6 commit 29714c5

2 files changed

Lines changed: 306 additions & 0 deletions

File tree

‎Instructions/Labs/16-diagnose-slow-query-copilot.md‎

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
---
2+
lab:
3+
title: 'Lab 16 – Diagnose a slow query with Copilot'
4+
module: Optimize query performance in Azure SQL
5+
description: "You'll use Microsoft Copilot in the Azure portal and GitHub Copilot in SSMS to investigate a regressed stored procedure, generate an index recommendation, validate it against a performance baseline, and apply the fix. By the end, you'll understand how to use Copilot to diagnose and resolve slow queries while following a validation gate."
6+
duration: 30 # duration in minutes
7+
level: 300 # 100 basic concepts, 200 foundations, 300 practical usage, 400 advanced scenarios, 500 expert design
8+
islab: true # if this is not a lab that should be listed in the catalog, set to false
9+
status: 'in-development' # in-development or released
10+
targetDate: '2026-07-01' # Set to the future date when you expect an in-development lab to be released
11+
---
12+
13+
You're the DBA for the **ContosoOps** database. The operations team reports that `dbo.usp_GetOpenWorkOrdersByTechnician` has been running slow for the last two days. Response times that used to average 150 ms are now averaging 3.2 seconds. Your job: use Copilot to investigate, generate a fix, validate it, and apply it — following the validation gate from the previous unit.
14+
15+
## Prerequisites
16+
17+
- Access to an Azure SQL Database instance in the Azure portal
18+
- SSMS installed with the **AI Assistance** workload (added via Visual Studio Installer)
19+
- An active GitHub Copilot subscription
20+
- The **ContosoOps** database with the `WorkOrders` and `Technicians` tables (or any equivalent database where you can identify a slow query from Query Store)
21+
22+
> [!NOTE]
23+
> If you don't have a ContosoOps database, you can follow along with any Azure SQL Database that has Query Store enabled and contains a stored procedure you can observe in performance data. The steps and validation patterns are identical regardless of the schema.
24+
25+
---
26+
27+
## Use Copilot in the portal to identify the problem
28+
29+
1. In the [Azure portal](https://portal.azure.com), navigate to your Azure SQL Database resource.
30+
31+
1. On the resource page, open the **Microsoft Copilot** pane from the portal's top toolbar. The Copilot pane opens on the right side of the portal.
32+
33+
1. In the Copilot chat, ask:
34+
35+
> "What queries have regressed in performance in the last 24 hours?"
36+
37+
1. Review the response. Copilot queries the integrated Query Store signals and returns a ranked list of queries with increased average duration. Confirm that `dbo.usp_GetOpenWorkOrdersByTechnician` appears as a top contributor. Note the **query ID** that Copilot references — you use it in the next task.
38+
39+
> [!TIP]
40+
> If Copilot returns multiple queries, ask a follow-up: "Show me the execution trend for query ID *\<id\>* over the last 48 hours." Copilot can surface duration and logical reads over time using the same Query Store data that **Query Performance Insight** exposes.
41+
42+
---
43+
44+
## Open the slow query in SSMS and analyze with GitHub Copilot
45+
46+
1. Open SSMS and connect to the ContosoOps database.
47+
48+
1. Open a new query window and run the following script to retrieve the stored procedure's query text and recent execution statistics from Query Store:
49+
50+
```sql
51+
-- Find the stored procedure's query text and current execution plan
52+
SELECT TOP 5
53+
qt.query_sql_text,
54+
rs.avg_duration / 1000.0 AS avg_duration_ms,
55+
p.plan_id
56+
FROM sys.query_store_query_text AS qt
57+
JOIN sys.query_store_query AS q ON qt.query_text_id = q.query_text_id
58+
JOIN sys.query_store_plan AS p ON q.query_id = p.query_id
59+
JOIN sys.query_store_runtime_stats AS rs ON p.plan_id = rs.plan_id
60+
WHERE qt.query_sql_text LIKE '%GetOpenWorkOrdersByTechnician%'
61+
ORDER BY rs.avg_duration DESC;
62+
```
63+
64+
1. In the results, copy the `query_sql_text` value for the slow procedure. Select the query text directly in the editor.
65+
66+
1. Right-click the selected text and choose **Explain with Copilot**. GitHub Copilot opens a chat panel and describes the query. In the explanation, Copilot identifies a table scan on `WorkOrders` caused by a filter predicate on `TechnicianID` and `Status` with no supporting index.
67+
68+
---
69+
70+
## Ask Copilot for an index recommendation
71+
72+
1. In the Copilot chat, type:
73+
74+
> "This query has a table scan on WorkOrders filtered by TechnicianID and Status. What index would you recommend?"
75+
76+
1. Copilot responds with an index recommendation similar to the following:
77+
78+
```sql
79+
-- Copilot-suggested index
80+
CREATE NONCLUSTERED INDEX IX_WorkOrders_TechnicianID_Status
81+
ON dbo.WorkOrders (TechnicianID, Status)
82+
INCLUDE (WorkOrderID, OpenedDate, Description);
83+
```
84+
85+
1. Read the explanation Copilot provides alongside the index. It should describe the key column order — `TechnicianID` first as the equality predicate, `Status` second as a range or equality predicate — and explain why the `INCLUDE` columns avoid a key lookup.
86+
87+
> [!NOTE]
88+
> The `INCLUDE` columns Copilot suggests depend on the columns it can infer from the query text. If your stored procedure selects additional columns not in Copilot's context, add them to the `INCLUDE` list before creating the index.
89+
90+
---
91+
92+
## Validate the recommendation
93+
94+
Before you create the index, apply the validation gate from the previous unit.
95+
96+
1. Check whether a similar index already exists to avoid redundant indexes:
97+
98+
```sql
99+
-- Check for existing indexes on WorkOrders
100+
SELECT
101+
i.name AS index_name,
102+
i.type_desc,
103+
c.name AS column_name,
104+
ic.is_included_column,
105+
ic.key_ordinal
106+
FROM sys.indexes AS i
107+
JOIN sys.index_columns AS ic ON i.object_id = ic.object_id AND i.index_id = ic.index_id
108+
JOIN sys.columns AS c ON ic.object_id = c.object_id AND ic.column_id = c.column_id
109+
WHERE i.object_id = OBJECT_ID('dbo.WorkOrders')
110+
ORDER BY i.index_id, ic.key_ordinal;
111+
```
112+
113+
If an index on `TechnicianID` already exists, check whether it includes `Status` and the `INCLUDE` columns. If it does, you don't need the new index — ask Copilot to refine the recommendation based on the existing index.
114+
115+
1. Capture a pre-index baseline by running the stored procedure in non-production with I/O and time statistics enabled:
116+
117+
```sql
118+
SET STATISTICS IO ON;
119+
SET STATISTICS TIME ON;
120+
GO
121+
122+
EXEC dbo.usp_GetOpenWorkOrdersByTechnician @TechnicianID = 42;
123+
GO
124+
125+
SET STATISTICS IO OFF;
126+
SET STATISTICS TIME OFF;
127+
```
128+
129+
Note the **logical reads** and **elapsed time** from the Messages tab. This is your baseline.
130+
131+
1. Enable **Include Actual Execution Plan** in SSMS (or press **Ctrl+M**), run the stored procedure again, and inspect the plan. Confirm that the table scan on `WorkOrders` appears in the current plan. This validates what Copilot identified.
132+
133+
---
134+
135+
## Apply and verify
136+
137+
1. Create the Copilot-suggested index in your non-production environment:
138+
139+
```sql
140+
CREATE NONCLUSTERED INDEX IX_WorkOrders_TechnicianID_Status
141+
ON dbo.WorkOrders (TechnicianID, Status)
142+
INCLUDE (WorkOrderID, OpenedDate, Description);
143+
```
144+
145+
1. Run the stored procedure again with `SET STATISTICS IO ON; SET STATISTICS TIME ON;` and compare the new logical reads and elapsed time against the baseline you captured in step 13. Confirm that logical reads decreased and elapsed time dropped toward the historical 150 ms baseline.
146+
147+
1. Inspect the actual execution plan again. Confirm that the plan now shows an **Index Seek** on `IX_WorkOrders_TechnicianID_Status` rather than the table scan.
148+
149+
1. After validating in non-production, schedule the index creation for production during a maintenance window. After the maintenance window, check Query Store the following day to confirm that the average duration for `dbo.usp_GetOpenWorkOrdersByTechnician` has returned to its historical baseline.
150+
151+
---
152+
153+
## Expected outcome
154+
155+
A targeted nonclustered index eliminates the table scan on `WorkOrders`. Average duration for `dbo.usp_GetOpenWorkOrdersByTechnician` returns to approximately 150 ms. You followed the full validation gate: you checked for redundant indexes, captured a performance baseline, inspected the execution plan, validated in non-production, and scheduled production deployment through a maintenance window.
156+
157+
> [!IMPORTANT]
158+
> The index creation itself causes a brief table lock in standard editions. For large tables in production, use the `ONLINE = ON` option if your service tier supports it, and schedule the operation during low-traffic hours.

‎Instructions/Labs/17-design-machine-learning-solution-case-study.md‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
---
2+
lab:
3+
title: 'Lab 17 – Design a machine learning solution case study'
4+
module: Design a machine learning training solution
5+
description: "You'll work through a Contoso Retail case study to design a machine learning training solution for a product recommendation system, making decisions about data strategy, service selection, compute resources, and deployment. By the end, you'll understand how to make informed machine learning design choices that balance cost, performance, complexity, and team skills."
6+
duration: 15 # duration in minutes
7+
level: 300 # 100 basic concepts, 200 foundations, 300 practical usage, 400 advanced scenarios, 500 expert design
8+
islab: no # if this is not a lab that should be listed in the catalog, set to false
9+
status: 'in-development' # in-development or released
10+
targetDate: '2099-01-01' # Set to the future date when you expect an in-development lab to be released
11+
---
12+
13+
# Design a machine learning solution - Case study
14+
15+
**Estimated Time: 15 minutes**
16+
17+
> [!NOTE]
18+
> To complete this exercise, read the case study carefully. Apply the design principles you've learned throughout this module to make informed decisions. At the end, you'll test your understanding by answering knowledge check questions.
19+
20+
Welcome to Contoso Retail! You've been hired as the **lead data scientist** to help us design a machine learning training solution.
21+
22+
## Understand the problem
23+
24+
At Contoso Retail, we operate both physical stores and an e-commerce platform. We want to build a **product recommendation system** that suggests items to customers based on their browsing and purchase history.
25+
26+
Our goal is to increase customer engagement and sales by showing personalized product recommendations:
27+
28+
- In our **mobile app**, customers should see recommendations immediately when they view a product.
29+
- For our **weekly email campaign**, we want to include the top 5 recommended products for each of our 2 million customers.
30+
31+
Our data engineering team has been collecting customer interaction data for the past two years, including:
32+
33+
- Browsing history (products viewed, time spent)
34+
- Purchase history (items bought, purchase dates, amounts)
35+
- Customer demographics (age, location, preferences)
36+
- Product catalog (categories, prices, descriptions, images)
37+
38+
The data is currently stored in multiple systems:
39+
40+
- Transactional data in Azure SQL Database (updated in real time)
41+
- Clickstream data from our website stored as JSON files in Azure Blob Storage (logged every hour)
42+
- Product images stored in Azure Blob Storage
43+
- Customer profiles in our CRM system (Dynamics 365)
44+
45+
We need your help deciding **how to design the machine learning training solution** to build this recommendation system.
46+
47+
## Consider the requirements
48+
49+
As you design the solution, think about these key areas.
50+
51+
### Data ingestion and preparation
52+
53+
- **Consider the data sources**: We have data in Azure SQL Database, Blob Storage (JSON files), Blob Storage (images), and Dynamics 365. How should we consolidate this data?
54+
- **Consider the data format**: The data is in different formats (structured, semi-structured, and unstructured). What format should we use for training?
55+
- **Consider the data pipeline**: Should we build a data ingestion pipeline? If so, how often should it run?
56+
57+
### Machine learning task and service
58+
59+
- **Consider the machine learning task**: What type of machine learning task is this? Classification, regression, recommendation, or something else?
60+
- **Consider the service**: Should we use Azure Machine Learning, Azure Databricks, Microsoft Fabric, or Microsoft Foundry? What factors influence this choice?
61+
- **Consider existing skills**: Our team has strong Python experience but limited Spark knowledge. How does this affect our choice?
62+
63+
### Compute resources
64+
65+
- **Consider the data size**: We have 2 million customers and millions of product interactions. What compute type is appropriate?
66+
- **Consider the model complexity**: Recommendation systems can be simple (collaborative filtering) or complex (deep learning). How does this affect compute needs?
67+
- **Consider cost**: We have a limited budget for this initial phase. Should we start with CPU or GPU? General purpose or memory optimized?
68+
69+
### Deployment requirements
70+
71+
- **Consider the deployment types**: We need both real-time recommendations (mobile app) and batch predictions (email campaign). How should we handle these different needs?
72+
- **Consider the frequency**: Mobile app recommendations need to be instant. Email campaigns are sent weekly. Should we use different endpoints?
73+
- **Consider the scale**: Our app has 100,000 active daily users. Our email campaign targets 2 million customers. How does scale affect our deployment decisions?
74+
75+
## Your task
76+
77+
Based on these requirements, you need to make design decisions about:
78+
79+
1. **Data strategy**: How will you ingest, transform, and store the data for training?
80+
2. **Service selection**: Which Azure service(s) will you use for training, and why?
81+
3. **Compute strategy**: What compute resources will you provision for training?
82+
4. **Deployment approach**: How will you handle both real-time and batch prediction requirements?
83+
84+
Think through each decision carefully, considering trade-offs between cost, performance, complexity, and team capabilities. The knowledge check questions test your ability to make informed design choices based on this scenario.
85+
86+
## Knowledge check
87+
88+
Answer the following questions based on the Contoso Retail case study. Select an answer for each question, then expand **Show answer** to check your reasoning.
89+
90+
**1. Based on the Contoso Retail case study, what data ingestion strategy would be most appropriate for consolidating data from Azure SQL Database, Blob Storage (JSON), and Dynamics 365?**
91+
92+
- Manually export data from each source and combine in Excel before training.
93+
- Create an ETL pipeline using Azure Synapse Analytics to extract, transform, and load data into a unified storage layer like Azure Data Lake Storage.
94+
- Keep data in separate sources and connect directly to each during model training.
95+
96+
<details>
97+
<summary>Show answer</summary>
98+
99+
**Create an ETL pipeline using Azure Synapse Analytics to extract, transform, and load data into a unified storage layer like Azure Data Lake Storage.**
100+
101+
The data is spread across structured, semi-structured, and unstructured sources that update on different schedules. An automated ETL pipeline consolidates these sources into a single, training-ready layer. Manual export doesn't scale to millions of interactions, and connecting directly to each source during training adds latency and complexity.
102+
103+
</details>
104+
105+
**2. For the Contoso Retail recommendation system, which Azure service would be most suitable given the team's Python experience and the need to train on large-scale customer interaction data?**
106+
107+
- Microsoft Foundry, because it provides pre-built recommendation models.
108+
- Azure Machine Learning, because it supports the Python SDK, handles large datasets, and provides comprehensive tools for custom model training.
109+
- Azure Databricks, because it's required for any large-scale machine learning.
110+
111+
<details>
112+
<summary>Show answer</summary>
113+
114+
**Azure Machine Learning, because it supports the Python SDK, handles large datasets, and provides comprehensive tools for custom model training.**
115+
116+
The team has strong Python skills but limited Spark knowledge, which makes Azure Machine Learning a better fit than Azure Databricks. Azure Databricks is Spark-based and isn't required for all large-scale machine learning. Microsoft Foundry focuses on generative AI rather than custom recommendation training.
117+
118+
</details>
119+
120+
**3. Considering Contoso Retail needs both real-time recommendations (mobile app) and batch predictions (weekly email campaign), what deployment strategy should they implement?**
121+
122+
- Deploy two separate models: a real-time endpoint for the mobile app and a batch endpoint for the email campaign.
123+
- Deploy only a real-time endpoint and call it 2 million times for the email campaign.
124+
- Deploy only a batch endpoint and accept 5-10 minute delays for mobile app recommendations.
125+
126+
<details>
127+
<summary>Show answer</summary>
128+
129+
**Deploy two separate models: a real-time endpoint for the mobile app and a batch endpoint for the email campaign.**
130+
131+
The two scenarios have different latency and throughput needs. A real-time (online) endpoint delivers instant recommendations in the mobile app, while a batch endpoint efficiently scores 2 million customers for the weekly email. Forcing one endpoint type to handle both leads to either excessive cost or unacceptable latency.
132+
133+
</details>
134+
135+
**4. What compute resource would be most appropriate for training the initial Contoso Retail recommendation model, given the 2 million customer dataset and budget constraints?**
136+
137+
- Start with CPU general-purpose compute, monitor performance, and scale to memory-optimized or GPU if needed.
138+
- Immediately provision the largest GPU memory-optimized compute to ensure fast training.
139+
- Use only local development machines to minimize Azure costs.
140+
141+
<details>
142+
<summary>Show answer</summary>
143+
144+
**Start with CPU general-purpose compute, monitor performance, and scale to memory-optimized or GPU if needed.**
145+
146+
With a limited initial budget, start with cost-effective CPU general-purpose compute and scale up only when the model complexity or training time justifies it. Provisioning the largest GPU upfront wastes budget, and local machines can't handle the scale of 2 million customers and millions of interactions.
147+
148+
</details>

0 commit comments

Comments
 (0)