You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: Instructions/Labs/17-design-machine-learning-solution-case-study.md
+69-22Lines changed: 69 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,16 +19,25 @@ lab:
19
19
20
20
Welcome to Contoso Retail! You've been hired as the **lead data scientist** to help us design a machine learning training solution.
21
21
22
+
## Learning objectives
23
+
24
+
After completing this exercise, you'll be able to:
25
+
26
+
- Choose a data ingestion strategy for consolidating diverse data sources.
27
+
- Select the right Azure service for a machine learning workload based on team skills and scale.
28
+
- Provision cost-appropriate compute for model training.
29
+
- Design a deployment approach that serves both real-time and batch prediction needs.
30
+
22
31
## Understand the problem
23
32
24
33
At Contoso Retail, we operate both physical stores and an e-commerce platform. We want to build a **product recommendation system** that suggests items to customers based on their browsing and purchase history.
25
34
26
35
Our goal is to increase customer engagement and sales by showing personalized product recommendations:
27
36
28
37
- In our **mobile app**, customers should see recommendations immediately when they view a product.
29
-
- For our **weekly email campaign**, we want to include the top 5 recommended products for each of our 2 million customers.
38
+
- For our **weekly email campaign**, we want to include the top 5 recommended products for each of our **2 million customers**.
30
39
31
-
Our data engineering team has been collecting customer interaction data for the past two years, including:
40
+
Our data engineering team has been collecting customer interaction data for the past **two years**, including:
32
41
33
42
- Browsing history (products viewed, time spent)
34
43
- Purchase history (items bought, purchase dates, amounts)
@@ -37,10 +46,12 @@ Our data engineering team has been collecting customer interaction data for the
37
46
38
47
The data is currently stored in multiple systems:
39
48
40
-
- Transactional data in Azure SQL Database (updated in real time)
41
-
- Clickstream data from our website stored as JSON files in Azure Blob Storage (logged every hour)
42
-
- Product images stored in Azure Blob Storage
43
-
- Customer profiles in our CRM system (Dynamics 365)
49
+
| Data source | Service | Data type | Format | Update frequency |
50
+
| --- | --- | --- | --- | --- |
51
+
| Transactional data | Azure SQL Database | Structured | Relational tables | Real time |
We need your help deciding **how to design the machine learning training solution** to build this recommendation system.
46
57
@@ -83,66 +94,102 @@ Based on these requirements, you need to make design decisions about:
83
94
84
95
Think through each decision carefully, considering trade-offs between cost, performance, complexity, and team capabilities. The knowledge check questions test your ability to make informed design choices based on this scenario.
85
96
97
+
> [!TIP]
98
+
> There's rarely a single "right" answer in solution design. Focus on the trade-offs between cost, performance, complexity, and your team's skills — and be ready to justify your choices.
99
+
100
+
## Compare your decisions
101
+
102
+
The following diagram shows one solution architecture that satisfies all of the requirements. Sketch your own design first, then expand the reference solution to compare.
E --> F[(Azure Data Lake Storage<br/>unified training data)]
111
+
F --> G[Azure Machine Learning<br/>model training]
112
+
G --> H[Real-time endpoint<br/>mobile app]
113
+
G --> I[Batch endpoint<br/>weekly email campaign]
114
+
```
115
+
116
+
<details>
117
+
<summary>Show reference solution</summary>
118
+
119
+
-**Data strategy**: Use an ETL pipeline (for example, Azure Synapse Analytics or Azure Data Factory) to extract data from Azure SQL Database, Blob Storage, and Dynamics 365 on a schedule, then transform and land it in a unified Azure Data Lake Storage layer in a training-ready format such as Parquet.
120
+
-**Service selection**: Use Azure Machine Learning. It supports the Python SDK that matches the team's skills, scales to large datasets, and provides end-to-end tooling for custom model training without requiring Spark expertise.
121
+
-**Compute strategy**: Start with CPU general-purpose compute to control cost during the initial phase. Monitor training time, then scale to memory-optimized or GPU compute only if model complexity or data volume requires it.
122
+
-**Deployment approach**: Deploy two endpoints from the trained model — a real-time (online) endpoint for instant mobile app recommendations, and a batch endpoint to score all 2 million customers efficiently for the weekly email campaign.
123
+
124
+
</details>
125
+
86
126
## Knowledge check
87
127
88
128
Answer the following questions based on the Contoso Retail case study. Select an answer for each question, then expand **Show answer** to check your reasoning.
89
129
90
130
**1. Based on the Contoso Retail case study, what data ingestion strategy would be most appropriate for consolidating data from Azure SQL Database, Blob Storage (JSON), and Dynamics 365?**
91
131
92
-
- Manually export data from each source and combine in Excel before training.
93
-
- Create an ETL pipeline using Azure Synapse Analytics to extract, transform, and load data into a unified storage layer like Azure Data Lake Storage.
94
-
- Keep data in separate sources and connect directly to each during model training.
132
+
-**A.**Manually export data from each source and combine in Excel before training.
133
+
-**B.**Create an ETL pipeline using Azure Synapse Analytics to extract, transform, and load data into a unified storage layer like Azure Data Lake Storage.
134
+
-**C.**Keep data in separate sources and connect directly to each during model training.
95
135
96
136
<details>
97
137
<summary>Show answer</summary>
98
138
99
-
**Create an ETL pipeline using Azure Synapse Analytics to extract, transform, and load data into a unified storage layer like Azure Data Lake Storage.**
139
+
✅ **Correct answer: B.**Create an ETL pipeline using Azure Synapse Analytics to extract, transform, and load data into a unified storage layer like Azure Data Lake Storage.
100
140
101
141
The data is spread across structured, semi-structured, and unstructured sources that update on different schedules. An automated ETL pipeline consolidates these sources into a single, training-ready layer. Manual export doesn't scale to millions of interactions, and connecting directly to each source during training adds latency and complexity.
102
142
103
143
</details>
104
144
105
145
**2. For the Contoso Retail recommendation system, which Azure service would be most suitable given the team's Python experience and the need to train on large-scale customer interaction data?**
106
146
107
-
- Microsoft Foundry, because it provides pre-built recommendation models.
108
-
- Azure Machine Learning, because it supports the Python SDK, handles large datasets, and provides comprehensive tools for custom model training.
109
-
- Azure Databricks, because it's required for any large-scale machine learning.
147
+
-**A.**Microsoft Foundry, because it provides pre-built recommendation models.
148
+
-**B.**Azure Machine Learning, because it supports the Python SDK, handles large datasets, and provides comprehensive tools for custom model training.
149
+
-**C.**Azure Databricks, because it's required for any large-scale machine learning.
110
150
111
151
<details>
112
152
<summary>Show answer</summary>
113
153
114
-
**Azure Machine Learning, because it supports the Python SDK, handles large datasets, and provides comprehensive tools for custom model training.**
154
+
✅ **Correct answer: B.**Azure Machine Learning, because it supports the Python SDK, handles large datasets, and provides comprehensive tools for custom model training.
115
155
116
156
The team has strong Python skills but limited Spark knowledge, which makes Azure Machine Learning a better fit than Azure Databricks. Azure Databricks is Spark-based and isn't required for all large-scale machine learning. Microsoft Foundry focuses on generative AI rather than custom recommendation training.
117
157
118
158
</details>
119
159
120
160
**3. Considering Contoso Retail needs both real-time recommendations (mobile app) and batch predictions (weekly email campaign), what deployment strategy should they implement?**
121
161
122
-
- Deploy two separate models: a real-time endpoint for the mobile app and a batch endpoint for the email campaign.
123
-
- Deploy only a real-time endpoint and call it 2 million times for the email campaign.
124
-
- Deploy only a batch endpoint and accept 5-10 minute delays for mobile app recommendations.
162
+
-**A.**Deploy two separate models: a real-time endpoint for the mobile app and a batch endpoint for the email campaign.
163
+
-**B.**Deploy only a real-time endpoint and call it 2 million times for the email campaign.
164
+
-**C.**Deploy only a batch endpoint and accept 5-10 minute delays for mobile app recommendations.
125
165
126
166
<details>
127
167
<summary>Show answer</summary>
128
168
129
-
**Deploy two separate models: a real-time endpoint for the mobile app and a batch endpoint for the email campaign.**
169
+
✅ **Correct answer: A.**Deploy two separate models: a real-time endpoint for the mobile app and a batch endpoint for the email campaign.
130
170
131
171
The two scenarios have different latency and throughput needs. A real-time (online) endpoint delivers instant recommendations in the mobile app, while a batch endpoint efficiently scores 2 million customers for the weekly email. Forcing one endpoint type to handle both leads to either excessive cost or unacceptable latency.
132
172
133
173
</details>
134
174
135
175
**4. What compute resource would be most appropriate for training the initial Contoso Retail recommendation model, given the 2 million customer dataset and budget constraints?**
136
176
137
-
- Start with CPU general-purpose compute, monitor performance, and scale to memory-optimized or GPU if needed.
138
-
- Immediately provision the largest GPU memory-optimized compute to ensure fast training.
139
-
- Use only local development machines to minimize Azure costs.
177
+
-**A.**Start with CPU general-purpose compute, monitor performance, and scale to memory-optimized or GPU if needed.
178
+
-**B.**Immediately provision the largest GPU memory-optimized compute to ensure fast training.
179
+
-**C.**Use only local development machines to minimize Azure costs.
140
180
141
181
<details>
142
182
<summary>Show answer</summary>
143
183
144
-
**Start with CPU general-purpose compute, monitor performance, and scale to memory-optimized or GPU if needed.**
184
+
✅ **Correct answer: A.**Start with CPU general-purpose compute, monitor performance, and scale to memory-optimized or GPU if needed.
145
185
146
186
With a limited initial budget, start with cost-effective CPU general-purpose compute and scale up only when the model complexity or training time justifies it. Provisioning the largest GPU upfront wastes budget, and local machines can't handle the scale of 2 million customers and millions of interactions.
147
187
148
188
</details>
189
+
190
+
## Key takeaways
191
+
192
+
-**Consolidate data before training.** Diverse sources and formats should flow through an automated pipeline into a single, training-ready storage layer.
193
+
-**Match the service to your team and scale.** Choose the platform that fits your team's existing skills and your data volume rather than the most powerful option.
194
+
-**Start small on compute and scale up.** Begin with cost-effective compute and add memory-optimized or GPU resources only when the workload demands it.
195
+
-**Design deployment around how predictions are consumed.** Use real-time endpoints for instant responses and batch endpoints for high-volume scheduled scoring.
0 commit comments