feat: add Edge prediction pattern

README.md

@@ -56,7 +56,7 @@ The serving patterns are a series of system designs for using machine learning m
 
 - [Serving template pattern](./Serving-patterns/Serving-template-pattern/design_en.md)
 
-- Edge prediction pattern: To do
+- [Edge prediction pattern](./Serving-patterns/Edge-prediction-pattern/design_en.md)
 
 - [Antipatterns](./Serving-patterns/Anti-patterns/README.md)
 

Serving-patterns/Edge-prediction-pattern/design_en.md

@@ -0,0 +1,34 @@
+# Edge prediction pattern
+
+## Usecase
+- When you need real-time or near-real-time prediction on a device with limited or no network connectivity.
+- When sending data to a cloud server is not feasible due to bandwidth, latency, cost or privacy constraints.
+- When you want to run inference on smartphones, IoT devices, cameras or embedded systems.
+
+## Architecture
+The edge prediction pattern runs a machine learning model directly on the end device, instead of sending data to a cloud or on-premise server for inference. The model is trained in the cloud or on a training server, and then optimized and distributed to edge devices for local inference. Since the inference runs on the device itself, it achieves low latency and can operate without network connectivity.<br>
+To deploy a model on an edge device, you typically need to optimize it for the target hardware. Common optimization techniques include quantization (reducing numerical precision from float32 to int8), pruning (removing unnecessary connections in neural network) and conversion to an edge-optimized format such as TensorFlow Lite, ONNX Runtime or OpenVINO. These optimizations reduce the model size and inference latency, often with minimal impact on accuracy.<br>
+The optimized model is distributed to edge devices through a model registry or OTA (Over-The-Air) update mechanism. The device downloads the model, loads it into an inference runtime, and serves predictions locally. Optionally, the device may send prediction logs and telemetry back to the cloud for monitoring and retraining purposes.<br>
+There are several tiers of edge deployment depending on the device capability. Micro edge devices, such as microcontrollers and sensors, run very small models for simple tasks. Thin edge devices, such as smartphones and smart cameras, can run more complex models including image classification and object detection. Thick edge devices, such as edge servers, can run multiple models simultaneously and may even support on-device training. You should select the optimization strategy and model complexity based on the target device tier.
+
+## Diagram
+![diagram](diagram.png)
+
+## Pros
+- Low latency inference since the prediction runs locally on the device.
+- Can operate offline or with intermittent network connectivity.
+- Reduces bandwidth cost by not transferring raw data to the cloud.
+- Enhances data privacy by keeping sensitive data on the device.
+
+## Cons
+- Limited compute, memory and storage on edge devices constrains model size and complexity.
+- Model update and versioning across a fleet of devices requires OTA update infrastructure.
+- Harder to monitor and debug compared to cloud-based serving.
+- Device fragmentation may require different model builds for different hardware.
+
+## Needs consideration
+- Model optimization strategy: quantization level, pruning ratio and target format should be selected based on the device hardware and accuracy requirements.
+- Model distribution and update mechanism: OTA updates with version management, rollback capability and delta updates to minimize bandwidth.
+- Monitoring and telemetry: collecting inference logs and performance metrics from devices to detect model drift and issues.
+- Fallback strategy: consider routing requests to cloud prediction if the edge model fails or encounters unsupported input.
+- Security: model files on the device should be protected from tampering, and OTA updates should be signed and verified.