Skip to content

Commit 4af20d0

Browse files
jlianvipellerggjjjmachavarapun
authored
VS Code extensions release (#161)
**What changed** - Restructured WASM samples to match the VS Code extension layout: graphs under wasm, Rust operators under operators, Python operators under operators. - Updated builder assets and GitHub Actions to point to new paths; refreshed Rust and Python build instructions and README links. - Added schema/state-store scenario placeholders and dataflow graph files alongside operators and sample data. - Fixed the WASM development doc copy to align WIT schema links, builder links, example paths, and Python build steps with the new structure. **Testing** - Docker build: Python map operator from map using `python-wasm-builder` (produced `bin/x86_64/release/map.wasm`). - Docker build: Rust operator previously validated from an operator directory. **Notes** - No API/flag changes—only path/layout updates. --------- Co-authored-by: vipeller <51135538+vipeller@users.noreply.github.com> Co-authored-by: Gayathri Jegan Mohan <33068020+ggjjj@users.noreply.github.com> Co-authored-by: Nagaraju Machavarapu <machavarapun@microsoft.com>
1 parent 008a8ab commit 4af20d0

115 files changed

Lines changed: 2922 additions & 2216 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ghcr_graph.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ jobs:
5858
run: |
5959
owner=$(echo "${{ github.repository_owner }}" | tr '[:upper:]' '[:lower:]')
6060
61-
cd samples/wasm/rust
61+
cd samples/wasm
6262
6363
if [ -n "${{ inputs.graph-name }}" ]; then
6464
echo "Pushing specific graph: ${{ inputs.graph-name }}"

.github/workflows/ghcr_module.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ jobs:
6565
6666
source $HOME/.cargo/env
6767
68-
cd samples/wasm/rust
68+
cd samples/wasm
6969
make graphs
7070
7171
if [ -n "${{ inputs.module-name }}" ]; then

.github/workflows/wasm-python-build.yml

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,15 @@ on:
55
push:
66
branches: [ main, wasm ]
77
paths:
8-
- 'samples/wasm/python/Dockerfile'
9-
- 'samples/wasm/python/schema/**'
10-
- 'samples/wasm/python/inject_pdb.py'
8+
- 'samples/wasm-python/Dockerfile'
9+
- 'samples/wasm-python/schema/**'
10+
- 'samples/wasm-python/inject_pdb.py'
1111
pull_request:
1212
branches: [ main ]
1313
paths:
14-
- 'samples/wasm/python/Dockerfile'
15-
- 'samples/wasm/python/schema/**'
16-
- 'samples/wasm/python/inject_pdb.py'
14+
- 'samples/wasm-python/Dockerfile'
15+
- 'samples/wasm-python/schema/**'
16+
- 'samples/wasm-python/inject_pdb.py'
1717

1818
jobs:
1919
build-and-push:
@@ -49,8 +49,8 @@ jobs:
4949
- name: Build and push Docker image
5050
uses: docker/build-push-action@v5
5151
with:
52-
context: samples/wasm/python
53-
file: samples/wasm/python/Dockerfile
52+
context: samples/wasm-python
53+
file: samples/wasm-python/Dockerfile
5454
push: true
5555
tags: ${{ steps.meta.outputs.tags }}
5656
labels: ${{ steps.meta.outputs.labels }}

.github/workflows/wasm-rust-build.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,11 @@ on:
55
push:
66
branches: [ main, wasm ]
77
paths:
8-
- 'samples/wasm/rust/Dockerfile'
8+
- 'samples/wasm/Dockerfile'
99
pull_request:
1010
branches: [ main ]
1111
paths:
12-
- 'samples/wasm/rust/Dockerfile'
12+
- 'samples/wasm/Dockerfile'
1313

1414
jobs:
1515
build-and-push:
@@ -45,8 +45,8 @@ jobs:
4545
- name: "Build and Push Rust Builder Image"
4646
uses: docker/build-push-action@v5
4747
with:
48-
context: samples/wasm/rust
49-
file: samples/wasm/rust/Dockerfile
48+
context: samples/wasm
49+
file: samples/wasm/Dockerfile
5050
push: true
5151
tags: ${{ steps.meta.outputs.tags }}
5252
labels: ${{ steps.meta.outputs.labels }}

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -421,3 +421,4 @@ target
421421
# and can be added to the global gitignore or merged into this file. For a more nuclear
422422
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
423423
#.idea/
424+
install.sh
Lines changed: 20 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,13 @@
1-
# Python WASM Operators Development Guide
1+
# Python WASM Operators (VS Code extension ready)
22

3-
This guide provides comprehensive documentation for developing WebAssembly (WASM) operators using Python for Azure IoT Operations data flow graphs.
3+
This guide shows how to build and run Python-based WASM operators for Azure IoT Operations. The folder follows the VS Code extension layout: operators live under `operators/`, and builds drop artifacts into each operator’s `bin/` directory.
44

55
## Table of Contents
66

77
- [Overview](#overview)
88
- [Prerequisites](#prerequisites)
99
- [Development Environment Setup](#development-environment-setup)
10+
- [Build Options](#build-options)
1011
- [Python Operator Development](#python-operator-development)
1112
- [Building Python WASM Operators](#building-python-wasm-operators)
1213
- [Available Operator Templates](#available-operator-templates)
@@ -82,8 +83,19 @@ sudo apt-get update
8283
sudo apt-get install -y clang lld musl-dev git perl make cmake
8384
```
8485

86+
## Build Options
87+
88+
You can build operators three ways:
89+
- **VS Code extension (recommended):** Open `samples/wasm-python``Ctrl+Shift+P`**Azure IoT Operations: Build All Data Flow Operators** → pick **release** or **debug**. Output: `operators/<name>/bin/<arch>/<mode>/`.
90+
- **Docker builder:** Use the published builder image (see [Building Python WASM Operators](#building-python-wasm-operators)). Run it from an operator directory; output goes to that operator’s `bin/` folder.
91+
- **Local componentize-py:** Install prerequisites and run `componentize-py` directly (see [Build WASM Component](#build-wasm-component)).
92+
93+
To run any module you need a graph YAML that references the built operator (for example `graph.dataflow.yaml`). Keep the graph file alongside the operators when running locally or with the extension.
94+
8595
## Python Operator Development
8696

97+
Operators live under `operators/`. The sections below cover the interfaces and workflow.
98+
8799
### Understanding the Interface
88100

89101
Python operators implement specific interfaces defined by the WebAssembly Interface Types (WIT). Unlike Rust, Python doesn't have a high-level SDK, so you work directly with the exported function interfaces.
@@ -239,7 +251,7 @@ componentize-py -d ./schema/ -w map-impl componentize map_debug -o my-operator-d
239251

240252
## Building Python WASM Operators
241253

242-
### Using the Streamlined Docker Builder
254+
### Using the streamlined Docker builder
243255

244256
The recommended approach uses a single Docker command with the built-in builder:
245257

@@ -286,12 +298,12 @@ bin/x86_64/debug/my-operator.wasm
286298
bin/x86_64/debug/my-operator_debug.py
287299
```
288300

289-
### Build the Docker Builder (One-time Setup)
301+
### Build the Docker builder (one-time setup)
290302

291303
The recommended approach is to use the published images from GitHub Container Registry. However, if you need to build the Docker builder image locally:
292304

293305
```bash
294-
cd /path/to/samples/wasm/python
306+
cd /path/to/samples/wasm-python
295307
docker build -t python-wasm-builder .
296308
```
297309

@@ -302,17 +314,17 @@ docker build -t python-wasm-builder .
302314
The repository includes templates for all supported operator types:
303315

304316
### Map Operator
305-
- **Path**: `examples/map/`
317+
- **Path**: `operators/map/`
306318
- **Purpose**: Transform individual data items
307319
- **Interface**: `process(timestamp, input) -> output`
308320

309321
### Filter Operator
310-
- **Path**: `examples/filter/`
322+
- **Path**: `operators/filter/`
311323
- **Purpose**: Allow/reject data based on conditions
312324
- **Interface**: `process(timestamp, input) -> bool`
313325

314326
### Branch Operator
315-
- **Path**: `examples/branch/`
327+
- **Path**: `operators/branch/`
316328
- **Purpose**: Route data to different output paths
317329
- **Interface**: `process(timestamp, input) -> int`
318330

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
import branch_impl;
2+
import payload
3+
from branch_impl import exports
4+
from branch_impl import imports
5+
from branch_impl.imports import types
6+
from payload import Payload
7+
8+
9+
def to_bytes(payload_variant):
10+
if isinstance(payload_variant, types.BufferOrBytes_Buffer):
11+
return payload_variant.value.read()
12+
if isinstance(payload_variant, types.BufferOrBytes_Bytes):
13+
return payload_variant.value
14+
raise ValueError("Unexpected payload type")
15+
16+
class Branch(exports.Branch):
17+
def init(self, configuration) -> bool:
18+
imports.logger.log(imports.logger.Level.INFO, "module3/branch", "Init invoked")
19+
return True
20+
21+
def process(self, timestamp: int, input: types.DataModel) -> bool:
22+
imports.logger.log(imports.logger.Level.INFO, "module3/branch", "processing from python")
23+
24+
if not isinstance(input, types.DataModel_Message):
25+
raise ValueError("Unexpected input type: Expected DataModel_Message")
26+
27+
message = input.value
28+
payload = to_bytes(message.payload)
29+
decoded = payload.decode("utf-8")
30+
p = Payload(decoded)
31+
32+
if p.is_temperature():
33+
imports.logger.log(imports.logger.Level.INFO, "module3/branch", "temperature")
34+
return False # first branch
35+
36+
imports.logger.log(imports.logger.Level.INFO, "module3/branch", "humidity")
37+
return True # second branch
File renamed without changes.

0 commit comments

Comments
 (0)