Skip to content

Add python telemetry hopper to land telemetry data in Azure Monitor#249

Merged
aditjha-msft merged 18 commits into
mainfrom
user/aditjha/pythonAzureMonitor
Jun 13, 2025
Merged

Add python telemetry hopper to land telemetry data in Azure Monitor#249
aditjha-msft merged 18 commits into
mainfrom
user/aditjha/pythonAzureMonitor

Conversation

@aditjha-msft
Copy link
Copy Markdown
Contributor

@aditjha-msft aditjha-msft commented May 27, 2025

This PR introduces the Python telemetry hopper which will be used in the container scenario to forward telemetry from image customizer to Azure Monitor. This approach is needed for a variety of reasons, most importantly because Azure Monitor doesn't provide a Go SDK for instrumentation, and other solutions come with security and auth challenges. See design doc here: Image Customizer OTel Integration.docx
This will work alongside PR: #241

if --disable-telemetry is passed in to the docker command, the telemetry hopper will not be started and collection and export will be disabled:

Telemetry collection in the container will only be enabled if explicitly provided the ENABLE_TELEMETRY environment variable. Otherwise telemetry is disabled by default.

 docker run \
   --rm \
   --privileged=true \
   -v /dev:/dev \
   -v "$HOME/staging:/mnt/staging:z" \
   mcr.microsoft.com/azurelinux/imagecustomizer:0.13.0 \
   imagecustomizer \
     --disable-telemetry
     --image-file "/mnt/staging/image.vhdx" \
     --config-file "/mnt/staging/image-config.yaml" \
     --build-dir "/build" \
     --output-image-format "vhdx" \
     --output-image-file "/mnt/staging/out/image.vhdx"

Checklist

  • Tests added/updated
  • Documentation updated (if needed)
  • Code conforms to style guidelines

@aditjha-msft aditjha-msft force-pushed the user/aditjha/pythonAzureMonitor branch 2 times, most recently from ee91462 to f866cfc Compare May 29, 2025 08:39
@aditjha-msft aditjha-msft marked this pull request as ready for review May 29, 2025 18:49
@aditjha-msft aditjha-msft requested a review from a team as a code owner May 29, 2025 18:49
Comment thread toolkit/scripts/telemetry_hopper/telemetry_hopper.py Outdated
aditjha-msft added a commit that referenced this pull request May 30, 2025
This PR introduces support for using the OTEL Go sdk to instrument the
tool using OTEL standards and practices. It provides initialization
logic needed for a global tracer to be used across the tool's execution.
For starters, the PR adds a customization time span, with more telemetry
to be added in follow up PRs that help with data collection tied to the
tool's KPIs.

The PR also adds logic to disable telemetry, and uses a no-op tracer
when `--disable-telemetry` is passed in. This follows OTEL standardized
method of disabling telemetry collection in a clean manner.

As part of telemetry initialization, the `telemetry.go` file has been
added to set up the OTEL tracer and the OTLP grpc exporter. The exporter
will send data to a Python service listening at the local port, which
will then forward the data to Azure Monitor.

In the effort of splitting up PRs in chunks, the PR for the complement
Python telemetry service is here:
#249.

---

### **Checklist**
- [x] Tests added/updated
- [x] Documentation updated (if needed)
- [x] Code conforms to style guidelines
@aditjha-msft aditjha-msft force-pushed the user/aditjha/pythonAzureMonitor branch from 7b45fd3 to 2a3dc66 Compare May 31, 2025 00:12
@aditjha-msft aditjha-msft requested a review from a team May 31, 2025 00:15
Comment thread toolkit/scripts/telemetry_hopper/requirements.txt Outdated
Comment thread toolkit/tools/go.mod Outdated
Comment thread toolkit/tools/imagecustomizer/container/imagecustomizer.Dockerfile Outdated
Comment thread toolkit/tools/imagecustomizer/container/imagecustomizer.Dockerfile Outdated
Comment thread toolkit/tools/imagecustomizer/container/entrypoint.sh Outdated
Comment thread toolkit/scripts/telemetry_hopper/telemetry_hopper.py Outdated
Comment thread toolkit/scripts/telemetry_hopper/telemetry_hopper.py
Comment thread toolkit/tools/imagecustomizer/container/entrypoint.sh
Comment thread toolkit/tools/imagecustomizer/container/entrypoint.sh
Copy link
Copy Markdown
Contributor

@romoh romoh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left comments

Comment thread toolkit/tools/imagecustomizer/container/imagecustomizer.Dockerfile Outdated
@aditjha-msft aditjha-msft requested a review from romoh June 3, 2025 23:05
Comment thread toolkit/tools/internal/telemetry/telemetry.go
@aditjha-msft aditjha-msft requested a review from cwize1 June 5, 2025 22:17
Comment thread toolkit/tools/internal/telemetry/telemetry.go Outdated
Comment thread toolkit/tools/imagecustomizer/container/imagecustomizer.Dockerfile Outdated
@aditjha-msft aditjha-msft requested a review from cwize1 June 6, 2025 00:08
Comment thread toolkit/scripts/telemetry_hopper/telemetry_hopper.py Outdated
Comment thread toolkit/scripts/telemetry_hopper/telemetry_hopper.py Outdated
Comment thread toolkit/scripts/telemetry_hopper/telemetry_hopper.py
Comment thread toolkit/scripts/telemetry_hopper/telemetry_hopper.py Outdated
@aditjha-msft aditjha-msft requested a review from gmileka June 11, 2025 21:20
Comment thread toolkit/scripts/telemetry_hopper/telemetry_hopper.py Outdated
Comment thread toolkit/tools/imagecustomizer/container/imagecustomizer.Dockerfile Outdated
@aditjha-msft aditjha-msft requested a review from cwize1 June 13, 2025 18:16
@aditjha-msft aditjha-msft requested review from romoh and removed request for gmileka and romoh June 13, 2025 18:24
@aditjha-msft aditjha-msft merged commit e60f862 into main Jun 13, 2025
8 checks passed
@aditjha-msft aditjha-msft deleted the user/aditjha/pythonAzureMonitor branch June 13, 2025 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants