Azure Exclude URL Errors

[mike-england]

Describe your environment
I've been having problem where excluded URLs are showing up as a kind of phantom error in azure application insights. I've opened a bug under the azure sdk for python project but they suggested I open one here. That ticket is:

Azure/azure-sdk-for-python#34616

Basically, my telemetry is reporting correctly, but I've excluded certain URLs, mostly because kubernetes constantly pings them as a health check and when I do that they report as a failed request with no response code.

Steps to reproduce

I'm going to include some sample code in a docker file that you can try:

app.py:

from azure.monitor.opentelemetry import configure_azure_monitor

configure_azure_monitor()

import flask

app = flask.Flask(__name__)

@app.route("/")
def test():
    print('got one')
    return "Test flask request"

@app.route("/health")
def health():
    print('health endpoint')
    return "Health request"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

Dockerfile:

FROM python:3.10-slim

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

COPY requirements.txt .
RUN python -m pip install -r requirements.txt

WORKDIR /app
COPY . /app

RUN adduser -u 5678 --disabled-password --gecos "" appuser && chown -R appuser /app
USER appuser

CMD ["python", "app.py"]
requirements.txt:

Flask==3.0.0
azure-monitor-opentelemetry==1.3.0

And to run it:

docker build -t azuretelemetry . && docker run --rm -p 8080:8080 -e APPLICATIONINSIGHTS_CONNECTION_STRING=$APPLICATIONINSIGHTS_CONNECTION_STRING -e OTEL_EXPERIMENTAL_RESOURCE_DETECTORS=$OTEL_EXPERIMENTAL_RESOURCE_DETECTORS -e OTEL_PYTHON_EXCLUDED_URLS=$OTEL_PYTHON_EXCLUDED_URLS azuretelemetry

Where OTEL_PYTHON_EXCLUDED_URLS="health,swagger.*"

What is the expected behavior?
The transactions don't show up in telemetry at all

What is the actual behavior?
Transactions show up as errors

Additional context
Azure/azure-sdk-for-python#34616

[lzchen]

I am not able to replicate your behavior in a local environment. Same code as yours but setting the environment variable manually and in a python script.

import os

from azure.monitor.opentelemetry import configure_azure_monitor

os.environ["OTEL_PYTHON_EXCLUDED_URLS"] = "health"

configure_azure_monitor()

import flask

app = flask.Flask(__name__)

@app.route("/")
def test():
    print('got one')
    return "Test flask request"

@app.route("/health")
def health():
    print('health endpoint')
    return "Health request"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

Might be an issue of how you are capturing environment variables in your docker image. Could you verify that the environment variable is indeed read properly and passed into the code as expected?

[mike-england]

Yes, I can confirm that the environment variables are being set. If I remove the exclude, everything works fine, and maybe that's my "solution" for now, other than I'll show thousands of kubernetes heartbeat messages.

[rads-1996]

Upon further investigation, it was discovered that HTTP_SERVER_REQUEST_DURATION metrics were being collected in the flask instrumentation for excluded URLs which were flagged as failed requests. Implementing a fix for it in PR - #3794