Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ DRF API Logger automatically captures and stores comprehensive API information:
- **📈 Analytics**: Built-in admin dashboard with charts and performance metrics
- **🔧 Highly Configurable**: Extensive filtering and customization options
- **🔬 API Profiling**: Per-request latency breakdown with auto-diagnosis (SQL, middleware, business logic)
- **Request Correlation**: Opt-in request IDs, traceparent parsing, route metadata, logging context, and signal metadata without new database columns

### 🌐 Community & Support

Expand Down Expand Up @@ -434,6 +435,74 @@ def my_api_view(request):
return Response({'status': 'ok'})
```

### Request Correlation

Enable correlation when you need to connect DRF API Logger events with
application logs, upstream gateway request IDs, distributed traces, or metrics
labels without changing the log table schema.

```python
DRF_API_LOGGER_ENABLE_CORRELATION = True
DRF_API_LOGGER_CORRELATION_REQUEST_ID_HEADERS = ["X-Request-ID", "X-Correlation-ID"]
DRF_API_LOGGER_CORRELATION_TRACE_ID_HEADERS = ["traceparent", "X-Trace-ID"]
DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT = True
```

Correlation is intentionally not persisted to `APILogsModel`. It does not add
model fields, migrations, admin columns, database indexes, or synthetic payload
fields to queued database log rows. When enabled, metadata is available through:

- `request.api_logger_correlation`
- `request.api_logger_low_cardinality`
- `request.api_logger_request_id`
- `request.api_logger_trace_id`
- `drf_api_logger.logging_context.get_correlation_context()`
- signal payload keys: `correlation` and `low_cardinality`

Example signal listener:

```python
from drf_api_logger import API_LOGGER_SIGNAL

def forward_to_metrics(**kwargs):
labels = kwargs.get("low_cardinality", {})
correlation = kwargs.get("correlation", {})
metrics.count(
"drf_api_logger.request",
tags={
"route": labels.get("route"),
"status_class": labels.get("status_class"),
},
)
logger.info(
"api request observed",
extra={
"request_id": correlation.get("request_id"),
"trace_id": correlation.get("trace_id"),
},
)

API_LOGGER_SIGNAL.listen += forward_to_metrics
```

Add opaque, non-sensitive context values through an allowlisted callback:

```python
DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC = "myapp.logging.api_logger_context"

def api_logger_context(request):
return {
"actor_id": getattr(request.user, "pk", None),
"tenant_id": getattr(request, "tenant_id", None),
"api_consumer_id": getattr(request, "api_consumer_id", None),
"client_id": getattr(request, "client_id", None),
}
```

Only `actor_id`, `tenant_id`, `api_consumer_id`, and `client_id` are accepted
from the callback. Use opaque IDs, not names, emails, tokens, or other
identifying values.

## 📊 Programmatic Access

### Querying Log Data
Expand Down
20 changes: 20 additions & 0 deletions docs/compliance.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,26 @@ records before they enter the background queue:

Returning ``None`` intentionally drops that log entry.

Request Correlation Controls
----------------------------

Request correlation is disabled by default. When
``DRF_API_LOGGER_ENABLE_CORRELATION`` is enabled, request IDs, trace IDs, route
metadata, and allowlisted opaque context are exposed through request attributes,
logging context, and signal payloads. The package does not persist correlation
metadata to ``APILogsModel`` and does not add migrations, model fields, admin
columns, database indexes, or synthetic database payload fields.

For privacy-sensitive deployments:

- Treat ``request_id`` and ``trace_id`` as high-cardinality operational IDs.
- Send only route, URL name, namespace, app name, and status class to metrics
labels through ``low_cardinality``.
- Use opaque values for ``actor_id``, ``tenant_id``, ``api_consumer_id``, and
``client_id``.
- Do not return names, emails, tokens, session IDs, or regulated identifiers
from ``DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC``.

Storage Controls
----------------

Expand Down
96 changes: 96 additions & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ Key Features
- Database logging and/or real-time signal notifications
- Built-in admin dashboard with charts and performance metrics
- Per-request API profiling with auto-diagnosis of bottlenecks
- Request correlation through request attributes, logging context, and signals


Supported Versions
Expand Down Expand Up @@ -176,6 +177,8 @@ Signal data structure:
'tracing_id': 'uuid4-string', # if tracing enabled
'profiling_data': { ... }, # if profiling enabled
'sql_query_count': 5, # if profiling enabled
'correlation': { ... }, # if correlation enabled
'low_cardinality': { ... }, # if correlation enabled
}


Expand Down Expand Up @@ -347,6 +350,26 @@ Configuration Reference
- str
- ``None``
- Header name to read tracing ID from
* - ``DRF_API_LOGGER_ENABLE_CORRELATION``
- bool
- ``False``
- Enable request correlation metadata without adding database columns
* - ``DRF_API_LOGGER_CORRELATION_REQUEST_ID_HEADERS``
- list
- ``['X-Request-ID', 'X-Correlation-ID']``
- Headers checked for inbound request IDs
* - ``DRF_API_LOGGER_CORRELATION_TRACE_ID_HEADERS``
- list
- ``['traceparent', 'X-Trace-ID']``
- Headers checked for W3C traceparent or trace IDs
* - ``DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC``
- str
- ``None``
- Dotted-path callback returning allowlisted opaque context IDs
* - ``DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT``
- bool
- ``False``
- Expose correlation metadata through a ContextVar during the view call
* - ``DRF_API_LOGGER_CUSTOM_HANDLER``
- str
- ``None``
Expand Down Expand Up @@ -440,6 +463,79 @@ Read tracing ID from request header:
DRF_API_LOGGER_TRACING_ID_HEADER_NAME = 'X-Trace-ID'


Request Correlation
===================

Enable request correlation to connect API logger events with application logs,
upstream gateways, distributed traces, or metrics labels:

.. code-block:: python

DRF_API_LOGGER_ENABLE_CORRELATION = True
DRF_API_LOGGER_CORRELATION_REQUEST_ID_HEADERS = ["X-Request-ID", "X-Correlation-ID"]
DRF_API_LOGGER_CORRELATION_TRACE_ID_HEADERS = ["traceparent", "X-Trace-ID"]
DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT = True

Correlation metadata is intentionally not stored in ``APILogsModel``. Enabling
it does not add model fields, migrations, admin columns, database indexes, or
synthetic fields in queued database log rows.

When enabled, request handlers can read:

- ``request.api_logger_correlation``
- ``request.api_logger_low_cardinality``
- ``request.api_logger_request_id``
- ``request.api_logger_trace_id``
- ``drf_api_logger.logging_context.get_correlation_context()``

Signal listeners receive ``correlation`` and ``low_cardinality`` keys. Use the
low-cardinality dictionary for metrics tags such as route, URL name, and status
class. Keep high-cardinality request and trace IDs in logs or trace systems, not
metrics labels.

.. code-block:: python

from drf_api_logger import API_LOGGER_SIGNAL

def forward_api_event(**kwargs):
labels = kwargs.get("low_cardinality", {})
correlation = kwargs.get("correlation", {})
metrics.count(
"drf_api_logger.request",
tags={
"route": labels.get("route"),
"status_class": labels.get("status_class"),
},
)
app_logger.info(
"api request observed",
extra={
"request_id": correlation.get("request_id"),
"trace_id": correlation.get("trace_id"),
},
)

API_LOGGER_SIGNAL.listen += forward_api_event

Add opaque, non-sensitive context through an allowlisted callback:

.. code-block:: python

DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC = "myapp.logging.api_logger_context"

def api_logger_context(request):
return {
"actor_id": getattr(request.user, "pk", None),
"tenant_id": getattr(request, "tenant_id", None),
"api_consumer_id": getattr(request, "api_consumer_id", None),
"client_id": getattr(request, "client_id", None),
}

Only ``actor_id``, ``tenant_id``, ``api_consumer_id``, and ``client_id`` are
accepted from the callback. Do not return names, emails, tokens, or other
identifying values.


Path Configuration
==================

Expand Down
18 changes: 18 additions & 0 deletions docs/operations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,24 @@ Important fields:
``inserted_count``
Number of rows successfully inserted by the current worker.

Request Correlation Operations
------------------------------

``DRF_API_LOGGER_ENABLE_CORRELATION`` is designed for joining DRF API Logger
events with application logs, traces, and metrics without changing database
storage. It does not add model fields, migrations, admin columns, or database
indexes, and queued database log rows keep the existing payload shape.

Recommended operating pattern:

- Send ``low_cardinality`` signal metadata to metrics labels, such as route,
URL name, and status class.
- Keep high-cardinality ``request_id`` and ``trace_id`` values in logs or trace
systems, not metrics label sets.
- Use ``DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT`` when application log records
inside the view should include the same request correlation metadata.
- Return only opaque IDs from ``DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC``.

Database Indexes
----------------

Expand Down
51 changes: 51 additions & 0 deletions docs/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,57 @@ Or accept an upstream request ID:
DRF_API_LOGGER_ENABLE_TRACING = True
DRF_API_LOGGER_TRACING_ID_HEADER_NAME = "X-Request-ID"

Request Correlation Without New DB Columns
------------------------------------------

Use correlation when the application needs request IDs, W3C ``traceparent``
trace IDs, route metadata, or metrics labels without adding fields to the
``drf_api_logs`` table.

.. code-block:: python

DRF_API_LOGGER_SIGNAL = True
DRF_API_LOGGER_ENABLE_CORRELATION = True
DRF_API_LOGGER_CORRELATION_REQUEST_ID_HEADERS = ["X-Request-ID", "X-Correlation-ID"]
DRF_API_LOGGER_CORRELATION_TRACE_ID_HEADERS = ["traceparent", "X-Trace-ID"]
DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT = True

Correlation data is exposed on the request object, in
``drf_api_logger.logging_context.get_correlation_context()``, and in signal
payload keys named ``correlation`` and ``low_cardinality``. It is not written to
``APILogsModel`` and does not require migrations, model fields, admin columns,
or database indexes.

.. code-block:: python

from drf_api_logger import API_LOGGER_SIGNAL

def send_api_metrics(**kwargs):
labels = kwargs.get("low_cardinality", {})
metrics.count(
"drf_api_logger.request",
tags={
"route": labels.get("route"),
"status_class": labels.get("status_class"),
},
)

API_LOGGER_SIGNAL.listen += send_api_metrics

For extra context, return only opaque IDs from an allowlisted callback:

.. code-block:: python

DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC = "myapp.logging.api_logger_context"

def api_logger_context(request):
return {
"actor_id": getattr(request.user, "pk", None),
"tenant_id": getattr(request, "tenant_id", None),
"api_consumer_id": getattr(request, "api_consumer_id", None),
"client_id": getattr(request, "client_id", None),
}

Retention and Pruning
---------------------

Expand Down
35 changes: 35 additions & 0 deletions docs/tutorials.rst
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,41 @@ Log the same ID in application code:
tracing_id = getattr(request, "tracing_id", "missing")
logger.info("example_view started", extra={"tracing_id": tracing_id})

Correlate Logs Without Persisting IDs
-------------------------------------

Use request correlation when application logs, gateway IDs, and metrics need to
line up, but the log table should keep its existing schema.

.. code-block:: python

DRF_API_LOGGER_SIGNAL = True
DRF_API_LOGGER_ENABLE_CORRELATION = True
DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT = True
DRF_API_LOGGER_CORRELATION_REQUEST_ID_HEADERS = ["X-Request-ID", "X-Correlation-ID"]
DRF_API_LOGGER_CORRELATION_TRACE_ID_HEADERS = ["traceparent", "X-Trace-ID"]

.. code-block:: python

from drf_api_logger import API_LOGGER_SIGNAL

def send_metrics(**kwargs):
labels = kwargs.get("low_cardinality", {})
metrics.count(
"drf_api_logger.request",
tags={
"route": labels.get("route"),
"status_class": labels.get("status_class"),
},
)

API_LOGGER_SIGNAL.listen += send_metrics

Correlation adds ``correlation`` and ``low_cardinality`` to signal payloads and
sets request attributes such as ``request.api_logger_request_id``. It does not
add model fields, migrations, admin columns, database indexes, or extra queued
database payload fields.

Community Snippets
------------------

Expand Down
Loading
Loading