Merge pull request #126 from vishalanandl177/codex/SCRUM-9-request-correlation-no-db

Add opt-in request correlation

Author: vishalanandl177

README.md

@@ -65,6 +65,7 @@ DRF API Logger automatically captures and stores comprehensive API information:
 - **📈 Analytics**: Built-in admin dashboard with charts and performance metrics
 - **🔧 Highly Configurable**: Extensive filtering and customization options
 - **🔬 API Profiling**: Per-request latency breakdown with auto-diagnosis (SQL, middleware, business logic)
+- **Request Correlation**: Opt-in request IDs, traceparent parsing, route metadata, logging context, and signal metadata without new database columns
 
 ### 🌐 Community & Support
 
@@ -434,6 +435,74 @@ def my_api_view(request):
     return Response({'status': 'ok'})
 ```
 
+### Request Correlation
+
+Enable correlation when you need to connect DRF API Logger events with
+application logs, upstream gateway request IDs, distributed traces, or metrics
+labels without changing the log table schema.
+
+```python
+DRF_API_LOGGER_ENABLE_CORRELATION = True
+DRF_API_LOGGER_CORRELATION_REQUEST_ID_HEADERS = ["X-Request-ID", "X-Correlation-ID"]
+DRF_API_LOGGER_CORRELATION_TRACE_ID_HEADERS = ["traceparent", "X-Trace-ID"]
+DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT = True
+```
+
+Correlation is intentionally not persisted to `APILogsModel`. It does not add
+model fields, migrations, admin columns, database indexes, or synthetic payload
+fields to queued database log rows. When enabled, metadata is available through:
+
+- `request.api_logger_correlation`
+- `request.api_logger_low_cardinality`
+- `request.api_logger_request_id`
+- `request.api_logger_trace_id`
+- `drf_api_logger.logging_context.get_correlation_context()`
+- signal payload keys: `correlation` and `low_cardinality`
+
+Example signal listener:
+
+```python
+from drf_api_logger import API_LOGGER_SIGNAL
+
+def forward_to_metrics(**kwargs):
+    labels = kwargs.get("low_cardinality", {})
+    correlation = kwargs.get("correlation", {})
+    metrics.count(
+        "drf_api_logger.request",
+        tags={
+            "route": labels.get("route"),
+            "status_class": labels.get("status_class"),
+        },
+    )
+    logger.info(
+        "api request observed",
+        extra={
+            "request_id": correlation.get("request_id"),
+            "trace_id": correlation.get("trace_id"),
+        },
+    )
+
+API_LOGGER_SIGNAL.listen += forward_to_metrics
+```
+
+Add opaque, non-sensitive context values through an allowlisted callback:
+
+```python
+DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC = "myapp.logging.api_logger_context"
+
+def api_logger_context(request):
+    return {
+        "actor_id": getattr(request.user, "pk", None),
+        "tenant_id": getattr(request, "tenant_id", None),
+        "api_consumer_id": getattr(request, "api_consumer_id", None),
+        "client_id": getattr(request, "client_id", None),
+    }
+```
+
+Only `actor_id`, `tenant_id`, `api_consumer_id`, and `client_id` are accepted
+from the callback. Use opaque IDs, not names, emails, tokens, or other
+identifying values.
+
 ## 📊 Programmatic Access
 
 ### Querying Log Data

docs/compliance.rst

@@ -61,6 +61,26 @@ records before they enter the background queue:
 
 Returning ``None`` intentionally drops that log entry.
 
+Request Correlation Controls
+----------------------------
+
+Request correlation is disabled by default. When
+``DRF_API_LOGGER_ENABLE_CORRELATION`` is enabled, request IDs, trace IDs, route
+metadata, and allowlisted opaque context are exposed through request attributes,
+logging context, and signal payloads. The package does not persist correlation
+metadata to ``APILogsModel`` and does not add migrations, model fields, admin
+columns, database indexes, or synthetic database payload fields.
+
+For privacy-sensitive deployments:
+
+- Treat ``request_id`` and ``trace_id`` as high-cardinality operational IDs.
+- Send only route, URL name, namespace, app name, and status class to metrics
+  labels through ``low_cardinality``.
+- Use opaque values for ``actor_id``, ``tenant_id``, ``api_consumer_id``, and
+  ``client_id``.
+- Do not return names, emails, tokens, session IDs, or regulated identifiers
+  from ``DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC``.
+
 Storage Controls
 ----------------
 

docs/index.rst

@@ -40,6 +40,7 @@ Key Features
 - Database logging and/or real-time signal notifications
 - Built-in admin dashboard with charts and performance metrics
 - Per-request API profiling with auto-diagnosis of bottlenecks
+- Request correlation through request attributes, logging context, and signals
 
 
 Supported Versions
@@ -176,6 +177,8 @@ Signal data structure:
        'tracing_id': 'uuid4-string',       # if tracing enabled
        'profiling_data': { ... },           # if profiling enabled
        'sql_query_count': 5,                # if profiling enabled
+       'correlation': { ... },              # if correlation enabled
+       'low_cardinality': { ... },          # if correlation enabled
    }
 
 
@@ -347,6 +350,26 @@ Configuration Reference
      - str
      - ``None``
      - Header name to read tracing ID from
+   * - ``DRF_API_LOGGER_ENABLE_CORRELATION``
+     - bool
+     - ``False``
+     - Enable request correlation metadata without adding database columns
+   * - ``DRF_API_LOGGER_CORRELATION_REQUEST_ID_HEADERS``
+     - list
+     - ``['X-Request-ID', 'X-Correlation-ID']``
+     - Headers checked for inbound request IDs
+   * - ``DRF_API_LOGGER_CORRELATION_TRACE_ID_HEADERS``
+     - list
+     - ``['traceparent', 'X-Trace-ID']``
+     - Headers checked for W3C traceparent or trace IDs
+   * - ``DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC``
+     - str
+     - ``None``
+     - Dotted-path callback returning allowlisted opaque context IDs
+   * - ``DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT``
+     - bool
+     - ``False``
+     - Expose correlation metadata through a ContextVar during the view call
    * - ``DRF_API_LOGGER_CUSTOM_HANDLER``
      - str
      - ``None``
@@ -440,6 +463,79 @@ Read tracing ID from request header:
    DRF_API_LOGGER_TRACING_ID_HEADER_NAME = 'X-Trace-ID'
 
 
+Request Correlation
+===================
+
+Enable request correlation to connect API logger events with application logs,
+upstream gateways, distributed traces, or metrics labels:
+
+.. code-block:: python
+
+   DRF_API_LOGGER_ENABLE_CORRELATION = True
+   DRF_API_LOGGER_CORRELATION_REQUEST_ID_HEADERS = ["X-Request-ID", "X-Correlation-ID"]
+   DRF_API_LOGGER_CORRELATION_TRACE_ID_HEADERS = ["traceparent", "X-Trace-ID"]
+   DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT = True
+
+Correlation metadata is intentionally not stored in ``APILogsModel``. Enabling
+it does not add model fields, migrations, admin columns, database indexes, or
+synthetic fields in queued database log rows.
+
+When enabled, request handlers can read:
+
+- ``request.api_logger_correlation``
+- ``request.api_logger_low_cardinality``
+- ``request.api_logger_request_id``
+- ``request.api_logger_trace_id``
+- ``drf_api_logger.logging_context.get_correlation_context()``
+
+Signal listeners receive ``correlation`` and ``low_cardinality`` keys. Use the
+low-cardinality dictionary for metrics tags such as route, URL name, and status
+class. Keep high-cardinality request and trace IDs in logs or trace systems, not
+metrics labels.
+
+.. code-block:: python
+
+   from drf_api_logger import API_LOGGER_SIGNAL
+
+   def forward_api_event(**kwargs):
+       labels = kwargs.get("low_cardinality", {})
+       correlation = kwargs.get("correlation", {})
+       metrics.count(
+           "drf_api_logger.request",
+           tags={
+               "route": labels.get("route"),
+               "status_class": labels.get("status_class"),
+           },
+       )
+       app_logger.info(
+           "api request observed",
+           extra={
+               "request_id": correlation.get("request_id"),
+               "trace_id": correlation.get("trace_id"),
+           },
+       )
+
+   API_LOGGER_SIGNAL.listen += forward_api_event
+
+Add opaque, non-sensitive context through an allowlisted callback:
+
+.. code-block:: python
+
+   DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC = "myapp.logging.api_logger_context"
+
+   def api_logger_context(request):
+       return {
+           "actor_id": getattr(request.user, "pk", None),
+           "tenant_id": getattr(request, "tenant_id", None),
+           "api_consumer_id": getattr(request, "api_consumer_id", None),
+           "client_id": getattr(request, "client_id", None),
+       }
+
+Only ``actor_id``, ``tenant_id``, ``api_consumer_id``, and ``client_id`` are
+accepted from the callback. Do not return names, emails, tokens, or other
+identifying values.
+
+
 Path Configuration
 ==================
 

docs/operations.rst

@@ -110,6 +110,24 @@ Important fields:
 ``inserted_count``
    Number of rows successfully inserted by the current worker.
 
+Request Correlation Operations
+------------------------------
+
+``DRF_API_LOGGER_ENABLE_CORRELATION`` is designed for joining DRF API Logger
+events with application logs, traces, and metrics without changing database
+storage. It does not add model fields, migrations, admin columns, or database
+indexes, and queued database log rows keep the existing payload shape.
+
+Recommended operating pattern:
+
+- Send ``low_cardinality`` signal metadata to metrics labels, such as route,
+  URL name, and status class.
+- Keep high-cardinality ``request_id`` and ``trace_id`` values in logs or trace
+  systems, not metrics label sets.
+- Use ``DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT`` when application log records
+  inside the view should include the same request correlation metadata.
+- Return only opaque IDs from ``DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC``.
+
 Database Indexes
 ----------------
 

docs/quickstart.rst

@@ -119,6 +119,57 @@ Or accept an upstream request ID:
    DRF_API_LOGGER_ENABLE_TRACING = True
    DRF_API_LOGGER_TRACING_ID_HEADER_NAME = "X-Request-ID"
 
+Request Correlation Without New DB Columns
+------------------------------------------
+
+Use correlation when the application needs request IDs, W3C ``traceparent``
+trace IDs, route metadata, or metrics labels without adding fields to the
+``drf_api_logs`` table.
+
+.. code-block:: python
+
+   DRF_API_LOGGER_SIGNAL = True
+   DRF_API_LOGGER_ENABLE_CORRELATION = True
+   DRF_API_LOGGER_CORRELATION_REQUEST_ID_HEADERS = ["X-Request-ID", "X-Correlation-ID"]
+   DRF_API_LOGGER_CORRELATION_TRACE_ID_HEADERS = ["traceparent", "X-Trace-ID"]
+   DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT = True
+
+Correlation data is exposed on the request object, in
+``drf_api_logger.logging_context.get_correlation_context()``, and in signal
+payload keys named ``correlation`` and ``low_cardinality``. It is not written to
+``APILogsModel`` and does not require migrations, model fields, admin columns,
+or database indexes.
+
+.. code-block:: python
+
+   from drf_api_logger import API_LOGGER_SIGNAL
+
+   def send_api_metrics(**kwargs):
+       labels = kwargs.get("low_cardinality", {})
+       metrics.count(
+           "drf_api_logger.request",
+           tags={
+               "route": labels.get("route"),
+               "status_class": labels.get("status_class"),
+           },
+       )
+
+   API_LOGGER_SIGNAL.listen += send_api_metrics
+
+For extra context, return only opaque IDs from an allowlisted callback:
+
+.. code-block:: python
+
+   DRF_API_LOGGER_CORRELATION_CONTEXT_FUNC = "myapp.logging.api_logger_context"
+
+   def api_logger_context(request):
+       return {
+           "actor_id": getattr(request.user, "pk", None),
+           "tenant_id": getattr(request, "tenant_id", None),
+           "api_consumer_id": getattr(request, "api_consumer_id", None),
+           "client_id": getattr(request, "client_id", None),
+       }
+
 Retention and Pruning
 ---------------------
 

docs/tutorials.rst

@@ -134,6 +134,41 @@ Log the same ID in application code:
        tracing_id = getattr(request, "tracing_id", "missing")
        logger.info("example_view started", extra={"tracing_id": tracing_id})
 
+Correlate Logs Without Persisting IDs
+-------------------------------------
+
+Use request correlation when application logs, gateway IDs, and metrics need to
+line up, but the log table should keep its existing schema.
+
+.. code-block:: python
+
+   DRF_API_LOGGER_SIGNAL = True
+   DRF_API_LOGGER_ENABLE_CORRELATION = True
+   DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT = True
+   DRF_API_LOGGER_CORRELATION_REQUEST_ID_HEADERS = ["X-Request-ID", "X-Correlation-ID"]
+   DRF_API_LOGGER_CORRELATION_TRACE_ID_HEADERS = ["traceparent", "X-Trace-ID"]
+
+.. code-block:: python
+
+   from drf_api_logger import API_LOGGER_SIGNAL
+
+   def send_metrics(**kwargs):
+       labels = kwargs.get("low_cardinality", {})
+       metrics.count(
+           "drf_api_logger.request",
+           tags={
+               "route": labels.get("route"),
+               "status_class": labels.get("status_class"),
+           },
+       )
+
+   API_LOGGER_SIGNAL.listen += send_metrics
+
+Correlation adds ``correlation`` and ``low_cardinality`` to signal payloads and
+sets request attributes such as ``request.api_logger_request_id``. It does not
+add model fields, migrations, admin columns, database indexes, or extra queued
+database payload fields.
+
 Community Snippets
 ------------------