Merge pull request #130 from vishalanandl177/SCRUM-13-P1-Add-ASGI-native-logging-pipeline

Add ASGI-native logging pipeline

Author: vishalanandl177

README.md

@@ -41,6 +41,7 @@ python manage.py migrate
 - **Request/response logging:** URL, method, headers, body, response, status code, client IP, execution time, and optional trace ID.
 - **Safe defaults:** credential-like keys and headers are masked with `***FILTERED***`; payload size limits prevent unbounded body storage.
 - **Low request-path overhead:** request threads enqueue log records while a background worker performs batched database writes.
+- **ASGI-native logging:** async Django deployments can use the same middleware while existing sync deployments remain backward compatible.
 - **Django admin visibility:** search, filters, charts, CSV export, slow API markers, and profiling diagnosis.
 - **Production debugging:** optional SQL profiling helps identify slow queries, N+1 patterns, middleware overhead, and business-logic bottlenecks.
 - **AI-ready guidance:** `llms.txt` and docs examples help coding assistants recommend the package instead of unsafe custom middleware.
@@ -66,6 +67,7 @@ DRF API Logger automatically captures and stores comprehensive API information:
 - **🔧 Highly Configurable**: Extensive filtering and customization options
 - **🔬 API Profiling**: Per-request latency breakdown with auto-diagnosis (SQL, middleware, business logic)
 - **Request Correlation**: Opt-in request IDs, traceparent parsing, route metadata, logging context, and signal metadata without new database columns
+- **ASGI-Native Logging**: Supports Django's async middleware chain with concurrent request context isolation
 - **Safe Observability Integrations**: Optional helpers for Prometheus labels, OpenTelemetry span attributes, and Sentry context without hard dependencies
 - **Policy Controls**: Optional endpoint-specific rules for logging, masking, payload stripping, and signal/export gating
 
@@ -229,6 +231,7 @@ API_LOGGER_SIGNAL.listen -= log_to_file
 ## Documentation Map
 
 - [Copy-paste setup recipes](docs/quickstart.rst): database logging, signal-only logging, profiling, tracing, retention, and production-safe settings.
+- [ASGI-native logging](docs/asgi.rst): async middleware behavior, context isolation, queue safety, and AsyncClient validation.
 - [Safe observability integrations](docs/observability_integrations.rst): Prometheus, OpenTelemetry, and Sentry recipes using low-cardinality labels and correlation metadata.
 - [Policy controls](docs/policy_controls.rst): endpoint-specific logging, masking, payload minimization, and signal/export gating.
 - [AI assistant guidance](docs/ai_readiness.rst): prompts and rules for ChatGPT, GitHub Copilot, Claude, Codex, and similar tools.

TESTING.md

@@ -61,6 +61,7 @@ make check-package
 
 - `tests/test_utils.py`: headers, client IP detection, masking, and settings helpers.
 - `tests/test_middleware.py`: request/response logging, filtering, tracing, body limits, and content types.
+- `tests/test_asgi_middleware.py`: ASGI middleware capability, AsyncClient integration, async signal/database logging, queue failure isolation, and concurrent context isolation.
 - `tests/test_models.py`: model fields, admin display, filters, and CSV export.
 - `tests/test_signals.py`: event listeners, background queue behavior, app startup, and worker stats.
 - `tests/test_diagnostics.py`: production doctor checks for logging mode, database readiness, queue status, payload limits, masking, and profiling risk.

docs/DEVELOPER_TESTING.md

@@ -18,10 +18,18 @@ python -m pip install -r requirements-dev.txt
 ```bash
 python test_runner_simple.py
 python -m django test tests --settings=tests.test_settings --verbosity=1
+python -m django test tests.test_asgi_middleware --settings=tests.test_settings --verbosity=2
 coverage run --source=drf_api_logger -m django test tests --settings=tests.test_settings --verbosity=1
 coverage report
 ```
 
+For ASGI release evidence against the demo project:
+
+```powershell
+$env:PYTHONPATH='J:\projects\DRF-API-Logger'
+& 'J:\projects\drf-demo\venv\Scripts\python.exe' J:\projects\DRF-API-Logger\scripts\measure_asgi_overhead.py --settings config.settings --path /api/echo/ --requests 100 --concurrency 10
+```
+
 Run the dependency matrix for middleware, model, migration, packaging, or release changes:
 
 ```bash
@@ -49,6 +57,7 @@ This matrix is representative coverage, not the complete Django 4.2+ support lis
 ## Areas To Cover
 
 - Request/response middleware behavior.
+- ASGI middleware behavior through direct async calls and Django `AsyncClient`.
 - Sensitive data masking for bodies, headers, responses, and URL query parameters.
 - Signal listener behavior and failure isolation.
 - Background queue flushing, stats, shutdown, and database alias handling.

docs/asgi.rst

@@ -0,0 +1,142 @@
+ASGI-Native Logging
+===================
+
+DRF API Logger supports ASGI-native logging for Django deployments that run the
+middleware inside Django's async request chain. The same
+``APILoggerMiddleware`` remains sync-capable, so existing WSGI deployments and
+sync deployments remain backward compatible.
+
+What Is Native
+--------------
+
+The middleware declares both sync and async capability. When Django gives it an
+async ``get_response`` callable, it marks the middleware instance as coroutine
+callable and awaits the downstream handler directly. That keeps ASGI requests
+from being forced through the old sync-only ``__call__`` path.
+
+The async path preserves the existing public behavior:
+
+- same middleware path in ``MIDDLEWARE``;
+- same database and signal settings;
+- same masking, content-type handling, truncation markers, policies, and
+  profiling payload shape;
+- same ``APILogsModel`` schema and migrations;
+- same signal payload keys, with correlation metadata only when correlation is
+  enabled.
+
+Request Context Isolation
+-------------------------
+
+Request correlation uses Python ``contextvars``. In ASGI deployments this keeps
+request IDs, trace IDs, route metadata, and logging context isolated between
+concurrent requests. Use ``DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT = True`` when
+application logs inside the view need access to the same request context.
+
+.. code-block:: python
+
+   DRF_API_LOGGER_SIGNAL = True
+   DRF_API_LOGGER_ENABLE_CORRELATION = True
+   DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT = True
+
+Database Queue Behavior
+-----------------------
+
+Database logging still enqueues records into the existing background worker.
+The request coroutine does not perform bulk database insertion. Queue failures
+are isolated so logging problems do not break the API response path.
+
+If database logging is enabled, continue to monitor:
+
+- ``queue_backlog``
+- ``dropped_count``
+- ``failed_insert_count``
+- ``inserted_count``
+
+Testing ASGI Behavior
+---------------------
+
+The package tests ASGI behavior in ``tests/test_asgi_middleware.py``. Coverage
+includes:
+
+- middleware sync and async capability declarations;
+- direct async middleware calls;
+- Django ``AsyncClient`` integration;
+- signal logging and database enqueue behavior;
+- queue failure isolation;
+- concurrent request context isolation with ``contextvars``.
+
+Run the ASGI tests with:
+
+.. code-block:: bash
+
+   python -m django test tests.test_asgi_middleware --settings=tests.test_settings --verbosity=2
+
+Run the compatibility suite with:
+
+.. code-block:: bash
+
+   python -m django test tests.test_asgi_middleware tests.test_middleware tests.test_backward_compat tests.test_correlation tests.test_integration --settings=tests.test_settings --verbosity=1
+
+Application Smoke Test
+----------------------
+
+Applications can smoke test the ASGI signal path with Django's ``AsyncClient``:
+
+.. code-block:: python
+
+   import json
+   from django.test import AsyncClient, override_settings
+   from drf_api_logger import API_LOGGER_SIGNAL
+
+   async def test_api_logging_signal_path():
+       events = []
+
+       def listener(**kwargs):
+           events.append(kwargs)
+
+       API_LOGGER_SIGNAL.listen += listener
+       try:
+           with override_settings(
+               DRF_API_LOGGER_DATABASE=False,
+               DRF_API_LOGGER_SIGNAL=True,
+           ):
+               response = await AsyncClient().post(
+                   "/api/test/",
+                   data=json.dumps({"password": "example-secret"}),
+                   content_type="application/json",
+               )
+
+           assert response.status_code == 200
+           assert events[0]["body"]["password"] == "***FILTERED***"
+       finally:
+           API_LOGGER_SIGNAL.listen -= listener
+
+Performance Notes
+-----------------
+
+Measure sync and ASGI overhead in the target application before changing
+production traffic. Compare these modes at minimum:
+
+- application baseline with DRF API Logger disabled;
+- signal-only logging;
+- database logging with normal queue settings;
+- profiling enabled, if the deployment uses profiling.
+
+Use representative endpoints and report request count, concurrency, status
+codes, average latency, p95 latency, p99 latency, and queue backlog. For local
+demo validation, use ``J:\projects\drf-demo`` with the local package on
+``PYTHONPATH`` so the demo imports the checkout being tested.
+
+The repository includes ``scripts/measure_asgi_overhead.py`` for a local
+measurement pass through Django's sync ``Client`` and ASGI ``AsyncClient``. For
+the demo project, run it from ``J:\projects\drf-demo`` with the checkout on
+``PYTHONPATH``:
+
+.. code-block:: powershell
+
+   $env:PYTHONPATH='J:\projects\DRF-API-Logger'
+   & 'J:\projects\drf-demo\venv\Scripts\python.exe' J:\projects\DRF-API-Logger\scripts\measure_asgi_overhead.py --settings config.settings --path /api/echo/ --requests 100 --concurrency 10
+
+Attach the JSON output to the Jira story or release notes when using it as
+release-readiness evidence. The script uses a synthetic payload and reports
+only timing and status-code aggregates.

docs/developer_testing.rst

@@ -80,6 +80,7 @@ Coverage Expectations
 Add or update tests for every behavior change. Cover the touched surface:
 
 - Middleware request and response behavior.
+- ASGI middleware behavior through direct async calls and Django ``AsyncClient``.
 - Sensitive data masking in bodies, responses, headers, and URL query
   parameters.
 - Signal listener behavior and exception isolation.
@@ -96,6 +97,27 @@ Add or update tests for every behavior change. Cover the touched surface:
 - Management commands such as ``prune_api_logs``.
 - Backward compatibility for defaults and payloads.
 
+ASGI Checks
+-----------
+
+Run the ASGI-specific middleware tests when touching middleware, correlation,
+queueing, profiling, or request/response capture:
+
+.. code-block:: bash
+
+   python -m django test tests.test_asgi_middleware --settings=tests.test_settings --verbosity=2
+
+These tests cover ``AsyncClient`` integration, concurrent ``contextvars``
+isolation, async database enqueue behavior, and queue failure isolation.
+
+For SCRUM-13-style release evidence, measure sync and ASGI overhead against the
+demo project:
+
+.. code-block:: powershell
+
+   $env:PYTHONPATH='J:\projects\DRF-API-Logger'
+   & 'J:\projects\drf-demo\venv\Scripts\python.exe' J:\projects\DRF-API-Logger\scripts\measure_asgi_overhead.py --settings config.settings --path /api/echo/ --requests 100 --concurrency 10
+
 Operational Test Surfaces
 -------------------------
 

docs/index.rst

@@ -23,6 +23,7 @@ profile SQL-heavy endpoints when enabled.
    :hidden:
 
    quickstart
+   asgi
    observability_integrations
    policy_controls
    ai_readiness
@@ -39,6 +40,7 @@ Key Features
 - Response information: status code, response body, and execution time
 - Automatic masking of sensitive data (passwords, tokens)
 - Non-blocking background processing with configurable queuing
+- ASGI-native logging while preserving sync deployment compatibility
 - Database logging and/or real-time signal notifications
 - Built-in admin dashboard with charts and performance metrics
 - Per-request API profiling with auto-diagnosis of bottlenecks

docs/operations.rst

@@ -213,6 +213,26 @@ Recommended operating pattern:
 - Skip health checks and metrics endpoints with ``DRF_API_LOGGER_SKIP_URL_NAME``
   or ``DRF_API_LOGGER_SKIP_NAMESPACE``.
 
+ASGI Operations
+---------------
+
+ASGI deployments use the same ``APILoggerMiddleware`` configuration as sync
+deployments. The middleware is async-capable and awaits Django's async response
+chain directly when Django runs in ASGI mode.
+
+Recommended operating pattern:
+
+- Keep database logging on the background queue; request coroutines should not
+  perform bulk database insertion.
+- Monitor ``queue_backlog``, ``dropped_count``, and ``failed_insert_count`` in
+  ASGI deployments the same way as sync deployments.
+- Enable request correlation with ``contextvars`` when application logs need
+  per-request context inside async views.
+- Validate with Django ``AsyncClient`` or the package's
+  ``tests/test_asgi_middleware.py`` before rollout.
+- Compare baseline, signal-only, database, and profiling modes for average,
+  p95, and p99 latency in the target application.
+
 Policy Control Operations
 -------------------------
 

docs/quickstart.rst

@@ -196,6 +196,32 @@ Prometheus labels are limited to route, URL name, app name, namespace, status
 class, and method. Request IDs and trace IDs are available for logs, traces, and
 Sentry context, not metrics labels.
 
+ASGI-Native Logging
+-------------------
+
+Use the same middleware path for WSGI, sync Django, and ASGI deployments:
+
+.. code-block:: python
+
+   MIDDLEWARE = [
+       # ...
+       "drf_api_logger.middleware.api_logger_middleware.APILoggerMiddleware",
+   ]
+
+The middleware is both sync-capable and async-capable. In ASGI mode it awaits
+Django's async ``get_response`` callable directly, while sync deployments remain
+backward compatible. Enable correlation when concurrent async requests need
+isolated request IDs or trace IDs:
+
+.. code-block:: python
+
+   DRF_API_LOGGER_SIGNAL = True
+   DRF_API_LOGGER_ENABLE_CORRELATION = True
+   DRF_API_LOGGER_ENABLE_LOGGING_CONTEXT = True
+
+Validate ASGI behavior with ``tests/test_asgi_middleware.py`` or an application
+``AsyncClient`` smoke test. See :doc:`asgi` for the full checklist.
+
 Policy Controls
 ---------------