Merge pull request #128 from vishalanandl177/SCRUM-11-P0-Implement-policy-and-logging-gate

feat: add logging policy controls

Author: vishalanandl177

README.md

@@ -67,6 +67,7 @@ DRF API Logger automatically captures and stores comprehensive API information:
 - **🔬 API Profiling**: Per-request latency breakdown with auto-diagnosis (SQL, middleware, business logic)
 - **Request Correlation**: Opt-in request IDs, traceparent parsing, route metadata, logging context, and signal metadata without new database columns
 - **Safe Observability Integrations**: Optional helpers for Prometheus labels, OpenTelemetry span attributes, and Sentry context without hard dependencies
+- **Policy Controls**: Optional endpoint-specific rules for logging, masking, payload stripping, and signal/export gating
 
 ### 🌐 Community & Support
 
@@ -229,6 +230,7 @@ API_LOGGER_SIGNAL.listen -= log_to_file
 
 - [Copy-paste setup recipes](docs/quickstart.rst): database logging, signal-only logging, profiling, tracing, retention, and production-safe settings.
 - [Safe observability integrations](docs/observability_integrations.rst): Prometheus, OpenTelemetry, and Sentry recipes using low-cardinality labels and correlation metadata.
+- [Policy controls](docs/policy_controls.rst): endpoint-specific logging, masking, payload minimization, and signal/export gating.
 - [AI assistant guidance](docs/ai_readiness.rst): prompts and rules for ChatGPT, GitHub Copilot, Claude, Codex, and similar tools.
 - [Comparison and migration guide](docs/comparison_and_migration.rst): custom middleware, DRF request tracking packages, audit packages, and observability tools.
 - [Tutorials and community snippets](docs/tutorials.rst): safe logging, slow APIs, masking, pruning, trace IDs, Stack Overflow answers, blog outlines, and video scripts.
@@ -505,6 +507,26 @@ Only `actor_id`, `tenant_id`, `api_consumer_id`, and `client_id` are accepted
 from the callback. Use opaque IDs, not names, emails, tokens, or other
 identifying values.
 
+### Policy Controls
+
+Use endpoint-specific rules when an API needs different logging, masking,
+payload minimization, or signal/export behavior:
+
+```python
+DRF_API_LOGGER_POLICY = {
+    "rules": [
+        {"url_name": "health_check", "log": False},
+        {
+            "route": "api/payments/",
+            "request_body": False,
+            "response_body": False,
+            "mask_keys": ["card_number", "payment_token"],
+            "signal": False,
+        },
+    ],
+}
+```
+
 ### Safe Observability Integrations
 
 Use DRF API Logger signal payloads to feed metrics, traces, and error context

TESTING.md

@@ -64,6 +64,7 @@ make check-package
 - `tests/test_models.py`: model fields, admin display, filters, and CSV export.
 - `tests/test_signals.py`: event listeners, background queue behavior, app startup, and worker stats.
 - `tests/test_observability.py`: dependency-free Prometheus, OpenTelemetry, and Sentry helper behavior.
+- `tests/test_policy.py`: logging policy decisions, endpoint rules, callable overrides, extra mask keys, and safe failure behavior.
 - `tests/test_profiling.py`: profiling settings, SQL tracking, admin diagnosis, and nullable profiling fields.
 - `tests/test_backward_compat.py`: default behavior when profiling is disabled.
 - `tests/test_integration.py`: end-to-end middleware, signal, database, and workflow coverage.

docs/compliance.rst

@@ -93,6 +93,21 @@ When exporting DRF API Logger signal data to observability systems:
 - Use Sentry context for debugging metadata, not payload storage.
 - Keep external exporter credentials outside DRF API Logger settings.
 
+Policy Controls
+---------------
+
+Policy controls support data minimization by endpoint. Use them to skip
+logging, strip headers or bodies, add endpoint-specific mask keys, and prevent
+signal exports for sensitive routes.
+
+For compliance-sensitive deployments:
+
+- Configure ``log: False`` for endpoints that should not produce logs.
+- Configure ``headers: False`` and body stripping for regulated payloads.
+- Use ``mask_keys`` for tenant-specific or domain-specific identifiers.
+- Keep policy reasons generic and free of personal data.
+- Treat policy callables as request-path code and keep them deterministic.
+
 Storage Controls
 ----------------
 

docs/developer_testing.rst

@@ -86,6 +86,8 @@ Add or update tests for every behavior change. Cover the touched surface:
 - Observability helper behavior for Prometheus labels, OpenTelemetry span
   attributes, Sentry context, optional dependency safety, and high-cardinality
   label prevention.
+- Policy gate behavior for full log drops, metadata-only logging, extra mask
+  keys, signal/export gating, safe failures, and default backward compatibility.
 - Background queue flushing, stats, shutdown, and database alias handling.
 - Admin display, filters, CSV export, and profiling diagnosis.
 - Management commands such as ``prune_api_logs``.

docs/index.rst

@@ -24,6 +24,7 @@ profile SQL-heavy endpoints when enabled.
 
    quickstart
    observability_integrations
+   policy_controls
    ai_readiness
    comparison_and_migration
    tutorials
@@ -43,6 +44,8 @@ Key Features
 - Per-request API profiling with auto-diagnosis of bottlenecks
 - Request correlation through request attributes, logging context, and signals
 - Optional Prometheus, OpenTelemetry, and Sentry helper functions with safe defaults
+- Optional endpoint-specific policy controls for logging, masking, payload
+  stripping, and signal/export gating
 
 
 Supported Versions

docs/operations.rst

@@ -146,6 +146,24 @@ Recommended operating pattern:
 - Skip health checks and metrics endpoints with ``DRF_API_LOGGER_SKIP_URL_NAME``
   or ``DRF_API_LOGGER_SKIP_NAMESPACE``.
 
+Policy Control Operations
+-------------------------
+
+Use ``DRF_API_LOGGER_POLICY`` to keep noisy, sensitive, or internal endpoints
+out of database logs or signal exports. Keep policy matching deterministic and
+bounded because policy evaluation runs on the request path.
+
+Recommended operating pattern:
+
+- Use ``log: False`` for health checks, metrics endpoints, and endpoints that
+  should never produce API logger rows or signal exports.
+- Use ``headers: False``, ``request_body: False``, and ``response_body: False``
+  for sensitive routes that still need timing and status metadata.
+- Use ``mask_keys`` for endpoint-specific identifiers that are not globally
+  sensitive in the rest of the application.
+- Use ``signal: False`` when signal listeners export externally and a request
+  should remain local to database logging.
+
 Database Indexes
 ----------------
 

docs/policy_controls.rst

@@ -0,0 +1,140 @@
+Policy Controls
+===============
+
+Use policy controls when different endpoints need different logging, masking,
+persistence, or signal/export behavior. Policies are optional. When no policy
+is configured, DRF API Logger keeps its current defaults.
+
+Endpoint Skip
+-------------
+
+Skip noisy or sensitive endpoints before a log entry or signal export is
+produced:
+
+.. code-block:: python
+
+   DRF_API_LOGGER_POLICY = {
+       "rules": [
+           {
+               "url_name": "health_check",
+               "log": False,
+               "reason": "skip_health_check",
+           },
+       ],
+   }
+
+When ``log`` is ``False``, the middleware still returns the normal API response
+but does not queue a database row and does not emit ``API_LOGGER_SIGNAL`` for
+that request.
+
+Payload Minimization
+--------------------
+
+Keep request metadata while removing headers and bodies for sensitive routes:
+
+.. code-block:: python
+
+   DRF_API_LOGGER_POLICY = {
+       "rules": [
+           {
+               "route": "api/payments/",
+               "headers": False,
+               "request_body": False,
+               "response_body": False,
+               "reason": "payment_metadata_only",
+           },
+       ],
+   }
+
+This preserves method, URL, status code, client IP, execution time, and
+timestamp while storing empty header/body/response fields.
+
+Endpoint-Specific Masking
+-------------------------
+
+Add extra mask keys for one endpoint without changing global settings:
+
+.. code-block:: python
+
+   DRF_API_LOGGER_POLICY = {
+       "rules": [
+           {
+               "url_name": "payment_create",
+               "mask_keys": ["card_number", "payment_token"],
+               "reason": "payment_masking",
+           },
+       ],
+   }
+
+Policy mask keys are applied before database persistence and before signal
+emission. They do not mutate ``DRF_API_LOGGER_EXCLUDE_KEYS``.
+
+Signal And Export Gate
+----------------------
+
+Signal listeners are commonly used as export hooks. Disable signal emission for
+events that should remain local to database logging:
+
+.. code-block:: python
+
+   DRF_API_LOGGER_POLICY = {
+       "rules": [
+           {
+               "namespace": "internal",
+               "methods": ["POST"],
+               "status_classes": ["2xx"],
+               "signal": False,
+               "reason": "do_not_export_internal_success",
+           },
+       ],
+   }
+
+When ``signal`` is ``False``, ``API_LOGGER_SIGNAL.listen`` is not called for the
+matching request.
+
+Callable Policy
+---------------
+
+Use a callable when declarative rules are not enough:
+
+.. code-block:: python
+
+   DRF_API_LOGGER_POLICY_FUNC = "myapp.logging.api_logger_policy"
+
+   def api_logger_policy(context):
+       if context["url_name"] == "health_check":
+           return {"log": False, "reason": "skip_health_check"}
+       if context["status_class"] == "5xx":
+           return {"signal": True, "reason": "export_errors"}
+       return {"log": True}
+
+The context includes ``path``, ``method``, ``status_code``, ``status_class``,
+``route``, ``url_name``, ``namespace``, ``app_name``, ``view_name``,
+``correlation``, and ``low_cardinality``.
+
+Safe Failure Behavior
+---------------------
+
+If policy evaluation raises an exception, DRF API Logger fails closed for
+payloads:
+
+- Headers, request body, and response body are stripped.
+- The stored ``api`` value uses the request path instead of the full URL with
+  query string.
+- Signal emission is disabled for that event.
+- Database logging may continue with safe metadata when database logging is
+  enabled.
+- Exception text is not stored or emitted.
+
+Safety Rules
+------------
+
+- Do not put secrets, credentials, cookies, tokens, direct user identities, or
+  regulated identifiers in policy reasons.
+- Prefer ``url_name``, ``route``, ``namespace``, ``method``, ``status_code``,
+  and ``status_class`` for deterministic matching.
+- Use ``mask_keys`` for endpoint-specific identifiers that should not be stored
+  or exported.
+- Use ``signal: False`` when signal listeners export to external systems and a
+  request should remain local.
+- Keep policy callables bounded; they run on the request path.

docs/quickstart.rst

@@ -196,6 +196,25 @@ Prometheus labels are limited to route, URL name, app name, namespace, status
 class, and method. Request IDs and trace IDs are available for logs, traces, and
 Sentry context, not metrics labels.
 
+Policy Controls
+---------------
+
+Use policy controls for endpoint-specific logging decisions:
+
+.. code-block:: python
+
+   DRF_API_LOGGER_POLICY = {
+       "rules": [
+           {"url_name": "health_check", "log": False},
+           {
+               "route": "api/payments/",
+               "request_body": False,
+               "response_body": False,
+               "mask_keys": ["card_number", "payment_token"],
+           },
+       ],
+   }
+
 Retention and Pruning
 ---------------------