Fix #59 (comment) aside from extra technical work

Author: emmuhamm

docs/explanation.md

@@ -128,5 +128,6 @@ For a hands on explanation of these, please read the [how-to guide on how to use
 
 ## Further reading
 
-- For how this routing model is implemented under the hood, see the [developer explanation](https://dune-daq.github.io/daqpytools/explanation).
-- For how to configure ERS and advanced routing in practice, see [Configuring ERS](./how-to/configure-ers.md) and [Routing messages to specific handlers](./how-to/route-messages.md).
+- For how this routing model is implemented under the hood, see the [developer explanation](https://dune-daq.github.io/daqpytools/dev).
+  - The architecture reference includes diagrams which explain exactly how the routing works. See [here](https://dune-daq.github.io/daqpytools/dev/reference/architecture/).
+- For how to configure ERS and advanced routing in practice, see [Configuring ERS](./how-to/configure-ers.md) and [Routing messages to specific handlers](./how-to/route-messages.md).

docs_dev/explanation.md

@@ -133,15 +133,18 @@ Defined in `routing.py`, strategies answer: "What `HandlerType` values are allow
 **`AllowedHandlersStrategy`** is the abstract base. Implementations:
 
 1. **`DefaultAllowedHandlerStrategy`**:
+   
    - Uses `record.handlers` if present (explicit routing metadata)
    - Falls back to `fallback_handlers` set if `record.handlers` is absent or None
 
 2. **`ERSAllowedHandlersStrategy`**:
+   
    - Reads `record.ers_handlers` dict and `record.levelno` (Python log level)
    - Maps the level to an ERS severity variable using `level_to_ers_var`
    - Returns the handler set for that severity
 
 3. **`StreamAwareAllowedHandlersStrategy`**:
+   
    - Looks at `record.stream`
    - If `stream == StreamType.ERS`, uses `ERSAllowedHandlersStrategy`
    - Otherwise uses `DefaultAllowedHandlerStrategy`
@@ -196,6 +199,12 @@ If routing isn't what you expect, debug:
 1. Does the record have explicit `extra["handlers"]`?
 2. If not, what's the fallback set?
 
+
+For a more concrete logical flow of the entire thing, with examples, please see the [architecture](./reference/architecture.md).
+
+### How HandleIDFillters, the routing strategies, and the fallback handlers work together
+
+
 ### Handler and Filter Registries
 
 The registries live in `handlers.py` and `filters.py`:

docs_dev/reference/architecture.md

@@ -42,13 +42,15 @@ When you call `log.info("something")`, here's the actual flow:
 1. **Python's logging creates a `LogRecord`** with your message, severity, and any `extra` metadata
 
 2. **Logger-level filters run first** (e.g., `ThrottleFilter`):
+   
    - If any filter returns `False`, the record stops here
    - It never reaches handlers
    - This is where global concerns like throttling happen
 
 3. **Record is offered to each attached handler**
 
 4. **Each handler's `HandleIDFilter` decides** whether to emit:
+   
    - The filter calls the routing strategy to resolve `allowed_handlers`:
      - If `extra["handlers"]` is present, use it
      - Otherwise use the fallback set
@@ -57,8 +59,87 @@ When you call `log.info("something")`, here's the actual flow:
    - Non-empty = emit; empty = drop the record
 
 5. **Format and emit** (if the record passed the filter):
+   
    - The handler formats it and emits (to file, stdout, kafka, etc.)
 
 This two-stage filtering is key: logger-level filters decide "should ANY handler see this?" while handler-level filters decide "should THIS handler see this?"
 
 ![Filtering](../img/Filter_activity.png)
+
+
+### Practical example
+
+Here the practical logic flow is explained more with demonstrations. 
+
+Lets say we have a logger that contains both a Rich and File handler, and we explicitly define what the fallback handlers is. 
+
+```python
+from daqpytools.logging import add_handler, HandlerType, get_daq_logger
+log = get_daq_logger(
+    "myapp",
+    log_level="INFO",
+    rich_handler=False,      # deliberately don't add handlers, we'll do it manually
+)
+
+add_handler(log,
+   HandlerType.Rich, 
+   use_parent_handlers=True
+   fallback_handler={HandlerType.Rich} # default behaviour
+   )
+add_handler(
+    log, 
+    HandlerType.File, 
+    use_parent_handlers=True,
+    fallback_handler={HandlerType.File} # default behaviour
+)
+
+```
+
+And now we investigate what happens if we send a log message with `log.info("msg", extra={"handlers": HandlerType.Rich})`. Here, the log message will be sent to both the Rich and File handlers for processing. In each handler, it will check to see what the allowed handlers are (lets call it `handlers_to_check_against`). In this case, since we set it explicitly when sending the mesage, `handlers_to_check_against = HandlerType.Rich`. 
+
+For the Rich handler, it checks if the Rich handler is in the `handlers_to_check_against` list. In this case, it is, so it will transmit.
+
+For the File handler, it will check if the File handler is in the `handlers_to_check_against` list. In this case, the file handler is _not_ in the list so it will not transmit the record.
+
+
+Using the same setup, lets investigate what happens if we send a log a log message with `log.info("msg")`. In this case, the handlers keyword is not explicitly populated.
+
+In this example, the record is sent independently to the two handlers. Since the handlers keyword is not set, the logic that populates `handlers_to_check_against` will check the `fallback_handlers` that are attached to each handler.
+
+In the rich handler, we have initialised it with a fallback_handler of `HandlerType.Rich`, so `handlers_to_check_against = fallback_handler = HandlerType.Rich`. The rich handler will then check if the rich handler is in the `handlers_to_check_against`, which it is. So it will transmit the record.
+
+In the file handler, we have initialised it with a fallback_handler of `HandlerType.File`, so `handlers_to_check_against = fallback_handler = HandlerType.File`. The file handler will then check if the file handler is in the `handlers_to_check_against`, which it is. So it will transmit the record.
+
+This sounds a bit circular, but the reasoning for this should become clear in the case where we set different fallback handlers. Consider the following setup, where the only difference is that we set the the file handler's fallback_handler to `HandlerType.Unknown`:
+
+
+```python
+from daqpytools.logging import add_handler, HandlerType, get_daq_logger
+log = get_daq_logger(
+    "myapp",
+    log_level="INFO",
+    rich_handler=False,      # deliberately don't add handlers, we'll do it manually
+)
+
+add_handler(log,
+   HandlerType.Rich, 
+   use_parent_handlers=True
+   fallback_handler={HandlerType.Rich} # default behaviour
+   )
+add_handler(
+    log, 
+    HandlerType.File, 
+    use_parent_handlers=True,
+    fallback_handler={HandlerType.Unknown} # Suppressing behaviour!
+)
+```
+
+Transmitting `log.info("msg", extra={"handlers": HandlerType.Rich})` will yield identical results to before. However, transmitting `log.info("msg")` yields different results.
+
+As before, the log record is _independently_ fed towards the two handlers. For the rich handler, the flow is identical and so it will transmit.
+
+In the case of the file handler, we have initialised it with a fallback_handler of `HandlerType.Unknown`, so `handlers_to_check_against = fallback_handler = HandlerType.Unknown`. The file handler will then check if the file handler is in the `handlers_to_check_against = HandlerType.Unknown`, which it is _not_ as `HandlerType.File` is not in `HandlerType.Unknown`. So it will **suppress** the record. 
+
+This has a very useful suppression effect, allowing the handlers to remain dormant by default. The only way to transmit with the file handler now is only if it was explicitly called for, by `log.info("msg", extra={"handlers": HandlerType.File})`
+
+

docs_dev/reference/patterns.md

@@ -47,10 +47,3 @@ log.info("skip me", extra={"skip_logging": True})
 ```
 
 ---
-
-## Next steps
-
-- Look at existing handlers in `handlers.py` for patterns
-- Look at `ThrottleFilter` for a complex filter example
-- Check test files in `tests/logging/` for usage examples
-- Add your handler/filter, submit a PR!

mkdocs.yml

@@ -22,6 +22,7 @@ theme:
 
 
 markdown_extensions:
+  - sane_lists
   - admonition
   - pymdownx.highlight:
       anchor_linenums: true