Fix #59 (comment) aside from extra technical work

Author: emmuhamm

docs/explanation.md

@@ -32,7 +32,7 @@ Every record has an attached severity level, which can be used to flag how impor
 
 ![log_level_overview](img/loglevels.png)
 
-More levels can be defined as required, see Python's logging manual.
+It's worth noting that the severity levels are just enums in Python, with `CRITICAL` being 50, with increments of 10 down to 0 for `NOTSET`. More levels can be defined as required, see Python's [logging manual](https://docs.python.org/3/library/logging.html#logging-levels).
 
 Each logging instance can have an attached severity level. If it has one, then only records that have the same severity level or higher will be transmitted.
 
@@ -72,11 +72,24 @@ By default, loggers inherit certain properties from the parent:
 - severity level of the logger 
 - handlers (and all attached properties, including severity level and filters on handlers)
 
+
+A good model to think about while developing is shown below, where its thought of that the loggers 'get' the same handlers as their ancestors. Note one exception: they _do not_ inherit filters attached directly to the parent logger itself.
 ![inheritance](img/inheritance.png)
 
-Note one exception: they _do not_ inherit filters attached directly to the parent logger itself.
 
-A useful diagram is the [logging flow in the official Python 3 docs](https://docs.python.org/3/howto/logging.html#logging-flow).
+Behind the scenes, what actually happens is that the log record gets passed to the ancestor loggers; this is beneficial as it doesn't duplicate the two handles and instead passes the record around. See the [logging flow in the official Python3 docs](https://docs.python.org/3/howto/logging.html#logging-flow).
+
+
+
+#### The root logger
+
+In the native Python logging framework the highest possible logger is the (usually unnamed) root logger. For example, calling `logging.getLogger("top")` will usually yield you a logger called `{root}."top"`. Calling `logging.getLogger()` gets you the `{root}` logger.
+
+As the root logger is the highest logger which every logger inherits from, modifying this logger will have a _global_ effect on all your loggers, which is almost always undesirable.
+
+**Importantly, changing the root logger will affect the logging instances in other repositories too! This can lead to some undesirable behaviours, such as [this](https://github.com/DUNE-DAQ/drunc/blob/df51ce36cffe08efab6bd2a7a47554554deed22b/src/drunc/utils/utils.py#L64-L71).
+
+This information is not actionable in and of itself, however this provides context on some of the [best practices](./how-to/best-practices.md) that have been laid out. 
 
 ---
 
@@ -109,6 +122,8 @@ You can think of streams as different "channels" where each has its own set of h
 
 This is why routing via `extra={"handlers": [...]}` matters — it tells the logger which stream/handlers to use for each record.
 
+For a hands on explanation of these, please read the [how-to guide on how to use handlers and filters](./how-to/use-handlers.md)
+
 ---
 
 ## Further reading

docs/how-to/add-handlers-at-runtime.md

@@ -1,9 +1,9 @@
 # How to add handlers at runtime
 
-You can configure handlers in two phases:
+Loggers and their respective handlers can be configured in two ways.
 
-1. Build a logger first with `get_daq_logger(...)`.
-2. Add more handlers/filters later, based on runtime context.
+1. Build a logger first with `get_daq_logger(...)`. This is covered in the [getting started tutorial](../tutorial.md)/.
+2. Add more handlers/filters later, based on runtime context. This is covered in this page.
 
 This is useful in long-running services where extra outputs (for example ERS Kafka) should only be attached after additional configuration becomes available.
 
@@ -32,6 +32,8 @@ log.info("Now routes to rich + stdout by default")
 
 ## Suppress by default with `fallback_handler={HandlerType.Unknown}`
 
+This feature takes heavy advantage of the `extra` feature of Python logging. Please read the documentation on how `extra` is used, [found here](https://dune-daq.github.io/daqpytools/dev/explanation/).
+
 You can make newly-added handlers opt-in only by setting fallback handlers to `HandlerType.Unknown`. This means records without explicit `extra["handlers"]` will not be emitted by those handlers.
 
 ```python
@@ -56,6 +58,8 @@ log.critical(
 )
 ```
 
+For a more in depth discussion on this feature, please see the development docs. 
+
 ---
 
 ## Passing arguments to handlers and filters via `**kwargs`

docs/how-to/best-practices.md

@@ -8,11 +8,9 @@ The docs so far give a nice overview of how the logging tools work, but now you
 
 ## Use of the root logger
 
-**Always** set up a named pseudo-root logger in your application as close to initialisation of your application as possible. Use the daqpytools implementation `setup_root_logger` to do so.
-
-For context, in the native Python logging framework the highest possible logger is the (usually unnamed) root logger. For example, calling `logging.getLogger("top")` will usually yield you a logger called `{root}."top"`. Calling `logging.getLogger()` gets you the `{root}` logger.
+For background information surrounding this tip, please see the explanation on the Python root logger found [here](../explanation.md).
 
-As the root logger is the highest logger which every logger inherits from, modifying this logger will have a _global_ effect on all your loggers, which is almost always undesirable.
+**Always** set up a named pseudo-root logger in your application as close to initialisation of your application as possible. Use the daqpytools implementation `setup_root_logger` to do so.
 
 To keep things safe and compartmentalisable, a pseudo-root logger should be defined very early on, and should contain no handlers. This has benefits of compartmentalising publishing, and making things clearer in the logs due to more traceable names.
 
@@ -36,11 +34,13 @@ As shown here, all loggers are initialised via `{pseudo_root_logger}.{parent}.{c
 
 _Ideally_, loggers should only be defined once. While they _are_ singleton objects and there are simple ways to call an already defined logger, preference should be made to use inheritance to call 'new' loggers to keep things traceable.
 
-A good place to define parent-level loggers with handlers (c.f `drunc.process_manager`) is the module's `__init__` file. Subsequent new loggers can be defined in the various files of that Python module. For example, in the `process_manager/utils.py`, a new logger called `drunc.process_manager.utils` can be defined and used for the duration of that file, where it automatically inherits the handlers defined from the parent-level logger.
+A good place to define parent-level loggers with handlers (c.f. `drunc.process_manager`) is the module's `__init__` file. Subsequent new loggers can be defined in the various files of that Python module. For example, in the `process_manager/utils.py`, a new logger called `drunc.process_manager.utils` can be defined and used for the duration of that file, where it automatically inherits the handlers defined from the parent-level logger.
 
 ## Calling and configuring loggers
 
-Use `get_daq_logger` to initialise it once.
+Once the pseudo-root logger is defined, you can use `get_daq_logger` to initialise it once. 
+
+A useful tip for package managers is to define a function that prepends a prefix to actually inherit from the pseudo-root logger to ensure that inheritance is followed. See [here](https://github.com/DUNE-DAQ/drunc/blob/df51ce36cffe08efab6bd2a7a47554554deed22b/src/drunc/utils/utils.py#L52-L61) for an example.
 
 Following the previous tip, if you feel the need to get an already-initialised logger with `get_daq_logger`, consider making a child.
 

docs/how-to/configure-ers.md

@@ -2,10 +2,25 @@
 
 This page covers how to attach and use ERS (error reporting system) handlers on a logger.
 
-For background on ERS streams and routing, see [Concepts](../explanation.md). For the `LogHandlerConf` routing API, see [Routing messages to specific handlers](./route-messages.md).
+For background on ERS streams and routing, see [Concepts](../explanation.md). For the `LogHandlerConf` routing API, see [Routing messages to specific handlers](./route-messages.md). For the definition of the handler itself as well as how it can be used, see [How to use handlers and filters](https://dune-daq.github.io/daqpytools/dev/explanation/).
 
 ---
 
+## Configuring ERS handlers onto a new logger (by construction)
+
+Use the `ers_kafka_session` variable to put in the relevant session name in `get_daq_logger`. There are several attributes that you can use to customise the ERS handler as well, such as changing the ERS application name as displayed on the ERS dashboards, exampled below. Please see the [API reference](https://dune-daq.github.io/daqpytools/APIref/handlers/protobufstream/) for full details on what can be passed in.  
+
+```python
+    from daqpytools.logging import get_daq_logger, 
+    main_logger: logging.Logger = get_daq_logger(
+        logger_name="logger_name",
+        log_level="INFO",
+        use_parent_handlers=True,
+        ers_kafka_session="session_name,
+        ers_app_name="Custom App Name", # Can be none!
+    )
+```
+
 ## Configuring ERS handlers on an existing logger
 
 Use `setup_daq_ers_logger` to attach ERS-derived handlers to an existing logger based on environment configuration.

docs/how-to/route-messages.md

@@ -6,7 +6,7 @@ For background on *why* routing works this way, see [Concepts](../explanation.md
 
 ---
 
-## Choosing handlers with HandlerTypes
+## Choosing handlers with HandlerTypes and the extra keyword
 
 You can route individual records to specific handlers by using `extra={"handlers": [...]}`:
 
@@ -26,7 +26,7 @@ Note: Asking for a handler type that isn't attached is a no-op. Using `HandlerTy
 
 ## Using LogHandlerConf for structured routing
 
-`LogHandlerConf` is a configuration dataclass that encapsulates the handler setup for different streams. It handles ERS environment variable parsing and creates routing metadata bundles that you attach to records via `extra`.
+`LogHandlerConf` is a configuration dataclass that encapsulates the handler setup for different streams. It can handle ERS environment variable parsing and creates routing metadata bundles that you attach to records via `extra`.
 
 ### Understanding LogHandlerConf
 

docs/how-to/use-handlers.md

@@ -28,8 +28,8 @@ As the name suggests, the file handler is used to transmit messages directly to
 
 Stream handlers are used to transmit messages directly to the terminal without any color formatting. This is of great use for the logs of the controllers in drunc, which has its own method of capturing logs via a capture of the terminal output and a pipe to the relevant log file.
 
-Note that stream handling consists of two handlers, one writing to `stdout` and one to `stderr`. The `stderr` stream emits only for records at `ERROR` or above.
-
+Note that stream handling consists of two handlers in daqpytools, one writing to `stdout` and one to `stderr`. The `stderr` stream is configured to emit only for records at `ERROR` or above.
+ 
 ![streams_demo](../img/demo_streams.png)
 
 ## ERS Kafka handler
@@ -100,17 +100,17 @@ main_logger: logging.Logger = get_daq_logger(
     throttle=True
 )
 
-emit_err = lambda i: main_logger.info(
+emit_log_record = lambda i: main_logger.info(
     f"Throttle test {i}",
     extra={"handlers": [HandlerType.Rich, HandlerType.Throttle]},
 )
 
 for i in range(50):
-    emit_err(i)
+    emit_log_record(i)
 main_logger.warning("Sleeping for 30 seconds")
 time.sleep(30)
 for i in range(1000):
-    emit_err(i)
+    emit_log_record(i)
 ```
 
 Which will behave as expected.

docs/tutorial.md

@@ -29,12 +29,14 @@ test_logger = get_daq_logger(
 )
 ```
 
-For now, **please see the docstring of `get_daq_logger` to see what stuff you can have and what to initialise with.**
+Please see the API reference of `get_daq_logger` [here](https://dune-daq.github.io/daqpytools/APIref/get_daq_logger/), or alternatively the code itself [here](https://github.com/DUNE-DAQ/daqpytools/blob/develop/src/daqpytools/logging/logger.py), to see what options exist in initialising `get_daq_logger`. 
 
 This gives you a named logger with a single Rich handler attached, emitting at `INFO` level and above. Loggers in daqpytools are singletons — calling `get_daq_logger` with the same name twice will return the same instance, so it's safe to call this once at module level and reuse it throughout your code.
 
 ## Step 2: Emit your first messages
 
+Emitting refers to the act of processing the log record and sending it out to its intended destination! You can emit your messages to the terminal, to a file, via a RESTapi, and so on. For this tutorial, we are simply emitting a message to the terminal. 
+
 ```python
 test_logger.info("Hello, world!")
 

docs_dev/explanation.md

@@ -85,16 +85,25 @@ This is extensible: you can add new `StreamType` values and teach the strategy d
 Defined in `specs.py`, there are two types:
 
 **`HandlerSpec`** describes how to build a handler:
+
 - `alias`: The `HandlerType` key
+
 - `handler_class`: The runtime handler class (used to detect existing instances)
+
 - `factory`: A callable that builds the handler from configuration
+
 - `fallback_types`: Which `HandlerType` values this handler represents for routing purposes
+
 - `target_stream`: Optional (for stream-specific handlers like stdout vs stderr)
 
 **`FilterSpec`** describes how to build a logger-level filter:
+
 - `alias`: The activation `HandlerType` token
+
 - `filter_class`: The runtime filter class
+
 - `factory`: A callable that builds the filter
+
 - `fallback_types`: Default handler types for the filter
 
 Specs are the "source of truth" for what a handler or filter is. When setup code needs to build something, it looks up the spec in a registry.

docs_dev/readme_toplevel.md

@@ -1,6 +1,6 @@
 # Logging in DUNE-DAQ — Documentation
 
-Welcome to the logging documentation for daqpytools (as of 5.6.0).
+Welcome to the logging documentation for daqpytools (as of fddaq-v5.6.0).
 
 This documentation is split into two sections depending on your role: