Event Observer API (#384)

* event classes, basic observer wip 1, mini unit test

* added utility functions for common event observer patterns

* add event types for request and response

* circ imp wip1

* apioptions management of observers + circ-imp refactor + unit tests

* some wiring to the astrapy database classes

* remove LOG observable event

* event observers, full wiring

* unit test for wired sync classes (not admin yet)

* all nonadmin ev-obs tests in place

* commander/observer attaching, testing complete

* sender in ev-obs; wiring for function name in ev-obs

* function name passed to ev-obs primitives

* additional metadata in request/response events for observers

* request id in observable events

* refactor event-observer API into a subdir

* context manager shorthand for capturing events

* add test for event_collector

* final testing and docstring for event_collector

* stricter arg around request/apicommander for ev-obs

* test-imports and README on ev-obs

* readme edits for ev-obs

* ev-obs: docstrings, last signature changes

* Add dev_ops_api bool flag to ObservableRequest

* changesfile and another note in the readme for ev-obs

* add fixes local mypy did not see (?)

---------

Co-authored-by: Stefano Lottini <236640031+sl-at-ibm@users.noreply.github.com>

Author: hemidactylus

CHANGES

@@ -1,5 +1,10 @@
 main
 ====
+Event Observer API, to listen to events:
+    - this complements the standard logging mechanism;
+    - events are: warnings, payloads/responses, ...
+    - useful for customized logging, observability and instrumentation;
+    - see `astrapy.event_observers` module and docstring of classes therein for more.
 mainteinance: enabled text-index integration tests on Astra
 maintenance: introduced the publish-and-release workflow machinery
 maintenance: split development/maintenance from user README

README.md

@@ -625,8 +625,9 @@ document = my_collection.find_one({"code": 123})
 
 ### Working with ObjectIds and UUIDs in Collections
 
-Astrapy repackages the ObjectId from `bson` and the UUID class and utilities
-from the `uuid` package and its `uuidv6` extension. You can also use them directly.
+Astrapy repackages the ObjectId from PyMongo's `bson` and the UUID class and
+utilities from the `uuid` package and its `uuidv6` extension. You can also
+use them directly.
 
 Even when setting a default ID type for a collection, you still retain the freedom
 to use any ID type for any document:
@@ -687,6 +688,116 @@ print(unescape_field_path("a&&&.b.c.d.12"))
 # prints: ['a&.b', 'c', 'd', '12']
 ```
 
+### Logging, instrumentation and observability
+
+#### Logging
+
+AstraPy's actions are logged using the standard `logging` facilities.
+To control the verbosity (logging level), a script preamble can have something
+like:
+
+```python
+import logging
+logging.basicConfig(level=logging.DEBUG)
+```
+
+#### Event observers with context manager
+
+When logging does not suffice, you can use the **Event Observer API**, i.e. attach
+an observer to the main astrapy class hierarchy (clients, databases,
+tables/collections; plus the admin classes).
+
+Events are, for example, errors and warnings, as well as requests sent
+and responses received.
+
+Events are generally issued during the lifecycle of a Data API or DevOps API
+HTTP request. Note that events are in correspondence with individual requests,
+and not class methods, some of which in fact involve multiple requests (e.g. the
+chunked `insert_many` or the database admin's `create_database` with
+`wait_until_active=True`).
+
+The simplest usage, when one needs to capture events for a limited
+span of statements, is through a context manager:
+
+```python
+from astrapy.event_observers import event_collector, ObservableEvent
+
+ev_lst: list[ObservableEvent] = []
+
+with event_collector(db, destination=event_list) as instrumented_db:
+    # Use instrumented_db: call its methods, spawn and use collections, etc:
+    instrumented_table = instrumented_db.get_table("my_table")
+    instrumented_table.insert_one({"id": 123, "name": "Fox"})
+
+for event in event_list:
+    print(event.event_type)
+```
+
+For more details and options, see the docstring of
+the `event_collector` utility.
+
+#### Event observers with utility factories
+
+An event observer can be created explicitly and attached through the API Options.
+
+```python
+from astrapy.api_options import APIOptions
+from astrapy.event_observers import (
+    ObservableEvent,
+    ObservableEventType,
+    Observer,
+)
+
+my_event_list: list[ObservableEvent] = []
+my_observer = Observer.from_event_list(
+    my_event_list,
+    # optional filtering by event type:
+    event_types=[ObservableEventType.REQUEST, ObservableEventType.RESPONSE],
+)
+
+instrumented_client = my_client.with_options(
+    api_options=APIOptions(event_observers={"my_obs001": my_observer})
+)
+
+# any event of the desired type emitted by 'instrumented_client',
+# or by classes it has spawned, will be accumulated into 'my_event_list'.
+```
+
+See also method `Observer.from_event_dict` for a similar alternative.
+
+This approach allows attaching multiple observers, if needed.
+
+#### Fully customized event observer
+
+For higher control, one can subclass `Observer` directly:
+
+```python
+from typing import Any
+from astrapy.api_options import APIOptions
+from astrapy.event_observers import ObservableEvent, Observer
+
+class MyObserver(Observer):
+    def receive(
+        self,
+        event: ObservableEvent,
+        sender: Any = None,
+        function_name: str | None = None,
+        request_id: str | None = None,
+    ) -> None:
+        received_item = {
+            "event_type": event.event_type.value,
+            "sender": sender,
+            "function_name": function_name,
+            "request_id": request_id,
+        }
+        print(f"Just received: {received_item}")
+
+my_observer = MyObserver()
+instrumented_collection = my_collection.with_options(
+    api_options=APIOptions(event_observers={"custom_obs001": my_observer})
+)
+```
+
 ## Appendices
 
 ### Appendix A: quick reference for key imports
@@ -857,6 +968,21 @@ from astrapy.authentication import (
 )
 ```
 
+Event observer API:
+
+```python
+from astrapy.event_observers import (
+    event_collector,
+    ObservableEventType,
+    ObservableEvent,
+    ObservableError,
+    ObservableWarning,
+    ObservableRequest,
+    ObservableResponse,
+    Observer,
+)
+```
+
 Miscellaneous utilities:
 
 ```python