diff --git a/docs/concepts/event-bus.rst b/docs/concepts/event-bus.rst index d63e84f8..0b676628 100644 --- a/docs/concepts/event-bus.rst +++ b/docs/concepts/event-bus.rst @@ -1,5 +1,7 @@ .. include:: ../common_refs.rst +.. _Event Bus: + Open edX Event Bus #################### @@ -29,7 +31,7 @@ The :term:`Event Bus` can help us achieve loose coupling between services, repla How Does the Open edX Event Bus Work? *************************************** -The Open edX platform uses the ``OpenEdxPublicSignals`` (Open edX-specific Django Signals) to send events within a service. The event bus extends these signals, allowing them to be broadcasted and handled across multiple services. That's how Open edX Events are used for internal and external communication. For more details on how these Open edX-specific Django Signals are used by the event bus, refer to the :doc:`../decisions/0004-external-event-bus-and-django-signal-events` Architectural Decision Record (ADR). +The Open edX platform uses the ``OpenEdxPublicSignals`` (Open edX-specific Django Signals) to send events within a service. The event bus extends these signals, allowing them to be broadcasted and handled across multiple services. That's how Open edX Events are used for internal and external communication. For more details on how these Open edX-specific Django Signals are used by the event bus, refer to the :ref:`ADR-4` Architectural Decision Record (ADR). Open edX Events provides an abstract implementation of the `publish/subscribe messaging pattern`_ (pub/sub), which is the chosen pattern for the event bus implementation, as explained in :doc:`openedx-proposals:architectural-decisions/oep-0052-arch-event-bus-architecture`. It implements two abstract classes, `EventProducer`_ and `EventConsumer`_, which allow concrete implementations of the event bus based on different message brokers, such as Pulsar. @@ -78,7 +80,7 @@ From Service B (Consumer) 2. When a new message is found, the ``EventConsumer`` deserializes the message and re-emits it as an event with the data that was transmitted. 3. The event sending and processing workflow repeats in Service B. -This approach of producing events via settings with the generic handler was chosen to allow for flexibility in the event bus implementation. It allows developers to choose the event bus implementation that best fits their needs, and easily switch between implementations if necessary. See more details in the :doc:`../decisions/0012-producing-to-event-bus-via-settings` Architectural Decision Record (ADR). +This approach of producing events via settings with the generic handler was chosen to allow for flexibility in the event bus implementation. It allows developers to choose the event bus implementation that best fits their needs, and easily switch between implementations if necessary. See more details in the :ref:`ADR-12` Architectural Decision Record (ADR). Event Bus vs Asynchronous Tasks ******************************** @@ -123,7 +125,7 @@ How is the Open edX Event Bus Used? The event bus is used to broadcast Open edX Events to multiple services, allowing them to react to changes or actions in the system. -We encourage you to review the :doc:`../reference/real-life-use-cases` page for examples of how the community uses the event bus in the Open edX ecosystem. Also, see the :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events` guide to start sending events to the event bus. +We encourage you to review the :ref:`Real-Life Use Cases` page for examples of how the community uses the event bus in the Open edX ecosystem. Also, see the :ref:`Use the Open edX Event Bus to Broadcast and Consume Events` guide to start sending events to the event bus. .. _general_signal_handler: https://github.com/openedx/openedx-events/blob/main/openedx_events/apps.py#L16-L44 .. _EventProducer: https://github.com/openedx/openedx-events/blob/main/openedx_events/event_bus/__init__.py#L71-L91 diff --git a/docs/concepts/index.rst b/docs/concepts/index.rst index 2eec03d9..991029cd 100644 --- a/docs/concepts/index.rst +++ b/docs/concepts/index.rst @@ -1,3 +1,5 @@ +.. _Concepts: + Concepts ======== diff --git a/docs/concepts/openedx-events.rst b/docs/concepts/openedx-events.rst index e9b3952c..016e3120 100644 --- a/docs/concepts/openedx-events.rst +++ b/docs/concepts/openedx-events.rst @@ -18,12 +18,12 @@ These events are built on top of Django signals, inheriting their behavior while .. note:: Django includes a "signal dispatcher", which helps decoupled applications get notified when actions occur elsewhere in the framework. In a nutshell, signals allow certain senders to notify a set of receivers that some action has taken place. They're especially useful when many pieces of code may be interested in the same events. -Events are primarily used as a communication method between internal services by leveraging Django Signals and external services using the :doc:`../concepts/event-bus`, making them the standard communication mechanism within the Open edX ecosystem. +Events are primarily used as a communication method between internal services by leveraging Django Signals and external services using the :ref:`Event Bus`, making them the standard communication mechanism within the Open edX ecosystem. How Do Open edX Events Work? **************************** -Open edX Events are implemented by a class called `OpenEdxPublicSignal`_, which inherits from `Django's Signals class` and adds behaviors specific to the Open edX ecosystem. Thanks to this design, ``OpenEdxPublicSignal`` leverages the functionality of Django signals, allowing developers to apply their existing knowledge of the Django framework. You can review the :doc:`Open edX Events Tooling <../reference/events-tooling>` documentation for more information on the tooling available for working with Open edX events. +Open edX Events are implemented by a class called `OpenEdxPublicSignal`_, which inherits from `Django's Signals class` and adds behaviors specific to the Open edX ecosystem. Thanks to this design, ``OpenEdxPublicSignal`` leverages the functionality of Django signals, allowing developers to apply their existing knowledge of the Django framework. You can review the :ref:`Events Tooling` documentation for more information on the tooling available for working with Open edX events. .. _events architecture: @@ -79,9 +79,9 @@ The `Django Signals Documentation`_ provides a more detailed explanation of how How Are Open edX Events Used? ***************************** -As mentioned previously, developers can listen to Open edX Events by registering signal receivers from their Open edX Django plugins that respond to the emitted events or by using the :doc:`../concepts/event-bus` to send events to external services. +As mentioned previously, developers can listen to Open edX Events by registering signal receivers from their Open edX Django plugins that respond to the emitted events or by using the :ref:`Event Bus` to send events to external services. -For more information on using Open edX Events, refer to the :doc:`../how-tos/create-a-new-event` how-to guide. We also encourage you to explore the :doc:`../reference/real-life-use-cases` section for real-life examples of how Open edX Events are used by the community. +For more information on using Open edX Events, refer to the :ref:`Create a New Open edX Event with Long-Term Support` how-to guide. We also encourage you to explore the :ref:`Real-Life Use Cases` section for real-life examples of how Open edX Events are used by the community. .. _triggering the COURSE_ENROLLMENT_CREATED event: https://github.com/openedx/edx-platform/blob/master/common/djangoapps/student/models/course_enrollment.py#L777-L795 .. _course_enrollment_post_save receiver: https://github.com/openedx/edx-platform/blob/master/openedx/core/djangoapps/notifications/handlers.py#L38-L53 diff --git a/docs/decisions/0002-events-naming-and-versioning.rst b/docs/decisions/0002-events-naming-and-versioning.rst index a13cc7a8..cd980311 100644 --- a/docs/decisions/0002-events-naming-and-versioning.rst +++ b/docs/decisions/0002-events-naming-and-versioning.rst @@ -1,3 +1,5 @@ +.. _ADR-2: + 2. Open edX events naming and versioning ======================================== diff --git a/docs/decisions/0003-events-payload.rst b/docs/decisions/0003-events-payload.rst index f9529886..994a796e 100644 --- a/docs/decisions/0003-events-payload.rst +++ b/docs/decisions/0003-events-payload.rst @@ -1,3 +1,5 @@ +.. _ADR-3: + 3. Open edX events payload conventions ====================================== diff --git a/docs/decisions/0004-external-event-bus-and-django-signal-events.rst b/docs/decisions/0004-external-event-bus-and-django-signal-events.rst index 4d81984c..0862b39f 100644 --- a/docs/decisions/0004-external-event-bus-and-django-signal-events.rst +++ b/docs/decisions/0004-external-event-bus-and-django-signal-events.rst @@ -1,3 +1,5 @@ +.. _ADR-4: + 4. External event bus and Django Signal events ============================================== @@ -17,7 +19,7 @@ Decision -------- Note that for purposes of this ADR, ``event`` refers to a dictionary of data that is emitted by a Django signal, specifically by an instance of ``OpenEdxPublicSignal`` -- Event definitions will be shared between internal and external events. The event definitions will continue to be in the form of ``OpenEdxPublicSignal``, as decided in ADR ":doc:`0003-events-payload`". +- Event definitions will be shared between internal and external events. The event definitions will continue to be in the form of ``OpenEdxPublicSignal``, as decided in ADR ":ref:`ADR-3`". - At this time, events will be published to the event bus from a Django signal handler. That is, when an internal event is sent via a Django signal, one of the handlers will take the event and publish it to the event bus. @@ -32,7 +34,7 @@ Consequences - An external event will never be sent without a corresponding internal event (at this time, based on the current design). -- The external event bus handler will listen for Django signals (``OpenEdxPublicSignals``) that have been configured to be sent as external events. These external events will be serialized using utilities in this library, as decided in ADR ":doc:`0005-external-event-schema-format`", before sending the messages over the wire. +- The external event bus handler will listen for Django signals (``OpenEdxPublicSignals``) that have been configured to be sent as external events. These external events will be serialized using utilities in this library, as decided in ADR ":ref:`ADR-5`", before sending the messages over the wire. - The use of the ``OpenEdxPublicSignal`` on both the event producing and event consuming sides for external events should hopefully provide a consistent mechanism to plug in for events. diff --git a/docs/decisions/0005-external-event-schema-format.rst b/docs/decisions/0005-external-event-schema-format.rst index 84257072..cf8a5ea0 100644 --- a/docs/decisions/0005-external-event-schema-format.rst +++ b/docs/decisions/0005-external-event-schema-format.rst @@ -1,3 +1,5 @@ +.. _ADR-5: + 5. External Event Schema Format =============================== @@ -12,7 +14,7 @@ Context * It is a best practice to use an explicit schema definition. Avro is the recommended serialization format for Kafka. -* The attrs objects that we currently have for signal-based events don't easily inter-operate with event bus client objects. Source ADR ":doc:`0003-events-payload`". +* The attrs objects that we currently have for signal-based events don't easily inter-operate with event bus client objects. Source ADR ":ref:`ADR-3`". * Industry best practices seem to suggest using a binary encoding of messages. diff --git a/docs/decisions/0006-event-schema-serialization-and-evolution.rst b/docs/decisions/0006-event-schema-serialization-and-evolution.rst index 1d22a537..e43c9971 100644 --- a/docs/decisions/0006-event-schema-serialization-and-evolution.rst +++ b/docs/decisions/0006-event-schema-serialization-and-evolution.rst @@ -1,3 +1,5 @@ +.. _ADR-6: + 6. Event schema serialization and evolution =========================================== @@ -17,9 +19,9 @@ Currently, the specification for events are written as OpenEdxPublicSignal insta Over time, as needs change, applications will need the schema for an event to change. -For internal events, there is some discussion of versioning events related to how they might change in ADR ":doc:`0002-events-naming-and-versioning`". For external events to be sent using an event bus, the same OpenEdxPublicSignal definitions will provide the schema. +For internal events, there is some discussion of versioning events related to how they might change in ADR ":ref:`ADR-2`". For external events to be sent using an event bus, the same OpenEdxPublicSignal definitions will provide the schema. -However, for the event bus, we will be introducing more rigorous event evolution rules using an explicit schema and schema registry. We had decided to use Avro in ":doc:`0005-external-event-schema-format`". +However, for the event bus, we will be introducing more rigorous event evolution rules using an explicit schema and schema registry. We had decided to use Avro in ":ref:`ADR-5`". Event schemas can evolve in various ways and each implementation of an event bus should choose a schema evolution configuration. The choices include how data can change, as well as the order in which producer and consumer needs to be deployed with new changes. diff --git a/docs/decisions/0007-optional-fields.rst b/docs/decisions/0007-optional-fields.rst index a94f9d93..4546c240 100644 --- a/docs/decisions/0007-optional-fields.rst +++ b/docs/decisions/0007-optional-fields.rst @@ -37,5 +37,5 @@ Consequences Deferred/Rejected Decisions --------------------------- -- Top-level fields will continue to be required, as stated in ":doc:`0006-event-schema-serialization-and-evolution`". We continue to defer the possibility of making these optional. +- Top-level fields will continue to be required, as stated in ":ref:`ADR-6`". We continue to defer the possibility of making these optional. - We are also deferring any work around making an explicit ``Optional`` keyword for attrs fields diff --git a/docs/decisions/0010-multiple-event-types-per-topic.rst b/docs/decisions/0010-multiple-event-types-per-topic.rst index 73600645..19cf3666 100644 --- a/docs/decisions/0010-multiple-event-types-per-topic.rst +++ b/docs/decisions/0010-multiple-event-types-per-topic.rst @@ -1,3 +1,5 @@ +.. _ADR-10: + 10. Multiple event types per topic ################################## diff --git a/docs/decisions/0012-producing-to-event-bus-via-settings.rst b/docs/decisions/0012-producing-to-event-bus-via-settings.rst index a9fe834e..8950d5f1 100644 --- a/docs/decisions/0012-producing-to-event-bus-via-settings.rst +++ b/docs/decisions/0012-producing-to-event-bus-via-settings.rst @@ -1,3 +1,5 @@ +.. _ADR-12: + 12. Enable producing to event bus via settings ############################################## diff --git a/docs/decisions/0016-event-design-practices.rst b/docs/decisions/0016-event-design-practices.rst index 7a52dec1..46779b54 100644 --- a/docs/decisions/0016-event-design-practices.rst +++ b/docs/decisions/0016-event-design-practices.rst @@ -1,3 +1,5 @@ +.. _ADR-16: + 16. Event Design Best Practices ############################### @@ -57,7 +59,7 @@ Responsibility and Granularity - Design events with a single responsibility in mind. Each event should represent a single action or fact that happened in the system. If an event contains multiple actions, consider splitting it into multiple events. For instance, if the course grade is updated to pass or fail, there should be two events: one for the pass action and another for the fail action. -.. note:: For the :doc:`Event Bus <../concepts/event-bus>`, events that are split across multiple actions are an exceptional case where the same event :term:`Topic` should be used to help maintain order across these events. +.. note:: For the :ref:`Event Bus `, events that are split across multiple actions are an exceptional case where the same event :term:`Topic` should be used to help maintain order across these events. - Manage the granularity of the event so it is not too coarse (generic with too much information) or too fine-grained (specific with too little information). When making a decision on the granularity of the event, start with the minimum required information for consumers to react to the event and add more information as needed with enough justification. If necessary, leverage API calls from the consumer side to retrieve additional information but always consider the trade-offs of adding dependencies with other services. - Ensure that the triggering logic is consistent and narrow. For instance, if an event is triggered when a user enrolls in a course, it should be triggered when the user enrolls in a course in all ways possible to enroll in a course. If the event is triggered when a user enrolls in a course through the API, it should also be triggered when the user enrolls in a course through the UI. diff --git a/docs/decisions/index.rst b/docs/decisions/index.rst index 6a769794..67ed465b 100644 --- a/docs/decisions/index.rst +++ b/docs/decisions/index.rst @@ -1,3 +1,5 @@ +.. _ADRs: + Architectural Decision Records (ADRs) ##################################### diff --git a/docs/how-tos/add-event-bus-support-to-an-event.rst b/docs/how-tos/add-event-bus-support-to-an-event.rst index 33c1eb0e..83dc4437 100644 --- a/docs/how-tos/add-event-bus-support-to-an-event.rst +++ b/docs/how-tos/add-event-bus-support-to-an-event.rst @@ -1,14 +1,16 @@ .. include:: ../common_refs.rst +.. _Add Event Bus Support: + Add Event Bus Support to an Open edX Event ############################################ Before sending an event across services, you need to ensure that the event is compatible with the Open edX Event Bus. This involves ensuring that the event, with its corresponding payload, can be emitted by a service through the event bus and that other services can consume it. This guide will walk you through the process of adding event bus support to an Open edX event. -For more details on how the :term:`Event Payload` is structured, refer to the :doc:`../decisions/0003-events-payload` decision record. +For more details on how the :term:`Event Payload` is structured, refer to the :ref:`ADR-3` decision record. .. note:: - This guide assumes that you have already created an Open edX event. If you have not, refer to the :doc:`../how-tos/create-a-new-event` how-to guide. + This guide assumes that you have already created an Open edX event. If you have not, refer to the :ref:`Create a New Open edX Event with Long-Term Support` how-to guide. Step 1: Does my Event Need Event Bus Support? =============================================== @@ -32,7 +34,7 @@ Step 2: Define the Event Payload An Open edX Event is compatible with the event bus when its payload can be serialized, sent, and deserialized by other services. The payload, structured as `attrs data classes`_, must align with the event bus schema format, which in this case is the :term:`Avro Schema`. This schema is used to serialize and deserialize the :term:`Event Payload` when sending it across services. -This ensures the event can be sent by the producer and then re-emitted by the same instance of |OpenEdxPublicSignal| on the consumer side, guaranteeing that the data sent and received is identical. Serializing this way should prevent data inconsistencies between services, e.g., timezone issues and precision loss. For more information on the event bus schema format, refer to the :doc:`../decisions/0004-external-event-bus-and-django-signal-events` and :doc:`../decisions/0005-external-event-schema-format` decision records. +This ensures the event can be sent by the producer and then re-emitted by the same instance of |OpenEdxPublicSignal| on the consumer side, guaranteeing that the data sent and received is identical. Serializing this way should prevent data inconsistencies between services, e.g., timezone issues and precision loss. For more information on the event bus schema format, refer to the :ref:`ADR-4` and :ref:`ADR-5` decision records. The data types used in the attrs classes that the current Open edX Event Bus with the chosen schema are: @@ -59,7 +61,7 @@ In the ``data.py`` files within each architectural subdomain, you can find examp Step 3: Ensure Serialization and Deserialization ================================================== -Before sending the event across services, you need to ensure that the :term:`Event Payload` can be serialized and deserialized correctly. The event bus concrete implementations use the :term:`Avro Schema` to serialize and deserialize the :term:`Event Payload` as mentioned in the :doc:`../decisions/0005-external-event-schema-format` decision record. The concrete implementation of the event bus handles serialization and deserialization with the help of methods implemented by this library. +Before sending the event across services, you need to ensure that the :term:`Event Payload` can be serialized and deserialized correctly. The event bus concrete implementations use the :term:`Avro Schema` to serialize and deserialize the :term:`Event Payload` as mentioned in the :ref:`ADR-5` decision record. The concrete implementation of the event bus handles serialization and deserialization with the help of methods implemented by this library. If you are interested in how the serialization and deserialization of the :term:`Event Payload` are handled by the event bus, you can refer to the concrete event bus implementation in the Open edX Event Bus repository. For example, here's how the Redis event bus handles `serialization`_ and `deserialization`_ when sending and receiving events. @@ -126,7 +128,7 @@ Run ``python manage.py generate_avro_schemas --help`` to see the available optio Step 5: Send the Event Across Services with the Event Bus ========================================================== -To validate that you can consume the event emitted by a service through the event bus, you can send the event across services. Here is an example of how you can send the event across services using the Redis event bus implementation following the `setup instructions in a Tutor environment`_. We recommend also following :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events` to understand how to use the event bus in your environment. +To validate that you can consume the event emitted by a service through the event bus, you can send the event across services. Here is an example of how you can send the event across services using the Redis event bus implementation following the `setup instructions in a Tutor environment`_. We recommend also following :ref:`Use the Open edX Event Bus to Broadcast and Consume Events` to understand how to use the event bus in your environment. .. note:: If you implemented a custom serializer for a type in the :term:`Event Payload`, the custom serializer support must be included in both the producer and consumer sides before it can be used. diff --git a/docs/how-tos/add-new-event-bus-concrete-implementation.rst b/docs/how-tos/add-new-event-bus-concrete-implementation.rst index e6502b4b..40ec918d 100644 --- a/docs/how-tos/add-new-event-bus-concrete-implementation.rst +++ b/docs/how-tos/add-new-event-bus-concrete-implementation.rst @@ -1,3 +1,5 @@ +.. _Add New Implementation of the Event Bus: + Add a New Concrete Implementation of the Event Bus ################################################### diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 8ea10df3..97f3a42f 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -1,11 +1,13 @@ .. include:: ../common_refs.rst +.. _Consume an Event: + Consume an Open edX Event ########################## -You have two ways of consuming an Open edX event: within the same service or in a different service. In this guide, we will show you how to consume an event within the same service. For consuming events across services, see :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. +You have two ways of consuming an Open edX event: within the same service or in a different service. In this guide, we will show you how to consume an event within the same service. For consuming events across services, see :ref:`Use the Open edX Event Bus to Broadcast and Consume Events`. -.. note:: We encourage you to also consider the practices outlined in the :doc:`../decisions/0016-event-design-practices` ADR for event consumption. +.. note:: We encourage you to also consider the practices outlined in the :ref:`ADR-16` ADR for event consumption. Throughout this guide, we will implement the use case to send the enrollment data to a webhook when a user enrolls in a course to better illustrate the steps involved in creating a consumer for an event. @@ -15,7 +17,7 @@ Assumptions - You have a development environment set up using `Tutor`_. - You have a basic understanding of Python and Django. - You have a basic understanding of Django signals. If not, you can review the `Django Signals Documentation`_. -- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Event Receiver`. If not, you can review the :doc:`../reference/glossary` documentation. +- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Event Receiver`. If not, you can review the :ref:`Glossary` documentation. Steps ******* diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 6cd08fbc..6970bfb3 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -1,20 +1,22 @@ .. include:: ../common_refs.rst +.. _Create a New Open edX Event with Long-Term Support: + Create a New Open edX Event with Long-Term Support #################################################### -Open edX Events are supported and maintained by the Open edX community. This mechanism is designed to be extensible and flexible, allowing developers to create new events that other services can consume. This guide describes how to create a new Open edX event with long-term support by following the practices outlined in the :doc:`../decisions/0016-event-design-practices` ADR. +Open edX Events are supported and maintained by the Open edX community. This mechanism is designed to be extensible and flexible, allowing developers to create new events that other services can consume. This guide describes how to create a new Open edX event with long-term support by following the practices outlined in the :ref:`ADR-16` ADR. Events designed with long-term support closely follow the practices described in the ADR to minimize breaking changes and maximize compatibility and support for future Open edX versions. -.. note:: Before starting, ensure you have reviewed the documentation on :doc:`docs.openedx.org:developers/concepts/hooks_extension_framework`, this documentation helps you decide if creating a new event is necessary. You should also review the documentation on :doc:`../decisions/0016-event-design-practices` to understand the practices that should be followed when creating a new event. +.. note:: Before starting, ensure you have reviewed the documentation on :ref:`docs.openedx.org:Hooks Extension Framework`, this documentation helps you decide if creating a new event is necessary. You should also review the documentation on :ref:`ADR-16` to understand the practices that should be followed when creating a new event. Throughout this guide, we will use an example of creating a new event that will be triggered when a user enrolls in a course from the course about page to better illustrate the steps involved in creating a new event. Key Outlines from Event Design Practices ****************************************** -The :doc:`../decisions/0016-event-design-practices` outlines the following key practices to follow when creating a new event: +The :ref:`ADR-16` outlines the following key practices to follow when creating a new event: - Clearly describe what happened and why. - Self-descriptive and self-contained as much as possible. @@ -36,9 +38,9 @@ Assumptions - You have a development environment set up using `Tutor`_. - You have a basic understanding of Python and Django. - You have a basic understanding of Django signals. If not, you can review the `Django Signals Documentation`_. -- You understand the concept of events or have reviewed the relevant :doc:`/concepts/index` docs. -- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Event Payload`. If not, you can review the :doc:`../reference/glossary` docs. -- You have reviewed the :doc:`../decisions/0016-event-design-practices` ADR. +- You understand the concept of events or have reviewed the relevant :ref:`Concepts` docs. +- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Event Payload`. If not, you can review the :ref:`Glossary` docs. +- You have reviewed the :ref:`ADR-16` ADR. - You have identified that you need to create a new event and have a use case for the event. Steps @@ -65,9 +67,9 @@ If you are confident that the event benefits the community, you can proceed to t Step 2: Place Your Event In an Architecture Subdomain ======================================================= -To implement the new event in the library, you should understand the purpose of the event and where it fits in the Open edX main architecture subdomains. This will help you place the event in the right architecture subdomain and ensure that the event is consistent with the framework's definitions. For more details on the Open edX Architectural Subdomains, refer to the :doc:`../reference/architecture-subdomains`. +To implement the new event in the library, you should understand the purpose of the event and where it fits in the Open edX main architecture subdomains. This will help you place the event in the right architecture subdomain and ensure that the event is consistent with the framework's definitions. For more details on the Open edX Architectural Subdomains, refer to the :ref:`Architecture Subdomains Reference`. -In our example, the event is related to the enrollment process, which is part of the ``learning`` subdomain. Therefore, the event should be placed in the ``/learning`` module in the library. The subdomain is also used as part of the :term:`event type `, which is used to identify the event. The event type should be unique and follow the naming convention for event types specified in the :doc:`../decisions/0002-events-naming-and-versioning` ADR. +In our example, the event is related to the enrollment process, which is part of the ``learning`` subdomain. Therefore, the event should be placed in the ``/learning`` module in the library. The subdomain is also used as part of the :term:`event type `, which is used to identify the event. The event type should be unique and follow the naming convention for event types specified in the :ref:`ADR-2` ADR. For the enrollment event, the event type could be ``org.openedx.learning.course.enrollment.v1``, where ``learning`` is the subdomain. @@ -87,7 +89,7 @@ For this, choose a specific point in the service where the event should be trigg Step 4: Determine the Content of the Event ============================================= -The event's content should comply with the practices outlined in the :doc:`../decisions/0016-event-design-practices`. The event should be self-descriptive and self-contained as much as possible. The event should contain all the necessary information for consumers to react to the event without having to make additional calls to other services when possible. +The event's content should comply with the practices outlined in the :ref:`ADR-16`. The event should be self-descriptive and self-contained as much as possible. The event should contain all the necessary information for consumers to react to the event without having to make additional calls to other services when possible. When determining the content of the event, consider the following: @@ -121,7 +123,7 @@ Implement the :term:`Event Definition` and :term:`Event Payload` for your event .. note:: Ideally, the data that is included in the event payload should be available at the time the event is triggered, and it should be directly related to the event that took place. So before defining the payload, inspect the triggering logic to review the data that is available at the time the event is triggered. -The event definition and payload must comply with the practices outlined in the :doc:`../decisions/0002-events-naming-and-versioning` and :doc:`../decisions/0003-events-payload` ADRs. Also, with the practices outlined in the :doc:`../decisions/0016-event-design-practices` ADR. Mainly: +The event definition and payload must comply with the practices outlined in the :ref:`ADR-2` and :ref:`ADR-3` ADRs. Also, with the practices outlined in the :ref:`ADR-16` ADR. Mainly: - The event should be self-descriptive and self-contained as much as possible. - The event should contain all the necessary information directly related to the event that took place. @@ -161,11 +163,11 @@ In our example, the event definition and payload for the enrollment event could - The payload should be an `attrs`_ class to ensure that the data is immutable by using the ``frozen=True`` argument and to ensure that the data is self-descriptive. - Use the ``attr.ib`` decorator to define the fields in the payload with the data type that the field should contain. Try to use the appropriate data type for each field to ensure that the data is consistent and maintainable, you can inspect the triggering logic to review the data that is available at the time the event is triggered. - Try using nested data classes to group related data together. This will help maintain consistency and make the event more readable. For instance, in the above example, we have grouped the data into User, Course, and Enrollment data. -- Try reusing existing data classes if possible to avoid duplicating data classes. This will help maintain consistency and reduce the chances of introducing errors. You can review the existing data classes in :doc:`../reference/events-data` to see if there is a data class that fits your use case. +- Try reusing existing data classes if possible to avoid duplicating data classes. This will help maintain consistency and reduce the chances of introducing errors. You can review the existing data classes in :ref:`Data Attributes` to see if there is a data class that fits your use case. - Each field in the payload should be documented with a description of what the field represents and the data type it should contain. This will help consumers understand the payload and react to the event. You should be able to justify why each field is included in the payload and how it relates to the event. - Use defaults for optional fields in the payload to ensure its consistency in all cases. -.. note:: When defining the payload, enforce :doc:`../concepts/event-bus` compatibility by ensuring that the data types used in the payload align with the event bus schema format. This will help ensure that the event can be sent by the producer and then be re-emitted by the same instance of `OpenEdxPublicSignal`_ on the consumer side, guaranteeing that the data sent and received is identical. For more information about adding event bus support to an event, refer to :doc:`../how-tos/add-event-bus-support-to-an-event`. +.. note:: When defining the payload, enforce :ref:`Event Bus` compatibility by ensuring that the data types used in the payload align with the event bus schema format. This will help ensure that the event can be sent by the producer and then be re-emitted by the same instance of `OpenEdxPublicSignal`_ on the consumer side, guaranteeing that the data sent and received is identical. For more information about adding event bus support to an event, refer to :ref:`Add Event Bus Support`. Event Definition ------------------ @@ -188,9 +190,9 @@ The :term:`Event Definition` should be implemented in the corresponding subdomai } ) -- The event definition should be documented using in-line documentation with at least ``event_type``, ``event_name``, ``event_key_field``, ``event_description``, ``event_data`` and ``event_trigger_repository``. This will help consumers understand the event and react to it. See :doc:`../reference/in-line-code-annotations-for-an-event` for more information. -- The :term:`Event Type` should be unique and follow the naming convention for event types specified in the :doc:`../decisions/0002-events-naming-and-versioning` ADR. This is used by consumers to identify the event. -- The ``event_name`` should be the variable name storing the event instance used to trigger the event. The name of the variable usually matches the ``{Subject}_{Action}`` of the event type. See more about the name in the :doc:`../decisions/0002-events-naming-and-versioning` ADR. +- The event definition should be documented using in-line documentation with at least ``event_type``, ``event_name``, ``event_key_field``, ``event_description``, ``event_data`` and ``event_trigger_repository``. This will help consumers understand the event and react to it. See :ref:`In-line Code Annotations` for more information. +- The :term:`Event Type` should be unique and follow the naming convention for event types specified in the :ref:`ADR-2` ADR. This is used by consumers to identify the event. +- The ``event_name`` should be the variable name storing the event instance used to trigger the event. The name of the variable usually matches the ``{Subject}_{Action}`` of the event type. See more about the name in the :ref:`ADR-2` ADR. - The ``event_key_field`` should be a field in the payload that uniquely identifies the event. This is used by consumers to identify the event. - The ``event_description`` should describe what the event is about and why it is triggered. - The ``event_data`` should be the payload class that is used to define the data that is included in the event. @@ -315,14 +317,14 @@ In our example, we could write a test that enrolls a user in a course and verifi Step 8: Consume the Event =========================== -Since the event is now implemented, you should consume it to verify that it is triggered and that the payload contains the necessary information. You can consume the event in a test environment using a Django Signal Receiver. This will help you verify that the event is triggered and that the payload contains the necessary information. You can follow the steps in :doc:`../how-tos/consume-an-event` to consume the event in a test environment with a Django Signal Receiver. You could also use the Open edX Event Bus to consume the event in a test environment. For more information on how to use the Open edX Event Bus, refer to the :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. +Since the event is now implemented, you should consume it to verify that it is triggered and that the payload contains the necessary information. You can consume the event in a test environment using a Django Signal Receiver. This will help you verify that the event is triggered and that the payload contains the necessary information. You can follow the steps in :ref:`Consume an Event` to consume the event in a test environment with a Django Signal Receiver. You could also use the Open edX Event Bus to consume the event in a test environment. For more information on how to use the Open edX Event Bus, refer to the :ref:`Use the Open edX Event Bus to Broadcast and Consume Events`. Step 9: Continue the Contribution Process ============================================ After implementing the event, you should continue the contribution process by creating a pull request in the repository. The pull requests should contain the changes you made to implement the event, including the event definition, payload, and the places where the event is triggered. -For more details on how the contribution flow works, refer to the :doc:`docs.openedx.org:developers/concepts/hooks_extension_framework` documentation. +For more details on how the contribution flow works, refer to the :ref:`docs.openedx.org:Hooks Extension Framework` documentation. .. _Add Extensibility Mechanism to IDV to Enable Integration of New IDV Vendor Persona: https://openedx.atlassian.net/wiki/spaces/OEPM/pages/4307386369/Proposal+Add+Extensibility+Mechanisms+to+IDV+to+Enable+Integration+of+New+IDV+Vendor+Persona .. _Add Program Certificate events: https://github.com/openedx/openedx-events/issues/250 diff --git a/docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst b/docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst index d7b32d03..4d828f57 100644 --- a/docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst +++ b/docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst @@ -1,11 +1,13 @@ .. include:: ../common_refs.rst +.. _Use the Open edX Event Bus to Broadcast and Consume Events: + Use the Open edX Event Bus to Broadcast and Consume Events ========================================================== -.. note:: Be sure to check out how to make your Open edX Event event bus compatible in the :doc:`../how-tos/add-event-bus-support-to-an-event` guide. +.. note:: Be sure to check out how to make your Open edX Event event bus compatible in the :ref:`Add Event Bus Support` guide. -After creating a new Open edX Event, you might need to send it across services instead of just within the same process. For this kind of use-cases, you might want to use the Open edX Event Bus. Here :doc:`../concepts/event-bus`, you can find useful information to start getting familiar with the Open edX Event Bus. +After creating a new Open edX Event, you might need to send it across services instead of just within the same process. For this kind of use-cases, you might want to use the Open edX Event Bus. Here :ref:`Event Bus`, you can find useful information to start getting familiar with the Open edX Event Bus. The Open edX Event Bus is a key component of the Open edX architecture, enabling services to communicate without direct dependencies on each other. This guide provides an overview of how to use the event bus to broadcast Open edX Events to multiple services, allowing them to react to changes or actions in the system. @@ -14,9 +16,9 @@ Assumptions - You have a development environment set up using `Tutor`_. - You have a basic understanding of Python and Django. -- You have basic understanding of the Open edX Event Bus. If not, you can review the :doc:`../concepts/event-bus` docs. -- You understand the concept of filters or have reviewed the relevant :doc:`/concepts/index` docs. -- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Topic`. If not, you can review the :doc:`/reference/glossary` docs. +- You have basic understanding of the Open edX Event Bus. If not, you can review the :ref:`Event Bus` docs. +- You understand the concept of filters or have reviewed the relevant :ref:`Concepts` docs. +- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Topic`. If not, you can review the :ref:`Glossary` docs. Steps ----- @@ -32,12 +34,12 @@ First, you need to install the Open edX Event Bus plugin in both the producing a pip install edx-event-bus-redis -.. note:: There are currently two community-supported concrete implementations of the Open edX Events Bus, Redis (`event-bus-redis`_) and Kafka (`event-bus-kafka`_). Redis is the default plugin for the Open edX Event Bus, but you can also use Kafka depending on your requirements. If none of these implementations meet your needs, you can implement your own plugin by following the :doc:`../how-tos/add-new-event-bus-concrete-implementation` documentation. +.. note:: There are currently two community-supported concrete implementations of the Open edX Events Bus, Redis (`event-bus-redis`_) and Kafka (`event-bus-kafka`_). Redis is the default plugin for the Open edX Event Bus, but you can also use Kafka depending on your requirements. If none of these implementations meet your needs, you can implement your own plugin by following the :ref:`Add New Implementation of the Event Bus` documentation. Step 2: Configure the Event Bus ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In :doc:`../reference/event-bus-configurations`, you can find the available configurations for the event bus that are used to set up the event bus in the Open edX platform. +In :ref:`Event Bus Configuration`, you can find the available configurations for the event bus that are used to set up the event bus in the Open edX platform. In both the producing and consuming services, you need to configure the event bus. This includes setting up the :term:`event types `, :term:`topics `, and other configurations for the :term:`Event Bus` to work with. In this step, you should configure the producer and consumer classes for the event bus implementation you are using: diff --git a/docs/index.rst b/docs/index.rst index 4c39b4e9..4f3265d3 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -3,10 +3,12 @@ You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. +.. _Open edX Events Docs: + Welcome to Open edX Events' Documentation! ############################################ -Open edX Events is a type of hook in the Hooks Extension Framework that allows extending the Open edX platform in a more stable and maintainable way. If you are new to this approach for extending Open edX, start by reading the :doc:`docs.openedx.org:developers/concepts/hooks_extension_framework` documentation. This documentation provides an overview of the framework's concepts and structure, which are useful to support your adoption of Open edX Events. +Open edX Events is a type of hook in the Hooks Extension Framework that allows extending the Open edX platform in a more stable and maintainable way. If you are new to this approach for extending Open edX, start by reading the :ref:`docs.openedx.org:Hooks Extension Framework` documentation. This documentation provides an overview of the framework's concepts and structure, which are useful to support your adoption of Open edX Events. This repository contains all the references, concepts, how-tos, quickstarts and Architectural Decision Records (ADRs) related to the Open edX Events necessary to implement this hook in your instance. diff --git a/docs/reference/architecture-subdomains.rst b/docs/reference/architecture-subdomains.rst index 45b6f496..ee68222b 100644 --- a/docs/reference/architecture-subdomains.rst +++ b/docs/reference/architecture-subdomains.rst @@ -1,3 +1,5 @@ +.. _Architecture Subdomains Reference: + Architecture Subdomains ########################## diff --git a/docs/reference/event-bus-configurations.rst b/docs/reference/event-bus-configurations.rst index 53e2019a..67f3d716 100644 --- a/docs/reference/event-bus-configurations.rst +++ b/docs/reference/event-bus-configurations.rst @@ -1,3 +1,5 @@ +.. _Event Bus Configuration: + Event Bus Configuration ######################## diff --git a/docs/reference/events-data.rst b/docs/reference/events-data.rst index c7d952da..823c5bbb 100644 --- a/docs/reference/events-data.rst +++ b/docs/reference/events-data.rst @@ -1,3 +1,5 @@ +.. _Data Attributes: + Open edX Events Data Attributes ################################ diff --git a/docs/reference/events-tooling.rst b/docs/reference/events-tooling.rst index a07ed2d7..d519870b 100644 --- a/docs/reference/events-tooling.rst +++ b/docs/reference/events-tooling.rst @@ -1,3 +1,5 @@ +.. _Events Tooling: + Open edX Events Tooling ######################## diff --git a/docs/reference/events.rst b/docs/reference/events.rst index f3fe9729..bcbaac4f 100644 --- a/docs/reference/events.rst +++ b/docs/reference/events.rst @@ -1,3 +1,5 @@ +.. _Existing Events: + Events ####### diff --git a/docs/reference/glossary.rst b/docs/reference/glossary.rst index 81cc8d41..1893e982 100644 --- a/docs/reference/glossary.rst +++ b/docs/reference/glossary.rst @@ -12,7 +12,7 @@ An event has multiple components that are used to define, trigger, and handle th An event trigger is the action that causes an event to be emitted. When a trigger action occurs, the associated event is emitted, and any registered event receivers are called to handle the event. For example, when a user enrolls in a course, the ``COURSE_ENROLLMENT_CREATED`` event is triggered. In this case, the event trigger is the user enrolling in the course. Event Payload - The event payload is the data associated with an event that is passed to event receivers when it's triggered. The payload of an event are data attribute classes (e.g. ``CourseEnrollmentData``, ``UserData``, etc.) that carry data about the event such as the event name, timestamp, and any additional metadata and information about the actual event. For more information, see the `Events Payload ADR`_. + The event payload is the data associated with an event that is passed to event receivers when it's triggered. The payload of an event are data attribute classes (e.g. ``CourseEnrollmentData``, ``UserData``, etc.) that carry data about the event such as the event name, timestamp, and any additional metadata and information about the actual event. For more information, see the :ref:`ADR-3`. Event Type The event type is a unique identifier for an event that distinguishes it from other events. For example, ``org.openedx.content_authoring.xblock.published.v1``. The event type is used to identify the event, its purpose, and version. In the event bus context, the event type is used to connect events to the appropriate topics in the ``EVENT_BUS_PRODUCER_CONFIG``. E.g., the event type ``org.openedx.learning.course.enrollment.created.v1`` is used to identify the ``COURSE_ENROLLMENT_CREATED`` event. @@ -47,5 +47,5 @@ An event has multiple components that are used to define, trigger, and handle th Avro Record Dictionary A dictionary whose structure is determined by an Avro schema. These dictionaries are the entities that are actually serialized to bytes and sent over the wire to the event bus. -.. _Events Payload ADR: :doc: `/decisions/0003-events-payload` + .. _Apache Avro: https://avro.apache.org/docs/current/spec.html diff --git a/docs/reference/in-line-code-annotations-for-an-event.rst b/docs/reference/in-line-code-annotations-for-an-event.rst index 8841c1d5..cfa9303b 100644 --- a/docs/reference/in-line-code-annotations-for-an-event.rst +++ b/docs/reference/in-line-code-annotations-for-an-event.rst @@ -1,3 +1,5 @@ +.. _In-line Code Annotations: + In-line Code Annotations for an Open edX Event ################################################ @@ -6,7 +8,7 @@ When creating a new Open edX Event, you must document the event definition using +-------------------------------------+----------------------------------------------------------------------------------------------------------------------------+ | Annotation | Description | +=====================================+============================================================================================================================+ -| event_type (Required) | Identifier across services of the event. Should follow the :doc:`../decisions/0002-events-naming-and-versioning` standard. | +| event_type (Required) | Identifier across services of the event. Should follow the :ref:`ADR-2` standard. | +-------------------------------------+----------------------------------------------------------------------------------------------------------------------------+ | event_name (Required) | Name of the variable storing the event instance. | +-------------------------------------+----------------------------------------------------------------------------------------------------------------------------+ diff --git a/docs/reference/naming-suggestions.rst b/docs/reference/naming-suggestions.rst index 4c07f036..f815140f 100644 --- a/docs/reference/naming-suggestions.rst +++ b/docs/reference/naming-suggestions.rst @@ -5,6 +5,6 @@ When naming a new event and contributing it back to this repository, consider th - Use a name that is descriptive to the event's purpose. For example, the event associated with a user's enrollment in a course is named ``COURSE_ENROLLMENT_CREATED``, which clearly indicates the event's purpose. - Use a name that is unique within the framework. -- Match the name to the ``event_type`` identifier. For example, the ``COURSE_ENROLLMENT_CREATED`` event has an ``event_type`` of ``org.openedx.learning.course.enrollment.created.v1``. You can use the ``event_type`` as a reference when naming the event. See :doc:`../decisions/0002-events-naming-and-versioning` for more information on naming and versioning events. +- Match the name to the ``event_type`` identifier. For example, the ``COURSE_ENROLLMENT_CREATED`` event has an ``event_type`` of ``org.openedx.learning.course.enrollment.created.v1``. You can use the ``event_type`` as a reference when naming the event. See :ref:`ADR-2` for more information on naming and versioning events. - Avoid using ``event`` in the name. It is implied that the variable is an event, so there is no need to include it in the name. -- Try reviewing the :doc:`existing events <../reference/events>` in the repository to ensure that the name you choose is unique and follows the naming conventions. +- Try reviewing the :ref:`existing events ` in the repository to ensure that the name you choose is unique and follows the naming conventions. diff --git a/docs/reference/real-life-use-cases.rst b/docs/reference/real-life-use-cases.rst index 7f06a9cd..d47da7cc 100644 --- a/docs/reference/real-life-use-cases.rst +++ b/docs/reference/real-life-use-cases.rst @@ -1,12 +1,14 @@ .. include:: ../common_refs.rst +.. _Real-Life Use Cases: + Real-Life Use Cases for Open edX Events ######################################## Overview ********** -As mentioned in the :doc:`docs.openedx.org:developers/concepts/hooks_extension_framework` docs, Open edX Events can be used to integrate application components with each other or with external services, allowing them to communicate, synchronize, and perform additional actions when specific triggers occur. +As mentioned in the :ref:`docs.openedx.org:Hooks Extension Framework` docs, Open edX Events can be used to integrate application components with each other or with external services, allowing them to communicate, synchronize, and perform additional actions when specific triggers occur. To illustrate the different solutions that can be implemented with this approach, we have compiled a list of use cases built using Open edX Events to address various challenges. The goal of this list is to serve as a reference for extension developers to implement their own solutions in their own plugins or IDAs based on the community's experience. @@ -18,7 +20,7 @@ The following list of real-life use cases showcases the different ways Open edX Cross-services communication ============================= -As mentioned in :doc:`../concepts/event-bus`, the suggested strategy for cross-service communication in the Open edX ecosystem is through an event-based architecture implemented via the :doc:`../concepts/event-bus`. This functionality used for asynchronous communication between services is built on top of sending Open edX Events (Open edX-specific Django signals) within a service. For more details on the Event Bus, please see :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. +As mentioned in :ref:`Event Bus`, the suggested strategy for cross-service communication in the Open edX ecosystem is through an event-based architecture implemented via the :ref:`Event Bus`. This functionality used for asynchronous communication between services is built on top of sending Open edX Events (Open edX-specific Django signals) within a service. For more details on the Event Bus, please see :ref:`Use the Open edX Event Bus to Broadcast and Consume Events`. Here are some examples of how the Event Bus can be used to facilitate communication between IDAs: