Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
baf302b
Start event-driven example
toby-coleman Mar 16, 2025
b29dd82
None
toby-coleman Mar 16, 2025
bdd4675
WIP diagram for event-driven example
toby-coleman Mar 16, 2025
a03c94a
Prepare snippets on code
toby-coleman Mar 16, 2025
d5c218f
Fix typo in docstring
toby-coleman Mar 16, 2025
c43dba9
Writeup event-driven tutorial
toby-coleman Mar 21, 2025
be278a6
Fix broken links
toby-coleman Mar 21, 2025
a51f9a6
Update for implemented feature
toby-coleman Mar 21, 2025
c9a2036
Update links
toby-coleman Mar 21, 2025
258bd33
Add YAML version
toby-coleman Mar 21, 2025
cb19233
Add info on logging to tutorial
toby-coleman Mar 21, 2025
956148d
Add tags
toby-coleman Mar 21, 2025
6814620
Add tags
toby-coleman Mar 21, 2025
0928c57
Add tags to demos
toby-coleman Mar 21, 2025
34f9b04
Add tag index
toby-coleman Mar 21, 2025
57cb065
Add car wash demo
toby-coleman Mar 22, 2025
400a22a
Update with text descriptions
toby-coleman Mar 22, 2025
3942f86
README fixes
toby-coleman Mar 22, 2025
9244758
Minor updates
toby-coleman Mar 22, 2025
7fb6e23
Fix logger for types not supported by msgspec
toby-coleman Mar 22, 2025
78cac07
Remove logging conversions
toby-coleman Mar 22, 2025
f75bf3a
Install ipywidgets to remove warnings in notebooks
toby-coleman Mar 22, 2025
21059df
Merge remote-tracking branch 'origin/main' into docs/event-based-example
toby-coleman Apr 1, 2025
b7692b0
Update diagram
toby-coleman Apr 1, 2025
11d3e02
Merge remote-tracking branch 'origin/main' into docs/event-based-example
toby-coleman Apr 10, 2025
dd8b96a
Merge remote-tracking branch 'origin/main' into docs/event-based-example
toby-coleman Sep 15, 2025
971a22c
Rename folder
toby-coleman Sep 15, 2025
642086a
Fixup event tutorial
toby-coleman Sep 16, 2025
9ffa0a0
Fixup carwash examples
toby-coleman Sep 16, 2025
729806e
Update LLM prompt
toby-coleman Sep 17, 2025
ad955e5
Unused import
toby-coleman Sep 17, 2025
65f85d4
Delete car wash example
toby-coleman Sep 23, 2025
80fcc85
Update instructions
toby-coleman Sep 23, 2025
8becf58
Swap out for momentum signal example
toby-coleman Oct 1, 2025
1c67def
Update text
toby-coleman Oct 1, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 82 additions & 1 deletion .github/instructions/models.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,4 +95,85 @@ import asyncio

async with process:
await process.run()
```
```

## Event-driven models

You can help users to implement event-driven models using Plugboard's event system. Components can emit and handle events to communicate with each other.

Examples of where you might want to use events include:
* A component that monitors a data stream and emits an event when a threshold is crossed;
* A component that listens for events and triggers actions in response, e.g. sending an alert;
* A trading algorithm that uses events to signal buy/sell decisions.

Events must be defined by inheriting from the `plugboard.events.Event` class. Each event class should define the data it carries using a Pydantic `BaseModel`. For example:

```python
from pydantic import BaseModel
from plugboard.events import Event

class MyEventData(BaseModel):
some_value: int
another_value: str

class MyEvent(Event):
data: MyEventData
```

Components can emit events using the `self.io.queue_event()` method or by returning them from an event handler. Event handlers are defined using methods decorated with `@EventClass.handler`. For example:

```python
from plugboard.component import Component, IOController as IO

class MyEventPublisher(Component):
io = IO(inputs=["some_input"], output_events=[MyEvent])

async def step(self) -> None:
# Emit an event
event_data = MyEventData(some_value=42, another_value=f"received {self.some_input}")
self.io.queue_event(MyEvent(source=self.name, data=event_data))

class MyEventSubscriber(Component):
io = IO(input_events=[MyEvent], output_events=[MyEvent])

@MyEvent.handler
async def handle_my_event(self, event: MyEvent) -> MyEvent:
# Handle the event
print(f"Received event: {event.data}")
output_event_data = MyEventData(some_value=event.data.some_value + 1, another_value="handled")
return MyEvent(source=self.name, data=output_event_data)
```

To assemble a process with event-driven components, you can use the same approach as for non-event-driven components. You will need to create connectors for event-driven components using `plugboard.events.event_connector_builder.EventConnectorBuilder`. For example:

```python
from plugboard.connector import AsyncioConnector, ConnectorBuilder
from plugboard.events.event_connector_builder import EventConnectorBuilder
from plugboard.process import LocalProcess

# Define components....
component_1 = ...
component_2 = ...

# Define connectors for non-event components as before
connect = lambda in_, out_: AsyncioConnector(spec=ConnectorSpec(source=in_, target=out_))
connectors = [
connect("component_1.output", "component_2.input"),
...
]

connector_builder = ConnectorBuilder(connector_cls=AsyncioConnector)
event_connector_builder = EventConnectorBuilder(connector_builder=connector_builder)
event_connectors = list(event_connector_builder.build(components).values())

process = LocalProcess(
components=[
component_1, component_2, ...
],
connectors=connectors + event_connectors,
)
```

## Exporting models

If the user wants to export their model you use in the CLI, you can do this by calling `process.dump("path/to/file.yaml")`.
79 changes: 78 additions & 1 deletion docs/examples/tutorials/event-driven-models.md
Original file line number Diff line number Diff line change
@@ -1 +1,78 @@
Tutorial coming soon.
---
tags:
- events
---
So far everything we have built in Plugboard has been a **discrete-time model**. This means that the whole model advances step-wise, i.e. `step` gets called on each [`Component`][plugboard.component.Component], calculating all of their outputs before advancing the simulation on.

In this tutorial we're going to introduce an **event-driven model**, where data can be passed around between components based on triggers that you can define. Event-based models can be useful in a variety of scenarios, for example when modelling parts moving around a production line, or to trigger expensive computation only when certain conditions are met in the model.

## Event-based model

Here's the model that we're going to build. Given a stream of random numbers, we'll trigger `HighEvent` whenever the value is above `0.8` and `LowEvent` whenever the value is below `0.2`. This allows us to funnel data into different parts of the model: in this case we'll just save the latest high/low values to a file at each step. In the diagram the _dotted lines_ represent the flow of event data: `FindHighLowValues` will publish events, while `CollectHigh` and `CollectLow` will subscribe to receive high and low events respectively.

```mermaid
flowchart LR
collect-high@{ shape: rounded, label: CollectHigh<br>**collect-high** } --> save-high@{ shape: rounded, label: FileWriter<br>**save-high** }
collect-low@{ shape: rounded, label: CollectLow<br>**collect-low** } --> save-low@{ shape: rounded, label: FileWriter<br>**save-low** }
random-generator@{ shape: rounded, label: Random<br>**random-generator** } --> find-high-low@{ shape: rounded, label: FindHighLowValues<br>**find-high-low** }
low_event@{ shape: hex, label: LowEvent } -.-> collect-low@{ shape: rounded, label: CollectLow<br>**collect-low** }
high_event@{ shape: hex, label: HighEvent } -.-> collect-high@{ shape: rounded, label: CollectHigh<br>**collect-high** }
find-high-low@{ shape: rounded, label: FindHighLowValues<br>**find-high-low** } -.-> high_event@{ shape: hex, label: HighEvent }
find-high-low@{ shape: rounded, label: FindHighLowValues<br>**find-high-low** } -.-> low_event@{ shape: hex, label: LowEvent }
```

## Defining events

First we need to define the events that are going to get used in the model. Each event needs a name, in this case `"high_event"` and `"low_event"` and a `data` type associated with it. Use a [Pydantic](https://docs.pydantic.dev/latest/) model to define the format of this `data` field.

```python
--8<-- "examples/tutorials/005_events/hello_events.py:events"
```

## Building components to create and consume events

So far all of our process models have run step-by-step until completion. When a model contains event-driven components, we need a way to tell them to stop at the end of the simulation, otherwise they will stay running and listening for events forever.

In this example, our `Random` component will drive the process by generating input random values. When it has completed `iters` iterations, we call `self.io.close()` to stop the model, causing other components in the model to shutdown.

```python
--8<-- "examples/tutorials/005_events/hello_events.py:source-component"
```

Next, we will define `FindHighLowValues` to identify high and low values in the stream of random numbers and publish `HighEvent` and `LowEvent` respectively.

```python
--8<-- "examples/tutorials/005_events/hello_events.py:event-publisher"
```

1. See how we use the [`IOController`][plugboard.component.IOController] to declare that this [`Component`][plugboard.component.Component] will publish events.
2. Use `self.io.queue_event` to send an event from a [`Component`][plugboard.component.Component]. Here we are senging the `HighEvent` or `LowEvent` depending on the input value.

Finally, we need components to subscribe to these events and process them. Use the `Event.handler` decorator to identify the method on each [`Component`][plugboard.component.Component] that will do this processing.

```python
--8<-- "examples/tutorials/005_events/hello_events.py:event-consumers"
```

1. Specify the events that this [`Component`][plugboard.component.Component] will subscribe to.
2. Use this decorator to indicate that we handle `HighEvent` here...
3. ...and we handle `LowEvent` here.

!!! note
In a real model you could define whatever logic you need inside your event handler, e.g. create a file, publish another event, etc. Here we just store the event on an attribute so that its value can be output via the `step()` method.

## Putting it all together

Now we can create a [`Process`][plugboard.process.Process] from all these components. The outputs from `CollectLow` and `CollectHigh` are connected to separate [`FileWriter`][plugboard.library.FileWriter] components so that we'll get a CSV file containing the latest high and low values at each step of the simulation.

!!! info
We need a few extra lines of code to create connectors for the event-based parts of the model. If you define your process in YAML this will be done automatically for you, but if you are defining the process in code then you will need to use the [`EventConnectorBuilder`][plugboard.events.EventConnectorBuilder] to do this.

```python hl_lines="15-17"
--8<-- "examples/tutorials/005_events/hello_events.py:main"
```

1. These connectors are for the normal, non-event driven parts of the model and connect [`Component`][plugboard.component.Component]` inputs and outputs.
2. These lines will set up connectors for the events in the model.

Take a look at the `high.csv` and `low.csv` files: the first few rows will usually be empty, and then as soon as high or low values are identified they will start to appear in the CSVs. As usual, you can run this model from the CLI using `plugboard process run model.yaml`.
2 changes: 2 additions & 0 deletions examples/demos/finance/.meta.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
tags:
- finance
1 change: 1 addition & 0 deletions examples/demos/finance/001_momentum_signal/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.csv
2 changes: 2 additions & 0 deletions examples/demos/finance/001_momentum_signal/.meta.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
tags:
- events
Loading
Loading