Skip to content

Document thread_subscriptions stream writer and fix documentation of quarantined_media_changes stream writer. #19746

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
reivilibre wants to merge 5 commits into
base: develop
Choose a base branch
from
+24 −3
Open

Document thread_subscriptions stream writer and fix documentation of quarantined_media_changes stream writer. #19746

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
220a1a3
Fix `quarantined_media_changes` not being configured for Complement
reivilibre Apr 30, 2026
5fc3678
Tweak streams cheatsheet
reivilibre Apr 30, 2026
eb6a81c
Document `thread_subscriptions` stream_writer
reivilibre Apr 30, 2026
d4d5b84
Fix documentation of `quarantined_media_changes` `stream_writer`
reivilibre Apr 30, 2026
a6140b3
Newsfile
reivilibre Apr 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/19746.doc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Document `thread_subscriptions` stream writer and fix documentation of `quarantined_media_changes` stream writer.
10 changes: 10 additions & 0 deletions docker/configure_workers_and_start.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -517,6 +517,9 @@ def add_worker_roles_to_shared_config(
"typing",
"push_rules",
"thread_subscriptions",
# We can't include `quarantined_media_changes` here
# because the worker type is actually called `media_repository`
# "quarantined_media_changes",
}

# Worker-type specific sharding config. Now a single worker can fulfill multiple
Expand All @@ -527,6 +530,13 @@ def add_worker_roles_to_shared_config(
if "federation_sender" in worker_types_set:
shared_config.setdefault("federation_sender_instances", []).append(worker_name)

if "media_repository" in worker_types_set:
# Handle media_repository being the writer for quarantined_media_changes
# as a special-case
shared_config.setdefault("stream_writers", {}).setdefault(
"quarantined_media_changes", []
Copy link
Copy Markdown
Contributor

@MadLittleMods MadLittleMods Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: #19746 (comment)

hmmm, does main make it into this list because the background job also writes to the stream.

Feel free to push this off to future PR's (not even for you to tackle)

).append(worker_name)

# Update the list of stream writers. It's convenient that the name of the worker
# type is the same as the stream to write. Iterate over the whole list in case there
# is more than one.
Expand Down
2 changes: 1 addition & 1 deletion docs/development/synapse_architecture/streams.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ necessary registration and event handling.
- [create a stream class and stream row class](https://github.com/element-hq/synapse/blob/4367fb2d078c52959aeca0fe6874539c53e8360d/synapse/replication/tcp/streams/_base.py#L728)
- will need an [ID generator](https://github.com/element-hq/synapse/blob/4367fb2d078c52959aeca0fe6874539c53e8360d/synapse/storage/databases/main/thread_subscriptions.py#L75)
- may need [writer configuration](https://github.com/element-hq/synapse/blob/4367fb2d078c52959aeca0fe6874539c53e8360d/synapse/config/workers.py#L177), if there isn't already an obvious source of configuration for which workers should be designated as writers to your new stream.
- if adding new writer configuration, add Docker-worker configuration, which lets us configure the writer worker in Complement tests: [[1]](https://github.com/element-hq/synapse/blob/4367fb2d078c52959aeca0fe6874539c53e8360d/docker/configure_workers_and_start.py#L331), [[2]](https://github.com/element-hq/synapse/blob/4367fb2d078c52959aeca0fe6874539c53e8360d/docker/configure_workers_and_start.py#L440)
- if adding new writer configuration (`stream_writers`), add Docker-worker configuration, which lets us configure the writer worker in Complement tests: [[1]](https://github.com/element-hq/synapse/blob/4367fb2d078c52959aeca0fe6874539c53e8360d/docker/configure_workers_and_start.py#L331), [[2]](https://github.com/element-hq/synapse/blob/4367fb2d078c52959aeca0fe6874539c53e8360d/docker/configure_workers_and_start.py#L440)
- most of the time, you will likely introduce a new datastore class for the concept represented by the new stream, unless there is already an obvious datastore that covers it.
- consider whether it may make sense to introduce a handler

Expand Down
14 changes: 12 additions & 2 deletions docs/workers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -576,10 +576,20 @@ configured as stream writer for the `device_lists` stream:
^/_matrix/client/(api/v1|r0|v3|unstable)/keys/device_signing/upload$
^/_matrix/client/(api/v1|r0|v3|unstable)/keys/signatures/upload$

##### The `thread_subscriptions` stream

This stream is only used for MSC4306 (experimental).
Copy link
Copy Markdown
Contributor

@MadLittleMods MadLittleMods Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This stream is only used for MSC4306 (experimental).
This stream is only used for [MSC4306](https://github.com/matrix-org/matrix-spec-proposals/pull/4306) (experimental).

It is not used when that experimental feature is disabled.
Copy link
Copy Markdown
Contributor

@MadLittleMods MadLittleMods Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
It is not used when that experimental feature is disabled.
It is not used when the `msc4306_enabled` experimental feature is disabled.


The `thread_subscriptions` stream supports multiple writers.
The following endpoints should be routed directly to one of the workers
configured as stream writer for the `thread_subscriptions` stream:

^/_matrix/client/unstable/io.element.msc4306/.*

##### The `quarantined_media_changes` stream

The `quarantined_media_changes` stream supports multiple writers. The following endpoints
can be handled by any worker, but should be routed directly to one of the workers
The `quarantined_media_changes` stream supports multiple writers. The following endpoints should be routed directly to one of the workers
Copy link
Copy Markdown
Contributor

@MadLittleMods MadLittleMods Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this should read like the presence stream above

Suggested change
The `quarantined_media_changes` stream supports multiple writers. The following endpoints should be routed directly to one of the workers
The `quarantined_media_changes` stream supports multiple writers.
The following endpoints should be routed directly to the workers configured as
the stream writer for the `quarantined_media_changes` stream:

Copy link
Copy Markdown
Contributor

@MadLittleMods MadLittleMods Apr 30, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an aside:

Re: https://github.com/matrix-org/matrix-ansible-private/pull/7898 (specifying the Synapse main process) as of the stream writers because of the background job,

stream_writers:
  quarantined_media_changes:
     - main
     - workers

We should also consider figuring out how to document the background jobs that also write to the stream

configured as stream writer for the `quarantined_media_changes` stream:

^/_synapse/admin/v1/quarantine_media/.*$
Expand Down
Loading