MSC4242: State DAGs (inbound joins) by kegsay · Pull Request #20194 · element-hq/synapse

kegsay · 2026-09-07T16:02:52Z

Process inbound requests to join a state DAG room.

Split out from #19425

Part of a series of 5x PRs to land the federation part of MSC4242 (#19718, #20127, #20133, inbound-joins (this PR), inbound-pulls).

Has some Complement tests:

PASS TestMSC4242JoinLeaveRejoinPublicRoom 9.19s
PASS TestMSC4242JoinPublicRoomWithRejectedStateDAGEvents 0.88s

Pull Request Checklist

Pull request is based on the develop branch
Pull request includes a changelog file. The entry should:
- Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
- Use markdown where necessary, mostly for code blocks.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
Code style is correct (run the linters)

MadLittleMods · 2026-09-29T23:36:35Z

+                    batched_auth_events = {
+                        event_id: event_map[event_id]


LLM is spotting another rejection cascade problem:

LLM description

Rejection doesn't cascade on rejoin (most serious).

Every event from the response goes into event_map as a fresh copy with rejected_reason=None. That includes events we already stored and rejected on an earlier join.

batched_auth_events is built from event_map (:2068). check_state_independent_auth_rules reads prev_state_events from batched_auth_events before it looks at the database (event_auth.py:204).

rejections only records decisions made in the current batch.

Result: a new event whose prev_state_events includes an event we rejected last time passes the check at event_auth.py:250. This reopens the concern from the earlier thread at :2082, just by a different route.

Suggested fix: when processing the seen events, fill rejections from the database with their reasons, or use the stored copies for seen events.

Previous problem: #20194 (comment)

2d9da36 - basically doing the suggested fix of replacing the wire-events with stored-events (which contain rejections).

Probably worth a test so we don't regress these subtleties

MadLittleMods · 2026-09-29T23:46:30Z

+                if from_send_join and all(
+                    prev_state_event_id in event_id_to_state_group
+                    for prev_state_event_id in event.prev_state_events
+                ):


LLM is mentioning that if an event's prev_state_events mix an event we already persisted and one that's new in this batch, we fall back to compute_state_after_events from the DB. But nothing in the batch is persisted until the end, so the new parent has no state group yet, and its branch is silently dropped from the state.

Also see #20194 (comment)

Probably worth a test so we don't regress these subtleties

MadLittleMods · 2026-09-29T23:49:50Z

+                # If this assumption is ever violated, `compute_state_after_events` will
+                # yell loudly about missing state groups.


LLM is calling out that _get_state_group_for_events raises RuntimeError for unknown events, but the @cachedList wrapper swallows it and the caller just gets partial results.

This sounds like a pre-existing bug with @cachedList but it makes our comment inaccurate and we should probably mark it with a FIXME with a link to a Synapse issue tracking this problem.

This is related to the comment #20194 (comment) as this should have caused a 500 but instead silently dropped the branch :/

0021dae

I just added a TODO for this as this PR is sprawling enough as it is.

The comment should be updated to call this out more clearly as something that's probably unexpected.

XXX: `_get_state_group_for_events` raises `RuntimeError` for unknown events, but the `@cachedList` wrapper swallows it and the caller just gets partial results. For the time-being, Callers MUST NOT rely on this to determine missing events, as cached responses return None for the state group instead of raising. TODO: audit every caller, as some of them may be relying on the current behaviour.

And should be copied over to _get_state_group_for_events as well

MadLittleMods · 2026-09-29T23:50:55Z

            raise SynapseError(HTTPStatus.BAD_REQUEST, "Too many prev_events")

-        if len(ev.auth_event_ids()) > 10:
+        if supports_msc4242_state_dag(ev) and len(ev.prev_state_events) > 20:


LLM is mentioning that we don't also have a check for this when creating local events which means other servers will just reject events that we create.

I see a 👍 but what's the plan here?

Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>

Co-authored-by: Eric Eastwood <erice@element.io>

- Enable MSC4242 complement tests - Fix an issue which caused cascade rejections to not be applied to events in the /send_join response becau se we didn't remember in-memory which events were rejected. - Accept rejected events in the /send_join response, it gives us a bit of robustness at no safety cost. - Recalculate room state if backfilled events caused the state DAG fwd extrems to change. - Don't filter out already seen events from event_map, so the statement "the event_map is the complete stat e DAG" still holds. - Add validation to length of prev_state_events - Don't validate auth_events length for state DAG events as it is done too early, before we've had time to calculate the auth events (and validation is moot anyway given it isn't attacker controlled anymore) - Let backfilled events update state dag fwd extrems - Add test for edge case with backfilled state events

…events that should have been rejected The cascade rejection only worked on in-flight events, not seen events. We now overwrite events in event_map with rejected events (thus setting .rejected_reason) to ensure we continue to cascade rejections correctly. Complement test asserts this.

… if one of the prev_state_events was already persisted Tested in Complement as TestMSC4242RejoinMergesOldAndNewStateDAGBranches

…vent is connected to the state DAG This PR only handles /send_join and connected events in /send, so everything else should return early with coherent error messages. The join event must be connected to the main state DAG but we previously didn't ensure this.

… scanning for other room versions

…d this

…-wide ..which would be bad.

MadLittleMods · 2026-10-02T19:12:06Z

@@ -0,0 +1 @@
+Add support for joining [MSC4242](https://github.com/matrix-org/matrix-spec-proposals/pull/4242) rooms over federation.


❌ CI failing, https://github.com/element-hq/synapse/actions/runs/37028432824/job/110910905270

[FAIL] Traceback (most recent call last): File "/home/runner/work/synapse/synapse/tests/federation/test_federation_server.py", line 519, in test_is_opt_in timeline_events = self._get_missing_events( File "/home/runner/work/synapse/synapse/tests/federation/test_federation_server.py", line 462, in _get_missing_events self.assertEqual(HTTPStatus.OK, channel.code, channel.json_body) File "/home/runner/work/synapse/synapse/tests/unittest.py", line 433, in assertEqual super().assertEqual(first=first, second=second, msg=msg) File "/home/runner/.cache/pypoetry/virtualenvs/matrix-synapse-pswDeSvb-py3.10/lib/python3.10/site-packages/twisted/trial/_synctest.py", line 433, in assertEqual super().assertEqual(first, second, msg) File "/opt/hostedtoolcache/Python/3.10.21/x64/lib/python3.10/unittest/case.py", line 845, in assertEqual assertion_func(first, second, msg=msg) File "/opt/hostedtoolcache/Python/3.10.21/x64/lib/python3.10/unittest/case.py", line 838, in _baseAssertEqual raise self.failureException(msg) twisted.trial.unittest.FailTest: <HTTPStatus.OK: 200> != 400 : {'errcode': 'M_UNRECOGNIZED', 'error': '/get_missing_events is not implemented for MSC4242 state DAG rooms'} tests.federation.test_federation_server.GetMissingEventsStateDagTests.test_is_opt_in

MadLittleMods · 2026-10-02T19:20:28Z

+                    batched_auth_events = {
+                        event_id: event_map[event_id]


Probably worth a test so we don't regress these subtleties

MadLittleMods · 2026-10-02T19:21:13Z

+                if from_send_join and all(
+                    prev_state_event_id in event_id_to_state_group
+                    for prev_state_event_id in event.prev_state_events
+                ):


Probably worth a test so we don't regress these subtleties

MadLittleMods · 2026-10-02T19:22:09Z

            raise SynapseError(HTTPStatus.BAD_REQUEST, "Too many prev_events")

-        if len(ev.auth_event_ids()) > 10:
+        if supports_msc4242_state_dag(ev) and len(ev.prev_state_events) > 20:


I see a 👍 but what's the plan here?

MadLittleMods · 2026-10-02T19:28:19Z

+                # If this assumption is ever violated, `compute_state_after_events` will
+                # yell loudly about missing state groups.


The comment should be updated to call this out more clearly as something that's probably unexpected.

XXX: `_get_state_group_for_events` raises `RuntimeError` for unknown events, but the `@cachedList` wrapper swallows it and the caller just gets partial results. For the time-being, Callers MUST NOT rely on this to determine missing events, as cached responses return None for the state group instead of raising. TODO: audit every caller, as some of them may be relying on the current behaviour.

And should be copied over to _get_state_group_for_events as well

MadLittleMods · 2026-10-02T20:46:57Z

+        if len(event.prev_state_events) == 0 and not is_create_event:
+            raise SynapseError(502, f"event {event.event_id} has no prev_state_events")


Why don't we reject the event instead of dropping it?

Related to the other spot where a rejected event in the DAG only logs "Joining anyway", on the grounds that refusing "would cost availability without buying any safety".

kegsay requested a review from a team as a code owner September 7, 2026 16:02

kegsay requested review from MadLittleMods and removed request for a team September 7, 2026 16:02

kegsay force-pushed the kegan/4242-inbound branch from f5a07a8 to 4fa82bf Compare September 8, 2026 07:38

MadLittleMods reviewed Sep 10, 2026

View reviewed changes

kegsay force-pushed the kegan/4242-inbound branch 2 times, most recently from 76ae862 to 338ed0c Compare September 11, 2026 09:13

Base automatically changed from kegan/4242-serving to develop September 11, 2026 10:44

kegsay force-pushed the kegan/4242-inbound branch from 338ed0c to 581e149 Compare September 11, 2026 10:44

kegsay requested a review from MadLittleMods September 11, 2026 14:37

kegsay mentioned this pull request Sep 11, 2026

MSC4242: State DAGs (inbound events) #20221

Open

2 of 3 tasks

MadLittleMods reviewed Sep 11, 2026

View reviewed changes

MadLittleMods reviewed Sep 14, 2026

View reviewed changes

Comment thread changelog.d/20194.misc

Comment thread synapse/handlers/federation_event.py Outdated

Comment thread synapse/handlers/federation_event.py Outdated

Comment thread synapse/handlers/federation_event.py

Comment thread synapse/handlers/federation_event.py

kegsay force-pushed the kegan/4242-inbound branch from 51b9c9d to 72a6801 Compare September 16, 2026 15:13

kegsay requested a review from MadLittleMods September 16, 2026 15:13

MadLittleMods reviewed Sep 17, 2026

View reviewed changes

kegsay requested a review from MadLittleMods September 21, 2026 13:44

kegsay force-pushed the kegan/4242-inbound branch from 8f19e17 to a835dac Compare September 21, 2026 13:46

MadLittleMods mentioned this pull request Sep 21, 2026

MSC4242: State DAGs (inbound invites) #20226

Open

1 of 3 tasks

MadLittleMods reviewed Sep 21, 2026

View reviewed changes

Comment thread synapse/storage/databases/main/rejections.py Outdated

MadLittleMods mentioned this pull request Sep 24, 2026

Add minimal /send_join MSC4242 Complement tests matrix-org/complement#926

Open

1 task done

kegsay requested a review from MadLittleMods September 28, 2026 14:56

MadLittleMods added A-State-DAGs MSC4242 A-Federated-Join A-Federation A-Join labels Sep 28, 2026

MadLittleMods reviewed Sep 29, 2026

View reviewed changes

kegsay and others added 26 commits October 2, 2026 16:38

Apply batched suggestions from code review

752f11f

Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>

Review comments

1e7cecc

Review comments

0e962a1

Linting

cc56001

Review comments

e98c653

Review comments

bf174a9

Update synapse/handlers/federation_event.py

e32d828

Co-authored-by: Eric Eastwood <erice@element.io>

Update synapse/handlers/federation_event.py

c3db7cf

Co-authored-by: Eric Eastwood <erice@element.io>

Review comments

0cbd358

Linting

ae5bff7

Update synapse/handlers/federation_event.py

40ef6c5

Co-authored-by: Eric Eastwood <erice@element.io>

Review comments

fbf97a6

Linting

f3dce9b

Stricter check to ensure missing await would be caught

dc0d0dc

Add missing RejectionsStore for workers

d73bc42

Review comments

b65f0b7

Linting

32876bb

bugfix: fix an issue where a rejoin would calculate state incorrectly…

0d1ded9

… if one of the prev_state_events was already persisted Tested in Complement as TestMSC4242RejoinMergesOldAndNewStateDAGBranches

Linting

55ad431

Ensure we sanity check events in the /send_join response

ef22e48

Only scan for rejected event IDs in state DAG rooms to avoid needless…

5cb407f

… scanning for other room versions

Assert there can be no missing auth event IDs since we already ensure…

982efb7

…d this

Swap 5xx for 4xx else MSC4242 rooms can trigger retry backoffs server…

82d52c9

…-wide ..which would be bad.

kegsay force-pushed the kegan/4242-inbound branch from 1ef48a8 to 82d52c9 Compare October 2, 2026 15:38

MadLittleMods reviewed Oct 2, 2026

View reviewed changes

MadLittleMods mentioned this pull request Oct 2, 2026

MSC4242: State DAGs matrix-org/matrix-spec-proposals#4242

Open

MadLittleMods reviewed Oct 2, 2026

View reviewed changes

		# If this assumption is ever violated, `compute_state_after_events` will
		# yell loudly about missing state groups.

		@@ -0,0 +1 @@
		Add support for joining [MSC4242](https://github.com/matrix-org/matrix-spec-proposals/pull/4242) rooms over federation.

		if len(event.prev_state_events) == 0 and not is_create_event:
		raise SynapseError(502, f"event {event.event_id} has no prev_state_events")

Conversation

kegsay commented Sep 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MadLittleMods Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

kegsay commented Sep 7, 2026 •

edited

Loading

MadLittleMods Sep 29, 2026 •

edited

Loading