MSC4311: Use full PDU's in stripped state (like invite_room_state) over federation and always include m.room.create event

[MadLittleMods]

Background

This PR was originally just trying to remove the flawed MSC4311 partial implementation as client side API's like /sync should still use stripped events. But it turns out we were just re-using the client logic for the federation side and things might break if we didn't include the full m.room.create event so this PR now introduces MSC4311 support to use full PDU's in the invite_room_state/knock_room_state in the federation API's.

The flawed implementation was originally introduced in 0eb7252 (no PR I assume because part of Hydra security fix)

Spawning from reviewing #19722 and noticing that we have TestMSC4311FullCreateEventOnStrippedState in Complement which already passes even though that test looks flawed:

I think this test is mixing up what MSC4311 proposes. Perhaps these were changes to the MSC that came after?

For the client API's like /sync, it only proposes that m.room.create is a required stripped state event.

For the federation API's, alongside requiring m.room.create, it also mandates using the full event PDU format for all events in the invite_room_state/knock_room_state on m.room.member events (in unsigned)

What does this PR do?

1. Always use stripped state for client API's
   1. Remove flawed MSC4311 partial implementation (as explained above)
   2. Sanitize stripped state when we receive events over federation
2. Use full PDU's when sending invite_room_state/knock_room_state over federation
3. Validate PDU's and warn when receiving invite_room_state/knock_room_state over federation

Complement tests: matrix-org/complement#796

---

Part of #19414

Dev notes

COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestMSC4311FullEventsOnStrippedStateFederation

COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestMSC4311StrippedStateClientApi

Todo

• Fix/remove TestMSC4311FullCreateEventOnStrippedState Complement test so it doesn't look for the full PDU in /sync.
  • It looks like Try to modify MSC4311 test to meet new proposal matrix-org/complement#796 tries to address this
• Figure out whether we care about addressing knock_room_state in this PR
• Validate knock_room_state

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file. The entry should:
  • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
  • Use markdown where necessary, mostly for code blocks.
  • End with either a period (.) or an exclamation mark (!).
  • Start with a capital letter.
  • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
• Code style is correct (run the linters)

[MadLittleMods]

Removing the flawed MSC4311 partial implementation

[MadLittleMods]

Split some logic out of get_stripped_room_state_from_event_context(...) into get_stripped_room_state_ids_from_event_context(...)

This way we can use the same event selection logic but fetch/format them however we see fit.

[MadLittleMods]

Removed as we actually do this now in FederationServer.on_invite_request(...)