[catalog] convert mz_audit_events to BuiltinMaterializedView#37281
Draft
mtabebe wants to merge 2 commits into
Draft
[catalog] convert mz_audit_events to BuiltinMaterializedView#37281mtabebe wants to merge 2 commits into
mtabebe wants to merge 2 commits into
Conversation
mtabebe
commented
Jun 24, 2026
Contributor
Problem: mz_internal.mz_comments is a BuiltinTable. This requires coordinated writes from a single environmentd process, conflicting with the multi-envd goal. Solution: Convert mz_comments to a BuiltinMaterializedView backed by a query over mz_internal.mz_catalog_raw. Each Comment row in the durable catalog has a `CommentObject` key whose variant is the object type and whose inner value carries the id. Testing: - test/sqllogictest/mz_comments.slt lockdown test - test/sqllogictest/comment.slt, oid.slt, information_schema_tables.slt, replacement-materialized-views.slt, mz_catalog_server_index_accounting.slt, catalog_server_explain.slt all pass.
becf59e to
4b62e21
Compare
Problem:
mz_catalog.mz_audit_events is a BuiltinTable. This requires coordinated
writes from a single environmentd process, conflicting with the
multi-envd goal. The table also carried a special-case bootstrap path
(`bootstrap_audit_log_table` in `coord.rs`) that diffed the in-memory
`AuditLogIterator` against the table contents on every boot.
Solution:
Convert mz_audit_events to a BuiltinMaterializedView backed by a query
over mz_internal.mz_catalog_raw, and delete the bootstrap path.
Each AuditLog row in the durable catalog has a `proto::AuditLogKey`
whose `event_type` and `object_type` are numeric `Serialize_repr` codes
that the MV maps back to the kebab-case strings produced by the prior
populator (`audit_log::{EventType,ObjectType}` Display). `details` goes
through a new SQL helper:
mz_internal.parse_catalog_audit_log_details(jsonb) -> jsonb
Breaking change:
The `mz_audit_events.details` JSON shape changes for variants where the
proto and audit-log structs diverge beyond `#[serde(flatten)]` /
`StringWrapper`.
queries:
Testing:
- New `src/expr/src/scalar/func/impls/jsonb.rs#tests` Rust unit tests
cover variant strip, flatten, recursive flatten, StringWrapper
unwrap, null option, non-FullNameV1 negative cases, and error inputs.
- test/sqllogictest/mz_audit_events.slt new lockdown test (schema,
OID stability, item type, NOT-NULL invariants, monotonicity, every
reachable (event_type, object_type) pair, representative details
extractions, NULL user, deterministic occurred_at, PUBLIC_SELECT).
- test/sqllogictest/singlereplica_audit_log.slt updated for new shape.
- test/sqllogictest/{information_schema_tables,oid,materialized_views,
cluster,catalog_server_explain,mz_catalog_server_index_accounting,
rbac_mcp_agent,rename,transform/predicate_pushdown,
autogenerated/mz_catalog}.slt all pass.
- test/testdrive/{catalog,audit-log}.td updated for new shape.
4b62e21 to
e66bff2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.