Commit 99871b3
committed
chore(clickhouse): bring the AI_EVENTS satellite under declarative HCL
Model the AI_EVENTS role in posthog/clickhouse/hcl/ (the third managed role
after OPS and LOGS) and wire it into both the offline guard and the live gate.
The ai_events column set is identical across every table. Factor it into two
abstract bases in roles/ai_events/shared and extend them:
- _ai_events_columns (columns only) -> the distributed ai_events reader (local);
- _ai_events_data (extends _ai_events_columns, adds order_by/partition/ttl +
indexes) -> the data tables sharded_ai_events (local) and ai_events (US/EU),
which then only set their own engine (zoo_path differs).
Env layering follows the actual topology:
- local/hobby run the MSK ingest variant (kafka_ai_events_json + ai_events_json_mv)
with sharded_ai_events + a distributed ai_events reader;
- US/EU run WarpStream (kafka_ai_events_json_ws + ai_events_json_ws_mv, MSK dropped
by migration 0248) writing into a single ai_events data table (US and EU
identical; the earlier per-EU sharded_ai_events has been dropped in prod);
- person / person_distinct_id2 are env-uniform Distributed shims (0240), also in
roles/ai_events/shared; dev has only the top-level shared objects.
- nodes: local/dev/prod-us/prod-eu ai_events lines.
- dump-live.sh / check-live.sh: add ai_events (:9100) so the convergence gate
introspects and diffs it too.
- check.sh: skip validation of the person / cross-cluster Distributed proxies.
Goldens are generated from the layers (abstract bases resolve away, so the flat
schema is unchanged). Verified against a live multinode ai_events node: dump-live
+ check-live report no drift for local/ai_events; prod goldens are authored from
the captured prod dumps.1 parent ceafe76 commit 99871b3
15 files changed
Lines changed: 7520 additions & 6 deletions
File tree
- posthog/clickhouse/hcl
- golden
- roles/ai_events
- local
- prod
- sql
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
| 29 | + | |
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| |||
86 | 86 | | |
87 | 87 | | |
88 | 88 | | |
89 | | - | |
| 89 | + | |
90 | 90 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
| 29 | + | |
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
38 | | - | |
39 | | - | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
40 | 41 | | |
41 | 42 | | |
42 | 43 | | |
| |||
0 commit comments