Skip to content

Commit 4969f96

Browse files
actions-useractions-user
committed
Update schema
1 parent 44f88cc commit 4969f96

21 files changed

Lines changed: 599 additions & 2 deletions

‎llms/clickhouse/llms-full.txt‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,18 @@ SELECT * FROM sepolia.table_name LIMIT 10;
239239
- **Partitioning**: slot_start_date_time (datetime), daily
240240
- **Networks**: mainnet (`mainnet.fct_attestation_first_seen_chunked_50ms`), sepolia (`sepolia.fct_attestation_first_seen_chunked_50ms`), holesky (`holesky.fct_attestation_first_seen_chunked_50ms`), hoodi (`hoodi.fct_attestation_first_seen_chunked_50ms`)
241241
- **Tags**:
242+
### fct_attestation_liveness_by_entity_head
243+
- **Database**: Network-specific (see Networks below)
244+
- **Description**: Attestation liveness aggregated by entity for the head chain. One or two rows per (slot, entity): one for attested, one for missed.
245+
- **Partitioning**: slot_start_date_time (datetime), daily
246+
- **Networks**: mainnet (`mainnet.fct_attestation_liveness_by_entity_head`), sepolia (`sepolia.fct_attestation_liveness_by_entity_head`), holesky (`holesky.fct_attestation_liveness_by_entity_head`), hoodi (`hoodi.fct_attestation_liveness_by_entity_head`)
247+
- **Tags**:
248+
### fct_attestation_observation_by_node
249+
- **Database**: Network-specific (see Networks below)
250+
- **Description**: Attestation observations by contributor nodes, aggregated per slot per node for performance
251+
- **Partitioning**: slot_start_date_time (datetime), daily
252+
- **Networks**: mainnet (`mainnet.fct_attestation_observation_by_node`), sepolia (`sepolia.fct_attestation_observation_by_node`), holesky (`holesky.fct_attestation_observation_by_node`), hoodi (`hoodi.fct_attestation_observation_by_node`)
253+
- **Tags**:
242254
### fct_block
243255
- **Database**: Network-specific (see Networks below)
244256
- **Description**: Block details for the finalized chain including orphaned blocks

‎llms/clickhouse/llms.txt‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,18 @@ SELECT * FROM sepolia.table_name LIMIT 10;
239239
- **Partitioning**: slot_start_date_time (datetime), daily
240240
- **Networks**: mainnet (`mainnet.fct_attestation_first_seen_chunked_50ms`), sepolia (`sepolia.fct_attestation_first_seen_chunked_50ms`), holesky (`holesky.fct_attestation_first_seen_chunked_50ms`), hoodi (`hoodi.fct_attestation_first_seen_chunked_50ms`)
241241
- **Tags**:
242+
### fct_attestation_liveness_by_entity_head
243+
- **Database**: Network-specific (see Networks below)
244+
- **Description**: Attestation liveness aggregated by entity for the head chain. One or two rows per (slot, entity): one for attested, one for missed.
245+
- **Partitioning**: slot_start_date_time (datetime), daily
246+
- **Networks**: mainnet (`mainnet.fct_attestation_liveness_by_entity_head`), sepolia (`sepolia.fct_attestation_liveness_by_entity_head`), holesky (`holesky.fct_attestation_liveness_by_entity_head`), hoodi (`hoodi.fct_attestation_liveness_by_entity_head`)
247+
- **Tags**:
248+
### fct_attestation_observation_by_node
249+
- **Database**: Network-specific (see Networks below)
250+
- **Description**: Attestation observations by contributor nodes, aggregated per slot per node for performance
251+
- **Partitioning**: slot_start_date_time (datetime), daily
252+
- **Networks**: mainnet (`mainnet.fct_attestation_observation_by_node`), sepolia (`sepolia.fct_attestation_observation_by_node`), holesky (`holesky.fct_attestation_observation_by_node`), hoodi (`hoodi.fct_attestation_observation_by_node`)
253+
- **Tags**:
242254
### fct_block
243255
- **Database**: Network-specific (see Networks below)
244256
- **Description**: Block details for the finalized chain including orphaned blocks

‎schema/beacon_api_.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -135,7 +135,7 @@ Data is partitioned **hourly** on **slot_start_date_time** for the following net
135135

136136
- **mainnet**: `2023-06-01` to `2025-10-25`
137137
- **holesky**: `2023-09-18` to `2025-10-25`
138-
- **sepolia**: `2023-08-31` to `2025-10-24`
138+
- **sepolia**: `2023-08-31` to `2025-10-25`
139139

140140
### Examples
141141

‎schema/canonical_beacon_.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1301,7 +1301,7 @@ Contains a validator state for an epoch.
13011301
Data is partitioned **hourly** on **epoch_start_date_time** for the following networks:
13021302

13031303
- **mainnet**: `2020-12-01` to `2025-10-25`
1304-
- **holesky**: `2023-09-23` to `2025-10-24`
1304+
- **holesky**: `2023-09-23` to `2025-10-25`
13051305
- **sepolia**: `2022-06-20` to `2025-10-20`
13061306

13071307
### Examples

‎schema/cbt.md‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@ CBT tables include dimension tables (prefixed with `dim_`), fact tables (prefixe
3030
- [`fct_attestation_correctness_canonical`](#fct_attestation_correctness_canonical)
3131
- [`fct_attestation_correctness_head`](#fct_attestation_correctness_head)
3232
- [`fct_attestation_first_seen_chunked_50ms`](#fct_attestation_first_seen_chunked_50ms)
33+
- [`fct_attestation_liveness_by_entity_head`](#fct_attestation_liveness_by_entity_head)
34+
- [`fct_attestation_observation_by_node`](#fct_attestation_observation_by_node)
3335
- [`fct_block`](#fct_block)
3436
- [`fct_block_blob_count`](#fct_block_blob_count)
3537
- [`fct_block_blob_count_head`](#fct_block_blob_count_head)
@@ -762,6 +764,145 @@ echo """
762764
| **chunk_slot_start_diff** | `UInt32` | *The different between the chunk start time and slot_start_date_time. "1500" would mean this chunk contains attestations first seen between 1500ms 1550ms into the slot* |
763765
| **attestation_count** | `UInt32` | *The number of attestations in this chunk* |
764766

767+
## fct_attestation_liveness_by_entity_head
768+
769+
Attestation liveness aggregated by entity for the head chain. One or two rows per (slot, entity): one for attested, one for missed.
770+
771+
### Availability
772+
Data is partitioned by **toStartOfMonth(slot_start_date_time)**.
773+
774+
Available in the following network-specific databases:
775+
776+
- **mainnet**: `mainnet.fct_attestation_liveness_by_entity_head`
777+
- **sepolia**: `sepolia.fct_attestation_liveness_by_entity_head`
778+
- **holesky**: `holesky.fct_attestation_liveness_by_entity_head`
779+
- **hoodi**: `hoodi.fct_attestation_liveness_by_entity_head`
780+
781+
### Examples
782+
783+
<details>
784+
<summary>Your Clickhouse</summary>
785+
786+
> **Note:** [`FINAL`](https://clickhouse.com/docs/en/sql-reference/statements/select/from#final-modifier) should be used when querying this table
787+
788+
```bash
789+
docker run --rm -it --net host clickhouse/clickhouse-server clickhouse client --query="""
790+
SELECT
791+
*
792+
FROM mainnet.fct_attestation_liveness_by_entity_head FINAL
793+
LIMIT 10
794+
FORMAT Pretty
795+
"""
796+
```
797+
</details>
798+
799+
<details>
800+
<summary>EthPandaOps Clickhouse</summary>
801+
802+
> **Note:** [`FINAL`](https://clickhouse.com/docs/en/sql-reference/statements/select/from#final-modifier) should be used when querying this table
803+
804+
```bash
805+
echo """
806+
SELECT
807+
*
808+
FROM mainnet.fct_attestation_liveness_by_entity_head FINAL
809+
LIMIT 3
810+
FORMAT Pretty
811+
""" | curl "https://clickhouse.xatu.ethpandaops.io" -u "$CLICKHOUSE_USER:$CLICKHOUSE_PASSWORD" --data-binary @-
812+
```
813+
</details>
814+
815+
### Columns
816+
| Name | Type | Description |
817+
|--------|------|-------------|
818+
| **updated_date_time** | `DateTime` | *Timestamp when the record was last updated* |
819+
| **slot** | `UInt32` | *The slot number* |
820+
| **slot_start_date_time** | `DateTime` | *The wall clock time when the slot started* |
821+
| **epoch** | `UInt32` | *The epoch number containing the slot* |
822+
| **epoch_start_date_time** | `DateTime` | *The wall clock time when the epoch started* |
823+
| **entity** | `String` | *The entity (staking provider) associated with the validators, unknown if not mapped* |
824+
| **status** | `String` | *Attestation status: attested or missed* |
825+
| **attestation_count** | `UInt32` | *Number of attestations for this entity/status combination* |
826+
827+
## fct_attestation_observation_by_node
828+
829+
Attestation observations by contributor nodes, aggregated per slot per node for performance
830+
831+
### Availability
832+
Data is partitioned by **toStartOfMonth(slot_start_date_time)**.
833+
834+
Available in the following network-specific databases:
835+
836+
- **mainnet**: `mainnet.fct_attestation_observation_by_node`
837+
- **sepolia**: `sepolia.fct_attestation_observation_by_node`
838+
- **holesky**: `holesky.fct_attestation_observation_by_node`
839+
- **hoodi**: `hoodi.fct_attestation_observation_by_node`
840+
841+
### Examples
842+
843+
<details>
844+
<summary>Your Clickhouse</summary>
845+
846+
> **Note:** [`FINAL`](https://clickhouse.com/docs/en/sql-reference/statements/select/from#final-modifier) should be used when querying this table
847+
848+
```bash
849+
docker run --rm -it --net host clickhouse/clickhouse-server clickhouse client --query="""
850+
SELECT
851+
*
852+
FROM mainnet.fct_attestation_observation_by_node FINAL
853+
LIMIT 10
854+
FORMAT Pretty
855+
"""
856+
```
857+
</details>
858+
859+
<details>
860+
<summary>EthPandaOps Clickhouse</summary>
861+
862+
> **Note:** [`FINAL`](https://clickhouse.com/docs/en/sql-reference/statements/select/from#final-modifier) should be used when querying this table
863+
864+
```bash
865+
echo """
866+
SELECT
867+
*
868+
FROM mainnet.fct_attestation_observation_by_node FINAL
869+
LIMIT 3
870+
FORMAT Pretty
871+
""" | curl "https://clickhouse.xatu.ethpandaops.io" -u "$CLICKHOUSE_USER:$CLICKHOUSE_PASSWORD" --data-binary @-
872+
```
873+
</details>
874+
875+
### Columns
876+
| Name | Type | Description |
877+
|--------|------|-------------|
878+
| **updated_date_time** | `DateTime` | *Timestamp when the record was last updated* |
879+
| **slot** | `UInt32` | *The slot number* |
880+
| **slot_start_date_time** | `DateTime` | *The wall clock time when the slot started* |
881+
| **epoch** | `UInt32` | *The epoch number containing the slot* |
882+
| **epoch_start_date_time** | `DateTime` | *The wall clock time when the epoch started* |
883+
| **attestation_count** | `UInt32` | *Number of attestations observed by this node in this slot* |
884+
| **avg_seen_slot_start_diff** | `UInt32` | *Average time from slot start to see attestations (milliseconds, rounded)* |
885+
| **median_seen_slot_start_diff** | `UInt32` | *Median time from slot start to see attestations (milliseconds, rounded)* |
886+
| **min_seen_slot_start_diff** | `UInt32` | *Minimum time from slot start to see an attestation (milliseconds)* |
887+
| **max_seen_slot_start_diff** | `UInt32` | *Maximum time from slot start to see an attestation (milliseconds)* |
888+
| **block_root** | `String` | *Representative beacon block root (from most common attestation target)* |
889+
| **username** | `LowCardinality(String)` | *Username of the node* |
890+
| **node_id** | `String` | *ID of the node* |
891+
| **classification** | `LowCardinality(String)` | *Classification of the node, e.g. "individual", "corporate", "internal" (aka ethPandaOps) or "unclassified"* |
892+
| **meta_client_name** | `LowCardinality(String)` | *Name of the client* |
893+
| **meta_client_version** | `LowCardinality(String)` | *Version of the client* |
894+
| **meta_client_implementation** | `LowCardinality(String)` | *Implementation of the client* |
895+
| **meta_client_geo_city** | `LowCardinality(String)` | *City of the client* |
896+
| **meta_client_geo_country** | `LowCardinality(String)` | *Country of the client* |
897+
| **meta_client_geo_country_code** | `LowCardinality(String)` | *Country code of the client* |
898+
| **meta_client_geo_continent_code** | `LowCardinality(String)` | *Continent code of the client* |
899+
| **meta_client_geo_longitude** | `Nullable(Float64)` | *Longitude of the client* |
900+
| **meta_client_geo_latitude** | `Nullable(Float64)` | *Latitude of the client* |
901+
| **meta_client_geo_autonomous_system_number** | `Nullable(UInt32)` | *Autonomous system number of the client* |
902+
| **meta_client_geo_autonomous_system_organization** | `Nullable(String)` | *Autonomous system organization of the client* |
903+
| **meta_consensus_version** | `LowCardinality(String)` | *Ethereum consensus client version* |
904+
| **meta_consensus_implementation** | `LowCardinality(String)` | *Ethereum consensus client implementation* |
905+
765906
## fct_block
766907

767908
Block details for the finalized chain including orphaned blocks

‎schema/clickhouse/holesky/fct_attestation_liveness_by_entity_head.sql‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
CREATE TABLE holesky.fct_attestation_liveness_by_entity_head
2+
(
3+
`updated_date_time` DateTime COMMENT 'Timestamp when the record was last updated' CODEC(DoubleDelta, ZSTD(1)),
4+
`slot` UInt32 COMMENT 'The slot number' CODEC(DoubleDelta, ZSTD(1)),
5+
`slot_start_date_time` DateTime COMMENT 'The wall clock time when the slot started' CODEC(DoubleDelta, ZSTD(1)),
6+
`epoch` UInt32 COMMENT 'The epoch number containing the slot' CODEC(DoubleDelta, ZSTD(1)),
7+
`epoch_start_date_time` DateTime COMMENT 'The wall clock time when the epoch started' CODEC(DoubleDelta, ZSTD(1)),
8+
`entity` String COMMENT 'The entity (staking provider) associated with the validators, unknown if not mapped' CODEC(ZSTD(1)),
9+
`status` String COMMENT 'Attestation status: attested or missed' CODEC(ZSTD(1)),
10+
`attestation_count` UInt32 COMMENT 'Number of attestations for this entity/status combination' CODEC(ZSTD(1))
11+
)
12+
ENGINE = Distributed('{cluster}', 'holesky', 'fct_attestation_liveness_by_entity_head_local', cityHash64(slot))
13+
COMMENT 'Attestation liveness aggregated by entity for the head chain. One or two rows per (slot, entity): one for attested, one for missed.'

‎schema/clickhouse/holesky/fct_attestation_liveness_by_entity_head_local.sql‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
CREATE TABLE holesky.fct_attestation_liveness_by_entity_head_local
2+
(
3+
`updated_date_time` DateTime COMMENT 'Timestamp when the record was last updated' CODEC(DoubleDelta, ZSTD(1)),
4+
`slot` UInt32 COMMENT 'The slot number' CODEC(DoubleDelta, ZSTD(1)),
5+
`slot_start_date_time` DateTime COMMENT 'The wall clock time when the slot started' CODEC(DoubleDelta, ZSTD(1)),
6+
`epoch` UInt32 COMMENT 'The epoch number containing the slot' CODEC(DoubleDelta, ZSTD(1)),
7+
`epoch_start_date_time` DateTime COMMENT 'The wall clock time when the epoch started' CODEC(DoubleDelta, ZSTD(1)),
8+
`entity` String COMMENT 'The entity (staking provider) associated with the validators, unknown if not mapped' CODEC(ZSTD(1)),
9+
`status` String COMMENT 'Attestation status: attested or missed' CODEC(ZSTD(1)),
10+
`attestation_count` UInt32 COMMENT 'Number of attestations for this entity/status combination' CODEC(ZSTD(1)),
11+
PROJECTION p_by_slot
12+
(
13+
SELECT *
14+
ORDER BY slot
15+
)
16+
)
17+
ENGINE = ReplicatedReplacingMergeTree('/clickhouse/{installation}/{cluster}/tables/{shard}/mainnet/fct_attestation_liveness_by_entity_head_local', '{replica}', updated_date_time)
18+
PARTITION BY toStartOfMonth(slot_start_date_time)
19+
ORDER BY (slot_start_date_time, entity, status)
20+
SETTINGS deduplicate_merge_projection_mode = 'rebuild', index_granularity = 8192
21+
COMMENT 'Attestation liveness aggregated by entity for the head chain. One or two rows per (slot, entity): one for attested, one for missed.'

‎schema/clickhouse/holesky/fct_attestation_observation_by_node.sql‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
CREATE TABLE holesky.fct_attestation_observation_by_node
2+
(
3+
`updated_date_time` DateTime COMMENT 'Timestamp when the record was last updated' CODEC(DoubleDelta, ZSTD(1)),
4+
`slot` UInt32 COMMENT 'The slot number' CODEC(DoubleDelta, ZSTD(1)),
5+
`slot_start_date_time` DateTime COMMENT 'The wall clock time when the slot started' CODEC(DoubleDelta, ZSTD(1)),
6+
`epoch` UInt32 COMMENT 'The epoch number containing the slot' CODEC(DoubleDelta, ZSTD(1)),
7+
`epoch_start_date_time` DateTime COMMENT 'The wall clock time when the epoch started' CODEC(DoubleDelta, ZSTD(1)),
8+
`attestation_count` UInt32 COMMENT 'Number of attestations observed by this node in this slot' CODEC(DoubleDelta, ZSTD(1)),
9+
`avg_seen_slot_start_diff` UInt32 COMMENT 'Average time from slot start to see attestations (milliseconds, rounded)' CODEC(DoubleDelta, ZSTD(1)),
10+
`median_seen_slot_start_diff` UInt32 COMMENT 'Median time from slot start to see attestations (milliseconds, rounded)' CODEC(DoubleDelta, ZSTD(1)),
11+
`min_seen_slot_start_diff` UInt32 COMMENT 'Minimum time from slot start to see an attestation (milliseconds)' CODEC(DoubleDelta, ZSTD(1)),
12+
`max_seen_slot_start_diff` UInt32 COMMENT 'Maximum time from slot start to see an attestation (milliseconds)' CODEC(DoubleDelta, ZSTD(1)),
13+
`block_root` String COMMENT 'Representative beacon block root (from most common attestation target)' CODEC(ZSTD(1)),
14+
`username` LowCardinality(String) COMMENT 'Username of the node' CODEC(ZSTD(1)),
15+
`node_id` String COMMENT 'ID of the node' CODEC(ZSTD(1)),
16+
`classification` LowCardinality(String) COMMENT 'Classification of the node, e.g. "individual", "corporate", "internal" (aka ethPandaOps) or "unclassified"' CODEC(ZSTD(1)),
17+
`meta_client_name` LowCardinality(String) COMMENT 'Name of the client',
18+
`meta_client_version` LowCardinality(String) COMMENT 'Version of the client',
19+
`meta_client_implementation` LowCardinality(String) COMMENT 'Implementation of the client',
20+
`meta_client_geo_city` LowCardinality(String) COMMENT 'City of the client' CODEC(ZSTD(1)),
21+
`meta_client_geo_country` LowCardinality(String) COMMENT 'Country of the client' CODEC(ZSTD(1)),
22+
`meta_client_geo_country_code` LowCardinality(String) COMMENT 'Country code of the client' CODEC(ZSTD(1)),
23+
`meta_client_geo_continent_code` LowCardinality(String) COMMENT 'Continent code of the client' CODEC(ZSTD(1)),
24+
`meta_client_geo_longitude` Nullable(Float64) COMMENT 'Longitude of the client' CODEC(ZSTD(1)),
25+
`meta_client_geo_latitude` Nullable(Float64) COMMENT 'Latitude of the client' CODEC(ZSTD(1)),
26+
`meta_client_geo_autonomous_system_number` Nullable(UInt32) COMMENT 'Autonomous system number of the client' CODEC(ZSTD(1)),
27+
`meta_client_geo_autonomous_system_organization` Nullable(String) COMMENT 'Autonomous system organization of the client' CODEC(ZSTD(1)),
28+
`meta_consensus_version` LowCardinality(String) COMMENT 'Ethereum consensus client version',
29+
`meta_consensus_implementation` LowCardinality(String) COMMENT 'Ethereum consensus client implementation'
30+
)
31+
ENGINE = Distributed('{cluster}', 'holesky', 'fct_attestation_observation_by_node_local', cityHash64(slot_start_date_time, meta_client_name))
32+
COMMENT 'Attestation observations by contributor nodes, aggregated per slot per node for performance'

0 commit comments

Comments
 (0)