Skip to content

Commit 099774a

Browse files
committed
fix: limit pg_stat_statements to top 50 queries by exec time
Instead of removing pg_stat_statements from the Prometheus sink entirely, keep the collection but cap cardinality at 50 queryids (ranked by total exec time). This gives 50 × 16 = 800 series max instead of 30K+. The previous weak filter (calls >= 5 OR exec_time >= 1000) let through thousands of queries on busy databases. ORDER BY + LIMIT 50 is a hard cap. https://claude.ai/code/session_01SzJxzZNQjDQphaHyaX3RU7
1 parent 820d253 commit 099774a

2 files changed

Lines changed: 7 additions & 7 deletions

File tree

config/pgwatch-prometheus/metrics.yml

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1014,6 +1014,8 @@ metrics:
10141014
join pg_database on pg_database.oid = pg_stat_statements.dbid
10151015
where pg_database.datname = current_database()
10161016
group by pg_database.datname, pg_stat_statements.queryid
1017+
order by sum(pg_stat_statements.total_exec_time) desc
1018+
limit 50
10171019
)
10181020
select
10191021
datname as tag_datname,
@@ -1035,7 +1037,6 @@ metrics:
10351037
temp_bytes_read::int8 as temp_bytes_read,
10361038
temp_bytes_written::int8 as temp_bytes_written
10371039
from aggregated_statements
1038-
where calls >= 5 or exec_time_total >= 1000
10391040
17: |
10401041
with aggregated_statements as ( /* pgwatch_generated */
10411042
select
@@ -1061,6 +1062,8 @@ metrics:
10611062
join pg_database on pg_database.oid = pg_stat_statements.dbid
10621063
where pg_database.datname = current_database()
10631064
group by pg_database.datname, pg_stat_statements.queryid
1065+
order by sum(pg_stat_statements.total_exec_time) desc
1066+
limit 50
10641067
)
10651068
select
10661069
datname as tag_datname,
@@ -1082,7 +1085,6 @@ metrics:
10821085
temp_bytes_read::int8 as temp_bytes_read,
10831086
temp_bytes_written::int8 as temp_bytes_written
10841087
from aggregated_statements
1085-
where calls >= 5 or exec_time_total >= 1000
10861088
gauges:
10871089
- calls
10881090
- plans_total
@@ -2532,8 +2534,7 @@ presets:
25322534
pg_stat_all_tables: 30
25332535
pg_class: 30
25342536
pg_stat_all_indexes: 30
2535-
# pg_stat_statements removed: handled by flask-pgss-api via Postgres sink
2536-
# Collecting per-queryid metrics here caused 30K+ series cardinality explosion
2537+
pg_stat_statements: 30
25372538
pg_stat_replication: 30
25382539
pg_stat_slru: 30
25392540
pg_statio_all_tables: 30

docker-compose.yml

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,8 @@
22
# Total allocation: ~5.1 vCPUs, ~7.6 GiB RAM (with headroom for host OS).
33
# CPU overcommitment is intentional - containers rarely hit limits simultaneously.
44
#
5-
# pgwatch-prometheus gets higher CPU (1.5) to handle metric collection load.
6-
# Note: pg_stat_statements is NOT collected here (handled by flask-pgss-api
7-
# via the Postgres sink) to avoid cardinality explosion in VictoriaMetrics.
5+
# pgwatch-prometheus collects metrics including pg_stat_statements (top 50 queries
6+
# by exec time to control cardinality in VictoriaMetrics).
87

98
services:
109
# Config Initializer - Copies static configs from image to volume

0 commit comments

Comments
 (0)