Skip to content

Commit a3cc5b9

Browse files
committed
store: Go back to the less clever but correct way of pruning
The change in aea5b57 has a logic error in that we look at a fixed number of rows from the table, filter out the ones we don't need to copy, and then insert the remaining rows. The error is that the logic assumes that if we do not insert any rows that we are done with copying, but that's wrong, since not inserting rows now just means that no rows in the batch needed to be copied even though a subsequent batch might still have such rows. This reverts commit aea5b57.
1 parent aea5b57 commit a3cc5b9

1 file changed

Lines changed: 2 additions & 21 deletions

File tree

store/postgres/src/relational/prune.rs

Lines changed: 2 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -91,22 +91,9 @@ impl TablePair {
9191
loop {
9292
let start = Instant::now();
9393
let LastVid { last_vid, rows } = conn.transaction(|| {
94-
// The query limits the number of rows we look at in each
95-
// batch, rather than the number of matching rows to make
96-
// the timing for each batch independent of how many rows we
97-
// actually need to copy. This has the downside that it will
98-
// force a sequential scan, but limiting by the number of
99-
// matching rows runs the risk of a batch taking
100-
// unpredictably long if there are only very few matches.
101-
// Strictly speaking, the `vid >= $3` and `limit` clauses on
102-
// the outer query are redundant.
10394
sql_query(&format!(
10495
"with cp as (insert into {dst}({column_list}) \
105-
select {column_list} from ( \
106-
select {column_list} from {src} \
107-
where vid >= $3 \
108-
order by vid \
109-
limit $4) a \
96+
select {column_list} from {src} \
11097
where lower(block_range) <= $2 \
11198
and coalesce(upper(block_range), 2147483647) > $1 \
11299
and coalesce(upper(block_range), 2147483647) <= $2 \
@@ -166,15 +153,9 @@ impl TablePair {
166153
loop {
167154
let start = Instant::now();
168155
let LastVid { rows, last_vid } = conn.transaction(|| {
169-
// See the comment on the query in `copy_final_entities` for
170-
// why this query is written this way
171156
sql_query(&format!(
172157
"with cp as (insert into {dst}({column_list}) \
173-
select {column_list} from ( \
174-
select {column_list} from {src} \
175-
where vid >= $2 \
176-
order by vid \
177-
limit $3) a
158+
select {column_list} from {src} \
178159
where coalesce(upper(block_range), 2147483647) > $1 \
179160
and block_range && int4range($1, null) \
180161
and vid >= $2 \

0 commit comments

Comments
 (0)