datastore: add support for truncation#3215
Merged
Merged
Conversation
Shubham8287
reviewed
Sep 2, 2025
gefjon
reviewed
Sep 2, 2025
08b70cf to
813f34c
Compare
cc9ef87 to
a81d360
Compare
a81d360 to
2ed26ea
Compare
Shubham8287
reviewed
Sep 16, 2025
Shubham8287
requested changes
Sep 16, 2025
Contributor
Shubham8287
left a comment
There was a problem hiding this comment.
If you need to reproduce the bug mentioned in comment, I can help but reasoning seems straight-forward.
Contributor
|
I am also getting this error while trying to replay from commitlog after module update. With following repo steps:
Not reproducible in master, I also tried with rebasing the PR on master. |
2ed26ea to
f44dcfa
Compare
Merged
# Description of Changes Based on,and to fix mentioned issues on #3215 i.e - 1. Mark table for truncation after anaylyzing `TxData::inserts` too. 2. `visit_truncates` can not call `schema_for_tables` for dropped tables as schema info has been deleted from `st_*` tables. Hence, added a field in `ReplayVisitor` to cache table names. This PR decouples truncation info from delete entries, since deletion alone can’t reliably tell about truncation without extra mutations later. Can be reviewed commit wise. # Testing Testing using "Add columns" PR.
Shubham8287
approved these changes
Sep 23, 2025
Contributor
Shubham8287
left a comment
There was a problem hiding this comment.
Merged https://github.com/clockworklabs/SpacetimeDB/pull/3261/files which fixes the previously mentiond issues.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Changes
Adds support for truncation in the datastore.
The commitlog format already supported it.
A table is considered truncated in a transaction when there are deletes and when the row count post-deletes is 0.
API and ABI breaking changes
None
Expected complexity level and risk
3?
Testing
Some existing datastore tests have been amended to check whether
TxDatacontain truncation or not.