Skip to content

"super db manage" is not compacting #6840

@philrz

Description

@philrz

It's noted in #6821 that some tests related to super db manage were disabled because their failure was correlated with those changes and thought likely due to the introduction of the "any" type. This issue is just to document the user-facing impact of this, which is that compaction via super db manage is not working at the moment.

Details

Repro is with super commit af84fb5, which is associated with the merge of #6821.

Pasting the following commands preps for the repro by creating three commits and deleting the data associated with one of them.

rm -rf store
super db -db store init
super db -db store create foo
super db -db store use foo
echo '{ts:1,msg:"foo"}' | super db -db store load -
echo '{ts:2,msg:"bar"}' | super db -db store load -
echo '{ts:3,msg:"baz"}' | super db -db store load -
super db -db store delete -where 'ts==2'

When do this prep and then execute the manage command, we see "objects_compacted":0 and the data/ directory still contains just the objects associated with each individual commit.

$ super -version
Version: v0.3.0-49-gaf84fb5d1

$ super db -db store manage
{"level":"info","ts":1776735693.032498,"logger":"pool","msg":"updating pool","name":"foo","id":"3CeAU7C6izLxbu6X2pTXxyTKFZ6","branch":"main","vectors":false}
{"level":"info","ts":1776735693.044023,"logger":"pool","msg":"compaction completed","name":"foo","id":"3CeAU7C6izLxbu6X2pTXxyTKFZ6","branch":"main","vectors":false,"runs_found":0,"objects_compacted":0,"vectors_created":0}

$ for file in $(find store/3*/data -type f ! -name "*-seek.bsup"); do   echo $file:;   super $file; done
store/3CeAU7C6izLxbu6X2pTXxyTKFZ6/data/3CeAU7AvqD06XMDCxffaETzdG43.bsup:
{ts:2,msg:"bar"}
store/3CeAU7C6izLxbu6X2pTXxyTKFZ6/data/3CeAU7Dd2YqShV8TyopxxXAsasy.bsup:
{ts:1,msg:"foo"}
store/3CeAU7C6izLxbu6X2pTXxyTKFZ6/data/3CeAUA8RJletwtn8pqv31ezQRFq.bsup:
{ts:3,msg:"baz"}

Whereas at commit be46cd0 that was right before the merge of #6821, if we paste the same prep commands and then run manage, we now see "objects_compacted":2 and an additional data/ object whose contents represent the pool's combined contents after compaction.

$ super -version
Version: v0.3.0-48-gbe46cd060

$ super db -db store manage
{"level":"info","ts":1776735776.3141901,"logger":"pool","msg":"updating pool","name":"foo","id":"3CeAfvZnVi0H6qpbJLX29kBwIhk","branch":"main","vectors":false}
{"level":"info","ts":1776735776.324377,"logger":"pool","msg":"compaction completed","name":"foo","id":"3CeAfvZnVi0H6qpbJLX29kBwIhk","branch":"main","vectors":false,"runs_found":1,"objects_compacted":2,"vectors_created":0}

$ for file in $(find store/3*/data -type f ! -name "*-seek.bsup"); do   echo $file:;   super $file; done
store/3CeAfvZnVi0H6qpbJLX29kBwIhk/data/3CeAfz9NhgwsqUToHdtscVXS30x.bsup:
{ts:3,msg:"baz"}
store/3CeAfvZnVi0H6qpbJLX29kBwIhk/data/3CeAfvFEGL6vuYa6VPXS3BQSKkj.bsup:
{ts:2,msg:"bar"}
store/3CeAfvZnVi0H6qpbJLX29kBwIhk/data/3CeAfwZE6nCASWVRM4gCiV0soKK.bsup:
{ts:1,msg:"foo"}
store/3CeAfvZnVi0H6qpbJLX29kBwIhk/data/3CeAguX82Tx8GIS0aPvKpz4o8IQ.bsup:
{ts:3,msg:"baz"}
{ts:1,msg:"foo"}

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions