Skip to content

Commit b09bbed

Browse files
Archive.delete: don't reuse msgpack Unpacker after an unpacking failure
msgpack does not support reusing an Unpacker instance after an unpacking failure - a new instance must be created instead. In the forced-delete path, item metadata was unpacked via "for item in unpacker" with an "except (TypeError, ValueError)" handler. msgpack's FormatError/StackError are ValueError subclasses, so a real unpacking failure was caught there and the (now broken) unpacker kept being reused for the following chunks, making them all fail too - leaking their chunk references. Restructure the loop to use next(unpacker) so each failure mode gets a tight handler: - StopIteration: buffer drained, feed next chunk - msgpack.UnpackException: corruption; in forced mode replace the unpacker with a fresh instance and skip the rest of the chunk - TypeError/ValueError: only wraps Item processing (bad types yielded) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 8abdd3b commit b09bbed

1 file changed

Lines changed: 22 additions & 9 deletions

File tree

src/borg/archive.py

Lines changed: 22 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1035,22 +1035,35 @@ def chunk_decref(id, stats, part=False):
10351035
data = self.key.decrypt(items_id, data)
10361036
unpacker.feed(data)
10371037
chunk_decref(items_id, stats)
1038-
try:
1039-
for item in unpacker:
1038+
while True:
1039+
try:
1040+
item = next(unpacker)
1041+
except StopIteration:
1042+
# no more (complete) items in the buffer, feed the next chunk
1043+
break
1044+
except msgpack.UnpackException:
1045+
# items metadata corrupted. the unpacker can't be reused after an
1046+
# unpacking failure, so create a fresh one and skip the rest of this chunk.
1047+
if forced == 0:
1048+
raise
1049+
error = True
1050+
unpacker = msgpack.Unpacker(use_list=False)
1051+
break
1052+
try:
10401053
item = Item(internal_dict=item)
10411054
if 'chunks' in item:
10421055
part = not self.consider_part_files and 'part' in item
10431056
for chunk_id, size, csize in item.chunks:
10441057
chunk_decref(chunk_id, stats, part=part)
1045-
except (TypeError, ValueError):
1046-
# if items metadata spans multiple chunks and one chunk got dropped somehow,
1047-
# it could be that unpacker yields bad types
1048-
if forced == 0:
1049-
raise
1050-
error = True
1058+
except (TypeError, ValueError):
1059+
# if items metadata spans multiple chunks and one chunk got dropped somehow,
1060+
# it could be that unpacker yields bad types
1061+
if forced == 0:
1062+
raise
1063+
error = True
10511064
if progress:
10521065
pi.finish()
1053-
except (msgpack.UnpackException, Repository.ObjectNotFound):
1066+
except Repository.ObjectNotFound:
10541067
# items metadata corrupted
10551068
if forced == 0:
10561069
raise

0 commit comments

Comments
 (0)