Duplicate timestamps silently drop files

[black97devil]

Repository: ToTheMax/Snapchat-All-Memories-Downloader​
Title: BUG: Duplicate timestamps silently drop files (122k→69k, 347→297)

text

Bug Description

Expected: All parsed JSON entries → matching files
Actual:

• Main export: 122,188 parsed → 69,228 files (43% loss)
• Test case: 347 entries → 297 files (14% loss)

Fresh empty folder + --no-skip-existing still produces fewer files.

Latest run:

Downloaded: 297 (X.X MB @ Y.YY MB/s) | Skipped: 0 | Failed: 0
text
Folder count: 297 files (50 entries vanished).

Root Cause Analysis

Filename collision - second-precision only:
@Property
def filename(self) -> str:
return self.date.strftime("%Y-%m-%d_%H-%M-%S") # No milliseconds!

text

Snapchat behavior: Rapid clips (stories, bursts) share identical timestamps:
Clip 1: "2025-06-01 14:32:10 UTC" → 2025-06-01_14-32-10.mp4 ✓
Clip 2: "2025-06-01 14:32:10 UTC" → sees file → DROPPED silently

text

Even --no-skip-existing fails - first clip overwrites prior run.

Reproduction Steps

1. Export Snapchat data → html/memories_history.html (or JSON)
2. Delete output folder completely
3. python main.py input.json -o ./test --no-skip-existing
4. dir /s /b ./test\*.* | find /c /v "" → < parsed count

Tested: 347→297, 122188→69228 consistently.

SUSPECTED Fix (Untested)

Add timestamp counter in download_all() ~line 128:

name_counts = {}
for memory in memories:
count = name_counts.get(memory.filename, 0)
name_counts[memory.filename] = count + 1
unique_name = f"{memory.filename}_{count}" if count else memory.filename
memory._unique_name = unique_name # Temp attr for download_memory()

text
jpg_path = output_dir / f"{unique_name}.jpg"
mp4_path = output_dir / f"{unique_name}.mp4"

... rest unchanged

text

In download_memory() ~line 90:
name = getattr(memory, '_unique_name', memory.filename)
output_path = output_dir / f"{name}{ext}"

text

Expected result: 2025-06-01_14-32-10.mp4, _1.mp4, _2.mp4 → 347 files.

Request

1. Confirm bug - does maintainer see same 347→297 loss?
2. Validate fix - test above patch on sample data
3. Better solution - use Date milliseconds if available in JSON? [web:38]

Environment

• Windows 11, Python 3.12+
• Multiple fresh exports tested
• Both HTML/JSON formats affected [memory:21]

[black97devil]

fixed by

Changed Lines Only (5 Lines Added/Modified)
Line 130 (added before to_download = []):

python
name_counts = {}
Lines 132-144 (replaced original 128-134 filename check):

python

OLD (lines 128-134):

for memory in memories:

jpg_path = output_dir / f"{memory.filename}.jpg"

mp4_path = output_dir / f"{memory.filename}.mp4"

if skip_existing and (jpg_path.exists() or mp4_path.exists()):

stats.skipped += 1

else:

to_download.append(memory)

NEW (lines 132-144):

for memory in memories:
base_name = memory.filename
count = name_counts.get(base_name, 0)
name_counts[base_name] = count + 1
unique_name = f"{base_name}_{count}" if count else base_name
memory._unique_name = unique_name
jpg_path = output_dir / f"{unique_name}.jpg"
mp4_path = output_dir / f"{unique_name}.mp4"
if skip_existing and (jpg_path.exists() or mp4_path.exists()):
stats.skipped += 1
else:
to_download.append(memory)
Line 99 (in download_memory, replaced output_path line):

python

OLD:

output_path = output_dir / f"{memory.filename}{ext}"

NEW:

name = getattr(memory, '_unique_name', memory.filename)
output_path = output_dir / f"{name}{ext}"
Line 152 (added debug print):

python
print(f"Parsed {len(memories)} memories, downloading {len(to_download)} unique files...")