Skip to content

fix: don't drop fork messages containing U+2028/U+2029/U+0085#1092

Open
Otis0408 wants to merge 1 commit into
anthropics:mainfrom
Otis0408:fix/fork-session-splitlines-dataloss
Open

fix: don't drop fork messages containing U+2028/U+2029/U+0085#1092
Otis0408 wants to merge 1 commit into
anthropics:mainfrom
Otis0408:fix/fork-session-splitlines-dataloss

Conversation

@Otis0408

@Otis0408 Otis0408 commented Jul 8, 2026

Copy link
Copy Markdown

Summary

fork_session() can silently drop a message whose content contains a raw U+2028 (LINE SEPARATOR), U+2029 (PARAGRAPH SEPARATOR), or U+0085 (NEL).

_parse_fork_transcript splits the raw transcript with str.splitlines():

for line in content.decode("utf-8", errors="replace").splitlines():

str.splitlines() breaks on U+2028/U+2029/U+0085 in addition to \n. But those characters are not JSONL line breaks — they are valid unescaped inside a JSON string (JSON only requires escaping U+0000–U+001F), and the CLI's Node JSON.stringify emits them raw. So a user/assistant message whose content contains one lands whole and valid inside a single .jsonl line. splitlines() fragments that line into pieces that each fail json.loads, so the message is dropped from the fork (and any child pointing at its parentUuid is orphaned).

The read path reads the same file fine, because it splits on "\n" only:

# _parse_transcript_entries (sessions.py)
end = content.find("\n", start)

So get_session_messages(sid) returns the message, but fork_session(sid) produces a fork that's missing it — an internal-contract divergence and real data loss. If the affected message is the session's only message, fork raises ValueError("... has no messages to fork") for a session that reads fine.

Fix

Split on "\n" to match the read path. line.strip() already normalizes trailing \r, so this is equivalent to the read path's tolerant behavior. fork_session_via_store is unaffected (it iterates already-parsed store objects rather than re-splitting text).

Tests

  • test_parse_fork_transcript_keeps_unicode_line_separators — unit test parametrized over U+2028 / U+2029 / U+0085, asserting the entry survives.
  • test_fork_preserves_message_with_line_separator — end-to-end: a session whose assistant message contains a raw U+2028 forks with the same message count.

Verified both fail on main (message dropped) and pass with the fix. Full suite: 1060 passed, 5 skipped; ruff and mypy clean.

_parse_fork_transcript split the raw transcript with str.splitlines(), whose
line-boundary set includes U+2028 (LINE SEPARATOR), U+2029 (PARAGRAPH
SEPARATOR) and U+0085 (NEL). Those are not JSONL line breaks: they are valid
*unescaped* inside a JSON string and are emitted raw by the CLI's Node
JSON.stringify, so a message whose content contains one lands whole and valid
inside a single .jsonl line. splitlines() fragmented such a line into pieces
that each fail json.loads, so fork_session silently dropped that message (and
orphaned any child pointing at it), even though the read path
(get_session_messages / _parse_transcript_entries) reads it fine because it
splits on "\n" only.

Split on "\n" to match the read path. Adds a unit test over all three
separators and an end-to-end test asserting fork preserves a message
containing a raw U+2028.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant