CASSANDRA-21131: Fix CSV COPY TO/FROM corrupting text values containing backslashes

[Jens-G]

Summary

COPY TO followed by COPY FROM corrupts text column values that contain backslashes: each round-trip doubles the backslash count. Reported in CASSANDRA-21131.

Before (one round-trip):

• Stored: V\S → exported CSV: V\\\\S → re-imported: V\\S ❌
• Stored: \"Marianne"\ → re-imported: \\"Marianne"\\ ❌

list<text>, set<text>, map<text,text>, tuples and UDTs with text fields are affected in the same way.

Root Cause

format_value_text in formatting.py doubles backslashes unconditionally:

escapedval = val.replace('\\', '\\\\')

This is intentional for terminal display (SELECT output shows V\\S so the backslash is visible). However, ExportProcess.format_value in copyutil.py calls the same function when writing CSV. The csv.writer (configured with escapechar='\\') then escapes backslashes a second time, quadrupling them in the CSV file. On COPY FROM the csv.reader unescapes once, leaving doubled backslashes in Cassandra.

Fix

Add an escape_backslash parameter (default True, preserving existing terminal display behaviour) to format_value_text, format_simple_collection, and all collection formatters. Pass escape_backslash=False from ExportProcess.format_value so the csv.writer handles all backslash escaping exclusively.

Changed functions:

• format_value_text — new parameter
• format_simple_collection — new parameter, propagated to element format_value calls
• format_value_list, format_value_set, format_value_tuple — new parameter, forwarded to format_simple_collection
• format_value_map — new parameter, propagated through subformat
• format_value_utype — new parameter, propagated through format_field_value
• ExportProcess.format_value in copyutil.py — passes escape_backslash=False

Test Plan

Two standalone Python test scripts (no running Cassandra cluster required) are attached to the JIRA ticket and verify the bug and fix:

• test_cassandra_21131.py — 10 test cases for plain text columns: 5/10 pass before fix → 10/10 after
• test_cassandra_21131_collections.py — 12 test cases for list/set/map<text>: 3/12 before → 12/12 after

Integration testing against a live cluster with the exact scenario from the bug report (COPY TO → TRUNCATE → COPY FROM → SELECT) is needed before merge.

Notes

• A separate but related bug (UNICODE_CONTROLCHARS_RE converting control chars like \n to repr-notation \\n during CSV export) was discovered and will be tracked in a separate ticket.
• The Generated-by: commit token is included per ASF generative tooling policy. The fix was developed with AI assistance (Claude Sonnet 4.6 / Anthropic) under human review and direction. All code has been verified manually.

🤖 Generated with Claude Code

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[bschoening]

@Jens-G It seems the exporter is sending values through the display formatter, which doubles backslashes for human-readable SELECT output, before handing them to the CSV writer. Claude suggests just stop using the display formatter for CSV export of text in copyutil.py. What would you think of that approach?

def format_value(self, val, cqltype):
    if val is None or val == EMPTY:
        return format_value_default(self.nullval, colormap=NO_COLOR_MAP)

    # Text-like values: pass through unmodified. The csv.writer will apply
    # CSV-level escaping using the dialect's escapechar. Running these
    # through the display formatter would double-escape backslashes, see
    # CASSANDRA-21131.
    if cqltype.type_name in ('text', 'varchar', 'ascii'):
        return val
    ...

[Jens-G]

TBH if it works I'm fine with either approach 👍 🚀

[bschoening]

Consider an alternative approach where formated_value() bypasses formatting for text

if cqltype.type_name in ('text', 'varchar', 'ascii'):
return val

[bschoening]

Consider an alternative approach where formatted_value() bypasses display formatting for text.

format_value():
...
if cqltype.type_name in ('text', 'varchar', 'ascii'):
    return val if val.isprintable() else None

[bschoening]

Please add unit tests

[arvindKandpal-ksolves]

Hi @Jens-G , if you are busy I can add the unit tests for this and you can review that — just let me know if that works. ?

[Jens-G]

+1 LGTM

[bschoening]

@Jens-G the PR needs additional work. Based on our discuss of RFC 4180, proper text values don't require formatting. In format_value():

if cqltype.type_name in ('text', 'varchar', 'ascii'):
    return val if val.isprintable() else None

that may eliminate the need to have a new argument for escape_backslash