Problem

When server-side rendering content that contains malformed UTF-16 (e.g., lone surrogate pairs like \uD83D without its pair \uDE00), Ruby's JSON.parse throws an error while JavaScript's JSON.parse accepts it. This causes SSR to fail completely instead of letting the browser handle the display (typically showing replacement characters).

React on Rails should be a general-purpose framework that doesn't crash on edge-case Unicode issues — it should pass content through to the browser and let the browser render it as it sees fit.

Root Cause

JavaScript and Ruby have different interpretations of "valid JSON" for Unicode:

When React renders content containing a lone surrogate (e.g., from corrupted user data, database encoding issues, or API responses), JavaScript's JSON.stringify produces valid JS JSON that Ruby's JSON.parse rejects.

Scope of Investigation

1. Identify all JSON.parse paths that could be affected

Known paths:

• length_prefixed_parser.rb:156 — payloadType: "object" path
• length_prefixed_parser.rb:122 — metadata JSON parsing
• ruby_embedded_java_script.rb:423 — legacy JSON fallback for old bundles

Need to audit for any others.

2. Identify other JSON.parse compatibility issues beyond surrogates

Ruby's JSON.parse also fails on:

• BOM at start of JSON
• Invalid escape sequences (\x, \a)
• Unescaped control characters
• Possibly others

Investigate which of these could realistically occur from JavaScript output.

3. Determine the right fix approach

Options to evaluate:

• Preprocessing JSON before parsing
• Alternative JSON parsers
• Catch-and-retry with sanitization
• Other approaches

Consider performance implications and edge cases.

Reproduction

require "json"

# This is valid JSON in JavaScript but fails in Ruby
json_str = '{"html": "Hello \\uD83D world"}'

JSON.parse(json_str)
# => JSON::ParserError: incomplete surrogate pair at 'Hello' at line 1 column 11

Context

• Reported by a client using an older version of React on Rails (pre-length-prefixed format)
• The length-prefixed format (v16.7.0+) mitigates this for payloadType: "string" since HTML is sent as raw bytes
• But the issue remains for object payloads, metadata, and legacy format fallback