Improve TextLine and line table performance by packing existing data into unused bits

[CyrusNajmabadi]

Background

TextLine is a readonly struct returned whenever code asks "what line is position X on?" — which happens constantly during compilation, IDE features, and diagnostics. The line table (SourceText.LineInfo) is consulted for every such lookup.

What this PR does

TextLine._data — merge two int fields into one ulong

Previously TextLine stored _start: int and _endIncludingBreaks: int as separate fields. This PR replaces them with a single ulong _data encoding:

Bits	Field
63–62	line break length (0, 1, or 2)
61–31	start position (31 bits)
30–0	total length = EndIncludingLineBreak − Start (31 bits)

This makes EndIncludingLineBreak a trivial Start + length with no branching, and End a simple EndIncludingLineBreak − breakLen. Previously, End required calling TextUtilities.GetStartAndLengthOfLineBreakEndingAt on every access to determine the line break length.

_lineStarts — pack prior line break length into each entry

SourceText.LineInfo stores line start positions in a SegmentedList. These were plain ints. Since source positions are always non-negative and fit in 31 bits in practice, the top bit(s) of a uint were free.

Each _lineStarts[i] (for i > 0) records where a line started — which is always immediately after a line break. That prior line break is always length 1 or 2, never 0. This allows bias-1 encoding in a single top bit: 0 = length 1, 1 = length 2. The first entry (_lineStarts[0]) always starts at position 0 and its top bit is never read for break length purposes, since the indexer reads break length from _lineStarts[index + 1].

Result: LineInfo's indexer can call TextLine.FromSpanUnsafe(..., lineBreakLength) directly instead of leaving the length unknown and recomputing it on first access.

No loss of representable values

• TextLine previously used two ints; it now uses one ulong. The 31-bit start and 31-bit length fields cover the full non-negative int range.
• _lineStarts previously used 32-bit int start positions; it now uses 31-bit positions. Source texts are capped at int.MaxValue bytes (~2 GB) by the existing SourceText API, so 31 bits (2^31 − 1 ≈ 2 GB) covers the identical range.

Testing

Added a comprehensive test suite (TextLineNewLineTests) that exercises line parsing across all four SourceText implementations (StringText, LargeText, SubText, CompositeText) and every supported line break type (\n, \r, \r\n, \u0085, \u2028, \u2029).

The tests are structured in layers:

1. Ground truth tests verify exact Start/End/EndIncludingLineBreak values for simple scenarios (newline at start, middle, end, consecutive newlines, mixed newlines, etc.), parameterized across all text kinds and all newline types — so every assertion runs against every implementation.

2. Cross-type consistency tests take 25 interesting content patterns and compare every pair of text kinds against each other, catching any disagreement between implementations even if the expected values above were somehow wrong.

3. Implementation-specific edge cases target the tricky boundaries unique to each type: LargeText tests every chunk boundary position including splitting \r\n across chunks and single-character-per-chunk scenarios. SubText tests exhaustively check every possible substring of several content patterns, verifying against StringText as reference. CompositeText tests try every possible 2-way and 3-way split point for each content pattern, including cases where \r\n spans segment boundaries.

[CyrusNajmabadi]

@ToddGrun to review.

[CyrusNajmabadi]

could i get reviews on this? @phil-allen-msft ?

[CyrusNajmabadi]

@jjonescz ptal :)

[CyrusNajmabadi]

@dotnet/roslyn-compiler @jcouv for another set of eyes. thanks :)

[333fred]

One small nit, otherwise looking good.

[CyrusNajmabadi]

Thanks @333fred . Fixed.

[CyrusNajmabadi]

This is good to merge whenever you are happy @333fred

[CyrusNajmabadi]

Thanks! I got a nice clean round number :)