Feature | Use hardcoded LCID mappings when decoding strings

[edwardneal]

Description

This builds on #4051 and #584 by changing the mapping from collations/sort IDs to code pages, replacing it with a static list of valid mappings rather than a call to CultureInfo.GetCultureInfo.

This is performance-neutral, but it also brings us closer to supporting globalization invariant mode.

I've simply lifted the full set of mappings from the JDBC driver (here) and confirmed that the existing collation transcoding tests continue to pass on the newest SQL 2025 CU and on a SQL Azure instance.

Issues

Tested by #4051.
Addresses a backlog item listed in #584.
Contributes to #3742.

Testing

Manual testing against an on-prem SQL2025 instance and against a SQL Azure instance passes.

[mdaigle]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[benrr101]

In general, I like the idea. The only thing I would like changed is to give more labels to the magic values. Right now it looks like a book of mysterious numbers with no real explanation. They ultimately correspond with a locale of some kind, yeah? So it'd be nice to be able to see which values correspond to which locale.

[edwardneal]

Thanks @benrr101. Yes - it maps LCID (defined here) to code page IDs (except for LCIDs 0x827 and 0x2409, which don't appear in the specs or are marked as reserved, but still have an entry in the LCID table.)

The corresponding mapping here references the MS-LCID spec, which I can do. Many of the code pages are named Windows-1252 or similar though, so I'd be replacing a numeric value of 1252 with something like Windows_1252, which doesn't seem particularly helpful.

Do you just want the link to MS-LCID, or something more structural?

Edit - I've just added the LCID descriptions, and I think it looks slightly clearer now. What do you think?

[cheenamalhotra]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[Copilot]

Pull request overview

This PR updates collation/LCID-to-codepage resolution in TdsParser to use hardcoded mappings (aligned with other SQL Server drivers) instead of relying on CultureInfo.GetCultureInfo, helping move toward compatibility with globalization invariant mode.

Changes:

• Replaced TdsParser.GetCodePage logic with a call to LocalesHelper.TryGetCodePage(...).
• Introduced LocalesHelper containing hardcoded LCID→codepage and sortId→codepage mapping tables.
• Removed the legacy TdsEnums.CODE_PAGE_FROM_SORT_ID table.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/TdsParser.cs	Switches codepage resolution to use the new hardcoded mapping helper.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/TdsEnums.cs	Removes the old sort-id→codepage mapping table.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/LocalesHelper.cs	Adds hardcoded LCID/sortId mapping tables and lookup logic.

[codecov]

Codecov Report

❌ Patch coverage is 99.41860% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 64.10%. Comparing base (be95ca2) to head (f02c0de).
⚠️ Report is 19 commits behind head on main.

Files with missing lines	Patch %	Lines
...qlClient/src/Microsoft/Data/SqlClient/TdsParser.cs	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4212      +/-   ##
==========================================
- Coverage   65.96%   64.10%   -1.87%     
==========================================
  Files         275      272       -3     
  Lines       42993    65961   +22968     
==========================================
+ Hits        28361    42285   +13924     
- Misses      14632    23676    +9044     

Flag	Coverage Δ
CI-SqlClient	?
PR-SqlClient-Project	64.10% <99.41%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[apoorvdeshmukh]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[benrr101]

Much better - I appreciate it! :D