feat: Add global locale support for number parsing#1165
Merged
Conversation
Add locale parameter to top-level API functions to support locale-specific number formats using Babel's CLDR data. - Add locale_number_parser module with Babel-based parsing - Add locale_context module for thread-safe locale management - Update try_float() to respect locale context - Add locale parameter to allotrope_from_io/file functions - Add babel >= 2.9.0 dependency - Add comprehensive tests for locale parsing and context This enables parsing numbers in different locales (e.g., German "1.234,56" vs US "1,234.56") by passing locale="de_DE" to the top-level API functions. Co-Authored-By: Claude Opus 4.1 <noreply@anthropic.com>
nathan-stender
force-pushed
the
feat/global-locale-support
branch
from
April 6, 2026 06:02
b4f38b3 to
aedef7a
Compare
Change from generic Exception to AllotropeConversionError to satisfy ruff B017 linting rule. Co-Authored-By: Claude Opus 4.1 <noreply@anthropic.com>
nathan-stender
force-pushed
the
feat/global-locale-support
branch
from
April 6, 2026 06:05
aedef7a to
1a782c8
Compare
…port Replace pd.to_numeric, astype(float), and float() calls with try_float utilities to enable locale-aware number parsing across all parsers. Changes: - roche_cedex_bioht: Replace to_num() with try_float_or_none + np.nan replacement - luminex_xponent: Replace pd.to_numeric with try_float_or_none for analyte columns - moldev_softmax_pro: Replace pd.to_numeric with try_float for index conversion - thermo_skanit: Replace float() with try_float for wavelength parsing - thermo_skanit: Replace astype(float) with try_float for column headers - agilent_gen5: Replace astype(float) with try_float for wavelength parsing - cytiva_biacore_t200_control: Replace astype(float) with try_float for sensorgram data - thermo_fisher_genesys30: Replace astype(float) with try_float for absorbance data - appbio_absolute_q: Replace astype(float) with try_float for fluorescence data All parser tests pass with no changes to golden files. Co-Authored-By: Claude Opus 4.1 <noreply@anthropic.com>
nathan-stender
changed the title
Add convenience function to simplify the common pattern of converting pandas Series to list[float] using locale-aware parsing. Changes: - Add series_to_float_list() helper to utils/pandas.py - Update 4 parsers to use the new helper: - cytiva_biacore_t200_control: sensorgram time and resonance data - thermo_fisher_genesys30: absorbance data - appbio_absolute_q: cycle index and fluorescence data - agilent_gen5: wavelength parsing Replaces verbose list comprehensions with cleaner helper calls: Before: [try_float(str(v), "name") for v in series] After: series_to_float_list(series, "name") Co-Authored-By: Claude Opus 4.1 <noreply@anthropic.com>
stephenworlow
approved these changes
Apr 6, 2026
nathan-stender
added a commit
that referenced
this pull request
Apr 7, 2026
Resolves P1 TODO #3 from codebase audit by adding support for locale-specific date formats (day-first vs month-first) using Babel's CLDR data. ## Problem Previously, TimestampParser always used American date format (MM/DD/YYYY), causing ambiguous dates like "01/02/2023" to parse incorrectly for European users who expect DD/MM/YYYY format. ## Solution - Use Babel CLDR data to determine if a locale uses day-first format - Integrate with existing locale context (from PR #1165) - ISO 8601 year-first formats (YYYY-MM-DD) always parse correctly - Defaults to American format when no locale context is set ## Implementation - Added `_should_use_day_first()` helper that: - Returns False for ISO 8601 year-first formats (YYYY-MM-DD) - Uses Babel to check locale's short date format pattern - Returns True if day appears before month in pattern - `TimestampParser.parse()` now gets locale from context and passes `dayfirst` parameter to `dateutil.parser.parse()` - Comprehensive test coverage (18 tests) for various locales and formats ## Testing - All new tests pass (18 timestamp parser tests + 3 locale integration tests) - No changes to existing behavior when locale not set - Tested: en_US, en_GB, de_DE, fr_FR, es_ES, it_IT, pt_PT, nl_NL, ja_JP, zh_CN, ko_KR Co-Authored-By: Claude Opus 4.1 <noreply@anthropic.com>
nathan-stender
added a commit
that referenced
this pull request
Apr 7, 2026
## Summary Adds locale-aware timestamp parsing to resolve P1 TODO #3 from codebase audit. Uses Babel's CLDR data to correctly parse ambiguous dates based on locale (e.g., "01/02/2023" → January 2 in US, February 1 in Europe). ## Problem Previously, `TimestampParser` always assumed American date format (MM/DD/YYYY): ```python # Line 23 in timestamp_parser.py (before) # TODO: TimestampParser should support localization -- e.g., passing "dayfirst=True" to dateutil.parser.parse. ``` This caused incorrect parsing for international users: - `"01/02/2023"` always parsed as January 2nd, never February 1st - European/Asian date formats failed for ambiguous dates - No way to specify locale-specific parsing behavior ## Solution Integrated with existing locale context infrastructure (from PR #1165) using **Babel CLDR** (industry standard): ### Key Changes 1. **`_should_use_day_first(time_str, locale_str)`** helper: - Detects ISO 8601 year-first (YYYY-MM-DD) → always month-before-day - Uses Babel to check locale's short date format pattern from CLDR - Returns `True` if day appears before month in pattern 2. **`TimestampParser.parse()`** enhancement: - Gets locale from context via `get_current_locale()` - Passes `dayfirst` parameter to `dateutil.parser.parse()` - Backward compatible: defaults to American format when no locale set ### How it works ```python from allotropy.to_allotrope import allotrope_from_file # American locale (month-first) allotrope_from_file("data.txt", "VENDOR", locale="en_US") # "01/02/2023" → 2023-01-02 (January 2nd) # German locale (day-first) allotrope_from_file("data.txt", "VENDOR", locale="de_DE") # "01/02/2023" → 2023-02-01 (February 1st) # ISO format works everywhere # "2023-01-02" → 2023-01-02 (always January 2nd) ``` ## Babel CLDR Data Uses Unicode CLDR via Babel - the authoritative source for locale data: - **Day-first**: `en_GB`, `de_DE`, `fr_FR`, `es_ES`, `it_IT`, most of Europe - **Month-first**: `en_US` (only major English variant) - **Year-first**: `ja_JP`, `zh_CN`, `ko_KR` (YMD format) ## Testing Comprehensive test coverage (18 tests): - ✅ ISO 8601 formats ignore locale (always YYYY-MM-DD) - ✅ Ambiguous dates parse correctly by locale (`"01/02/2023"`) - ✅ Unambiguous dates work everywhere (`"31/01/2023"`) - ✅ Multiple European locales tested - ✅ Asian year-first locales tested - ✅ Backward compatible (no locale → American format) - ✅ Real-world instrument date formats - ✅ All 546 existing tests pass ## Impact - **No breaking changes** - backward compatible - **No golden file changes** - existing behavior preserved when locale not set - **Enables international support** - fixes dates for European/Asian users - **Industry standard** - uses Babel CLDR, same as browsers/ICU 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.1 <noreply@anthropic.com>
nathan-stender
added a commit
that referenced
this pull request
Apr 8, 2026
### Added - Add locale-aware timestamp parsing using Babel CLDR data (#1167) - Add global locale support for number parsing (#1165) - Reduce Cytiva Biacore T200 Control memory usage with cycle streaming (#1164) - Cytiva T200 - Implement streaming decoder to reduce memory usage by 55% (#1163) ### Fixed - Default to day-first format for invalid/unknown locales (#1168) - Use immutable copy pattern for WellItem result attachment (#1166)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
localeparameter to top-level API functions (allotrope_from_io,allotrope_from_file, etc.) to support parsing numbers in locale-specific formats.locale_number_parsermodule using Babel's CLDR data for locale-aware number parsinglocale_contextmodule for thread-safe locale management using ContextVartry_float()to respect locale context when parsing numberslocaleparameter to all top-level API functions into_allotrope.pybabel >= 2.9.0dependencyTest plan
Usage example:
🤖 Generated with Claude Code