Add comprehensive MAP type support for SQLAlchemy ORM#589
Merged
Conversation
This implements complete MAP type support addressing GitHub issue #553: ## Core Implementation - Add _to_map converter function supporting JSON and Athena native formats - Create AthenaMap and MAP type classes for SQLAlchemy integration - Add visit_map methods to type compiler for SQL generation - Smart format detection for optimal performance ## Features - Full SQLAlchemy ORM support with MAP<key_type, value_type> syntax - Automatic conversion between string and dictionary representations - Support for both JSON and Athena native {key=value} formats - Type-safe key and value type specifications - Performance optimized with format detection ## Testing & Documentation - Comprehensive unit tests for types, compiler, and converter - Integration tests with actual Athena MAP queries - Complete documentation with usage examples and best practices - Migration guide for existing raw string handling Closes #553 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Handle complex structures (arrays/structs) by returning None for string fallback - Convert numeric keys to strings for test compatibility - Skip parsing MAP values containing complex nested structures - Maintain backward compatibility with existing behavior 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Move numeric key test from STRUCT to MAP (correct type)
- Update SQLAlchemy reflection test expectations for MAP conversion
- Add proper STRUCT test with named fields
- Ensure tests match the actual data type semantics
{1=2, 3=4} is MAP format, not STRUCT format.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Convert STRUCT, ARRAY, MAP, and complex combination tests to use @pytest.mark.parametrize - Fix MAP type test issues by using JSON casting for complex structures - Improve test readability and maintainability - Reduce code duplication in test methods - Fix line length issues for better code formatting - All tests now run independently with clear failure identification 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Convert all STRUCT converter tests to use @pytest.mark.parametrize - Consolidate 25 test cases into organized parametrized groups: - JSON format tests (5 cases) - Athena native format tests (8 cases) - Complex cases with special characters (3 cases) - Non-dict JSON format tests (3 cases) - DefaultTypeConverter tests (5 cases) - Remove code duplication and improve test maintainability - All 25 tests pass with clear individual case identification 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Update test_complex to expect MAP values as converted dictionaries instead of strings
- Update pandas test_as_pandas with same MAP expectations
- Fix DBAPI type assertions for MAP columns
- All CI failing tests now pass locally
MAP type converter now properly converts {1=2, 3=4} → {'1': 2, '3': 4}
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Replace hasattr() with isinstance() for AthenaStruct and AthenaMap - Add explicit type imports for better type safety - Simplify struct field checking logic with guaranteed type safety - More robust and performant type checking following SQLAlchemy patterns This eliminates potential AttributeError risks and provides clearer code semantics. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements complete MAP type support for SQLAlchemy ORM, addressing GitHub issue #553. Users can now work with MAP types seamlessly in their PyAthena applications with full type safety and automatic conversion.
Implementation Details
Core Features
_to_map()function supporting both JSON and Athena native formatsMAP<key_type, value_type>SQL syntaxUsage Examples
Table Definition:
Query Results:
Testing
Documentation
Test Plan
This follows the same architectural patterns established in the STRUCT type implementation (#587), ensuring consistency and maintainability.
Closes #553
🤖 Generated with Claude Code