Skip to content

Fix case-insensitive column name collision in SQLite storage#84

Merged
joedursun merged 2 commits into
massive-com:masterfrom
november-pain:fix/case-insensitive-column-dedup
Apr 8, 2026
Merged

Fix case-insensitive column name collision in SQLite storage#84
joedursun merged 2 commits into
massive-com:masterfrom
november-pain:fix/case-insensitive-column-dedup

Conversation

@november-pain
Copy link
Copy Markdown
Contributor

Fixes #83

Endpoints like /v2/aggs/ticker/AAPL/prev return both T (ticker) and t (timestamp). SQLite CREATE TABLE rejects columns that differ only in case, so query_data fails with duplicate column name: t.

This renames colliding columns in Table.from_records() by appending _2 to the later column. Original casing is preserved when there is no collision.

Example: ["T", "v", "t"]["T", "v", "t_2"]

Tests added for deduplication, multiple collisions, and a full store + query round-trip.

Endpoints like /v2/aggs/ticker/AAPL/prev return both T (ticker)
and t (timestamp). SQLite CREATE TABLE rejects columns that differ
only in case, causing query_data to fail with "duplicate column
name: t".

Rename colliding columns in Table.from_records() by appending _2
to the later column. Original casing is preserved when there is no
collision.

Fixes massive-com#83
Comment thread tests/test_store.py Outdated
Comment thread tests/test_store.py Outdated
@joedursun
Copy link
Copy Markdown
Collaborator

@november-pain thanks for the PR! Overall looks good. I left a few comments and once you address them I can run the tests and merge the PR assuming everything passes.

@november-pain
Copy link
Copy Markdown
Contributor Author

@november-pain thanks for the PR! Overall looks good. I left a few comments and once you address them I can run the tests and merge the PR assuming everything passes.

Hi, thanks. Pushed the fix.

@joedursun
Copy link
Copy Markdown
Collaborator

@november-pain there's just a single linter error that needs to be addressed. After that I'll merge it in.

I've got another PR with some startup performance improvements that will go out with this in v0.8.8.

@november-pain
Copy link
Copy Markdown
Contributor Author

@november-pain there's just a single linter error that needs to be addressed. After that I'll merge it in.

I've got another PR with some startup performance improvements that will go out with this in v0.8.8.

The lint failure is pre-existing on master — ty doesn't recognize the # type: ignore[list-item] directive in tests/test_functions.py:1042. Not related to this PR.

@joedursun joedursun merged commit 581350d into massive-com:master Apr 8, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

store_as + query_data fails with 'duplicate column name' on endpoints returning T and t columns

2 participants