Add GlobalSecondaryIndexDefinition for GSI container support#47468
Add GlobalSecondaryIndexDefinition for GSI container support#47468mbhaskar wants to merge 1 commit into
Conversation
fef6964 to
713f56a
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds client-side support for Cosmos DB Global Secondary Index (GSI) containers in azure-cosmos, including a new GlobalSecondaryIndexDefinition model, plumbing through container create/replace APIs (sync + async) with a dual-write wire format, and CI/live-test wiring for a dedicated GSI-enabled account.
Changes:
- Added
GlobalSecondaryIndexDefinition(validation + (de)serialization) and exported it fromazure.cosmos. - Added
global_secondary_index_definitionkwarg support tocreate_container,create_container_if_not_exists, andreplace_containerfor both sync and async database clients (writing bothglobalSecondaryIndexDefinitionandmaterializedViewDefinition). - Added unit + live tests plus pipeline/matrix/marker updates to run GSI live coverage.
Show a summary per file
| File | Description |
|---|---|
| sdk/cosmos/tests.yml | Injects GSI account secrets as env vars for pipeline live runs. |
| sdk/cosmos/live-platform-matrix.json | Adds a dedicated matrix entry to run cosmosGSI marked live tests. |
| sdk/cosmos/azure-cosmos/tests/test_global_secondary_index.py | Unit tests for constructor validation and dict round-tripping. |
| sdk/cosmos/azure-cosmos/tests/test_global_secondary_index_live.py | Live tests that exercise creating GSI containers (class + raw dict). |
| sdk/cosmos/azure-cosmos/pytest.ini | Registers the new cosmosGSI pytest marker. |
| sdk/cosmos/azure-cosmos/CHANGELOG.md | Documents the new GSI feature in the Unreleased section. |
| sdk/cosmos/azure-cosmos/azure/cosmos/database.py | Adds sync API kwarg plumbing + dual-write of GSI definition in create/replace. |
| sdk/cosmos/azure-cosmos/azure/cosmos/aio/_database.py | Async mirror of sync GSI kwarg plumbing + dual-write behavior. |
| sdk/cosmos/azure-cosmos/azure/cosmos/_global_secondary_index.py | Introduces GlobalSecondaryIndexDefinition model implementation. |
| sdk/cosmos/azure-cosmos/azure/cosmos/init.py | Exports GlobalSecondaryIndexDefinition from the public package namespace. |
Copilot's findings
Comments suppressed due to low confidence (1)
sdk/cosmos/azure-cosmos/CHANGELOG.md:11
- The new changelog entry under "Features Added" is missing the usual reference/link to the PR (most other bullets in this section end with "See PR ####" or similar). Keeping the same format makes it easier to trace changes back to discussions and reviews.
#### Bugs Fixed
#### Other Changes
- Files reviewed: 10/10 changed files
- Comments generated: 1
|
@sdkReviewAgent-2 |
|
/azp run python - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
✅ Review complete (38:22) Posted 5 inline comment(s). Steps: ✓ context, correctness, cross-sdk, design, history, past-prs, synthesis, test-coverage |
|
✅ Review complete (38:58) Posted 6 inline comment(s). Steps: ✓ context, correctness, cross-sdk, design, history, past-prs, synthesis, test-coverage |
simorenoh
left a comment
There was a problem hiding this comment.
In merging main it seems this PR has manually pulled in a lot of other changes and now touches 200k+ lines across several repos - let's please clean that up or open a new one.
- Create GlobalSecondaryIndexDefinition class with validation, serialization, and dual-write pattern (globalSecondaryIndexDefinition + materializedViewDefinition) - Add global_secondary_index_definition keyword to create_container, create_container_if_not_exists, and replace_container (sync and async) - Add unit tests (21 tests) and live test infrastructure - Add GSI pipeline configuration (pytest mark, matrix entry, env var mapping) - Update CHANGELOG.md and api.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
b6a8c4a to
564afb1
Compare
Yes AI quirk! asked it to resolve conflicts and this is the result! |
Summary
Implements Global Secondary Index (GSI) container support in the Python Cosmos SDK, mirroring the Java SDK implementation (Azure/azure-sdk-for-java#48480).
A GSI container is a derived container built from a source container using a SQL projection query. The service already supports this; this PR adds the client-side support.
Changes
New files
azure/cosmos/_global_secondary_index.py—GlobalSecondaryIndexDefinitionclass with constructor validation, serialization (_to_dict/_from_dict), and read-only server-populated properties (source_container_rid,status)tests/test_global_secondary_index.py— 20 unit tests covering constructor validation, serialization, deserialization, forward-compatibilitytests/test_global_secondary_index_live.py— Live integration tests using dedicated GSI test accountModified files
azure/cosmos/__init__.py— ExportGlobalSecondaryIndexDefinitionazure/cosmos/database.py— Addedglobal_secondary_index_definitionkwarg tocreate_container,create_container_if_not_exists, andreplace_container(all overloads) with dual-write patternazure/cosmos/aio/_database.py— Async mirror of all sync changesCHANGELOG.md— Feature entrypytest.ini— AddedcosmosGSImarker for live testslive-platform-matrix.json— AddedGSITestConfigmatrix entrytests.yml— Mappedgsi-pipeline-uri/gsi-pipeline-keyKV secrets to env varsDesign Decisions
Dual-Write Pattern
For backward compatibility with older service versions, the SDK writes the GSI definition under both:
"globalSecondaryIndexDefinition"(new key)"materializedViewDefinition"(legacy key)On read, the new key is preferred with fallback to the legacy key.
Status as String Constants (not enum)
Status values (
"Initializing","InitialBuildAfterCreate","InitialBuildAfterRestore","Active","DeleteInProgress") are kept as plain strings for forward-compatibility with future service-defined values.Pattern Consistency
Follows the same integration pattern as
computed_properties,vector_embedding_policy,full_text_policy, andchange_feed_policy— kwarg extraction viakwargs.pop(), added to definition dict if not None.Testing
gsi-pipeline-uri/gsi-pipeline-keyfrom KV (same secrets used by Java SDK pipeline)