Skip to content

Add logging, Timestamp warning, and Cosmos DB limitation docs#240

Open
philnach wants to merge 1 commit intoAzureCosmosDB:mainfrom
philnach:user/philnach/TableClarification
Open

Add logging, Timestamp warning, and Cosmos DB limitation docs#240
philnach wants to merge 1 commit intoAzureCosmosDB:mainfrom
philnach:user/philnach/TableClarification

Conversation

@philnach
Copy link
Copy Markdown
Collaborator

Closes #239

Problem

When using QueryFilter with Timestamp on a Cosmos DB Table API source, the DMT silently returns 0 results with no indication of why. Per Microsoft documentation Query by using an OData filter, "Azure Cosmos DB blocks the queries on timestamp properties." This is a known Cosmos DB limitation, not a DMT bug.

Changes

AzureTableAPIDataSourceExtension.cs

  • Log table name and QueryFilter value at read start
  • Log item count after enumeration
  • Warn when 0 items are returned
  • Detect Timestamp in QueryFilter and emit a specific warning about the Cosmos DB limitation
  • Replace manual GetAsyncEnumerator()/MoveNextAsync() with await foreach with proper WithCancellation(cancellationToken)

README.md

  • Add prominent callout about Cosmos DB Table API Timestamp filter limitation with links to official docs and SDK issues
  • Add workaround guidance (use custom datetime properties instead)

Testing

  • All 31 existing unit tests pass
  • Verified against a live Cosmos DB Table API account: confirmed all Timestamp filters return 0 rows while non-Timestamp filters (PartitionKey, custom properties) return correct results.

@philnach
Copy link
Copy Markdown
Collaborator Author

@markjbrown / @bowencode, whenever you get a moment if you can code review and approve the PR it'll help bring clarity to the Query with timestamp functionality in DMT.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AzureTableAPI Source: QueryFilter with Timestamp is not working

2 participants