Add Cosmos Linux emulator failure analysis report

Co-authored-by: AndriySvyryd <6539701+AndriySvyryd@users.noreply.github.com>

Author: Copilot

test/EFCore.Cosmos.FunctionalTests/cosmos-linux-emulator-test-failures.md

@@ -0,0 +1,293 @@
+# Cosmos Linux emulator (`vnext-latest`) — remaining test failures
+
+This document summarizes the test failures that remain after switching the test
+suite to the Linux Cosmos emulator image
+`mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-latest` and removing
+all the `IsNotLinuxEmulator` / `SkipOnLinuxEmulator` skip code and annotations.
+
+Each group below has a **non-EF repro** that uses only the `Microsoft.Azure.Cosmos`
+SDK so the underlying emulator behavior can be reported to the Cosmos DB emulator
+team without any EF Core involvement.
+
+## How to run the repros
+
+All repros connect to a running emulator with the well-known emulator credentials:
+
+```csharp
+using Microsoft.Azure.Cosmos;
+
+const string endpoint = "https://localhost:8081";
+const string key = "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";
+
+using var client = new CosmosClient(
+    endpoint,
+    key,
+    new CosmosClientOptions { ConnectionMode = ConnectionMode.Gateway });
+
+var db = (await client.CreateDatabaseIfNotExistsAsync("repro")).Database;
+```
+
+(`ConnectionMode.Gateway` and disabling SSL validation are how the EF test
+infrastructure talks to the local emulator.)
+
+## Methodology
+
+1. Changed the image to `vnext-latest` in `CosmosTestEnvironment.cs` and
+   `.github/workflows/copilot-setup-steps.yml`.
+2. Removed every Linux-emulator skip mechanism (`EmulatorType`, `IsLinuxEmulator`,
+   `IsNotLinuxEmulator`, `SkipOnLinuxEmulator`, the `[ConditionalClass]` guard, and
+   the `LinuxEmulatorSaveChangesInterceptor`).
+3. Ran the full `EFCore.Cosmos.FunctionalTests` suite, and ran the same suite at the
+   previous commit (`HEAD~1`) as a baseline to separate pre-existing failures from
+   the ones revealed by un-skipping.
+4. Re-ran every newly-failing test **class in isolation** to separate deterministic
+   emulator bugs from failures that are only caused by load/contention when the whole
+   suite hits a single local emulator at once.
+
+After isolation, **97 tests fail deterministically** because of genuine emulator
+behavior differences. They fall into the six groups below.
+
+| Group | Root cause | Failing tests |
+|-------|-----------|---------------|
+| 1 | Aggregate over an inline/array subquery returns the wrong shape | 37 |
+| 2 | Object / complex-type equality returns no rows | ~16 |
+| 3 | Partial hierarchical-partition-key query returns too many documents | 21 |
+| 4 | `ORDER BY` by expression / composite / shadow property is not rejected | 14 |
+| 5 | Full-text default-language validation is not enforced | 4 |
+| 6 | `MaxItemCount` / paging is not honored | 2 |
+
+> Note: ~140 additional failures seen only in the full run (e.g. all of
+> `NorthwindWhereQueryCosmosTest`, `CosmosBulkEndToEndTest`) are **not** included
+> above — they pass 100% when their class is run in isolation. See
+> [Non-deterministic failures](#non-deterministic-failures-not-emulator-bugs).
+
+---
+
+## Group 1 — Aggregate over an inline/array subquery returns the wrong shape (37)
+
+**Symptom.** Aggregating (`MIN`/`MAX`/`SUM`/`AVERAGE`/`COUNT`) over a subquery that
+projects an inline array fails inside the Cosmos client with
+`System.ArgumentException: RewrittenAggregateProjections was not an array for a value
+aggregate query`, or — for an empty inline array — with a Newtonsoft
+`JsonSerializationException: Deserialized JSON type 'JValue' is not compatible with
+expected type 'JObject'`.
+
+**Failing classes:** `PrimitiveCollectionsQueryCosmosTest` (22), the 13
+`Cosmos*TypeTest` classes (1 each), `ComplexPropertiesCollectionCosmosTest` (1),
+`OwnedQueryCosmosTest` (1).
+
+**Exact failing SQL (captured from the test run):**
+
+```sql
+-- MIN over a two-element inline array
+SELECT VALUE c
+FROM root c
+WHERE ((SELECT VALUE MIN(a) FROM a IN (SELECT VALUE [30, c["Int"]])) = 30)
+
+-- COUNT over an empty inline array
+SELECT VALUE c
+FROM root c
+WHERE ((SELECT VALUE COUNT(1) FROM a IN (SELECT VALUE []) WHERE (a > c["Id"])) = 1)
+```
+
+**Non-EF repro:**
+
+```csharp
+var container = (await db.CreateContainerIfNotExistsAsync("g1", "/pk")).Container;
+await container.CreateItemAsync(new { id = "1", pk = "1", Int = 30 }, new PartitionKey("1"));
+
+// Fails: "RewrittenAggregateProjections was not an array for a value aggregate query"
+var min = container.GetItemQueryIterator<Dictionary<string, object>>(
+    "SELECT VALUE c FROM root c " +
+    "WHERE ((SELECT VALUE MIN(a) FROM a IN (SELECT VALUE [30, c[\"Int\"]])) = 30)");
+await min.ReadNextAsync();
+
+// Fails: JsonSerializationException JValue/JObject
+var cnt = container.GetItemQueryIterator<Dictionary<string, object>>(
+    "SELECT VALUE c FROM root c " +
+    "WHERE ((SELECT VALUE COUNT(1) FROM a IN (SELECT VALUE []) WHERE (a > c[\"id\"])) = 1)");
+await cnt.ReadNextAsync();
+```
+
+---
+
+## Group 2 — Object / complex-type equality returns no rows (~16)
+
+**Symptom.** A predicate that compares a whole embedded object (complex/owned type)
+to an object literal/parameter returns zero documents on the emulator even when a
+matching document exists. Tests assert `Expected: 1, Actual: 0` (or
+`Sequence contains no elements`).
+
+**Failing classes:** `ComplexPropertiesStructuralEqualityCosmosTest` (8),
+`ComplexTypeQueryCosmosTest` (4), `ComplexTypeToJsonPropertyQueryCosmosTest` (4),
+plus single cases in `AdHocComplexTypeQueryCosmosTest` and `ConfigPatternsCosmosTest`.
+
+**Exact failing SQL pattern (object compared inside `EXISTS`):**
+
+```sql
+SELECT VALUE c
+FROM root c
+WHERE EXISTS (
+    SELECT 1
+    FROM a IN c["AssociateCollection"]
+    WHERE ((a["Id"] > @get_Item_Id) AND (a = @entity_equality_get_Item)))
+```
+
+**Non-EF repro:**
+
+```csharp
+var container = (await db.CreateContainerIfNotExistsAsync("g2", "/pk")).Container;
+await container.CreateItemAsync(
+    new { id = "1", pk = "1", obj = new { X = 1, Y = 2 } }, new PartitionKey("1"));
+
+// Emulator returns 0 documents; a matching document exists.
+var q = new QueryDefinition("SELECT VALUE c FROM root c WHERE (c[\"obj\"] = @v)")
+    .WithParameter("@v", new { X = 1, Y = 2 });
+var it = container.GetItemQueryIterator<Dictionary<string, object>>(q);
+var page = await it.ReadNextAsync();   // page.Count == 0 (expected 1)
+```
+
+---
+
+## Group 3 — Partial hierarchical-partition-key query returns too many documents (21)
+
+**Symptom.** Querying with only a prefix of a hierarchical (multi-level) partition key
+returns documents from outside the specified prefix. Tests assert e.g.
+`Expected: 2, Actual: 4` or `Expected: 4, Actual: 8`.
+
+**Failing classes:** `ReadItemPartitionKeyQueryNoDiscriminatorInIdTest` (6),
+`ReadItemPartitionKeyQueryRootDiscriminatorInIdTest` (6),
+`ReadItemPartitionKeyQueryDiscriminatorInIdTest` (6),
+`ReadItemPartitionKeyQueryTest` (3).
+
+**Non-EF repro:**
+
+```csharp
+var props = new ContainerProperties("g3", new List<string> { "/tenant", "/user" });
+var container = (await db.CreateContainerIfNotExistsAsync(props)).Container;
+
+PartitionKey Full(string t, string u) =>
+    new PartitionKeyBuilder().Add(t).Add(u).Build();
+
+await container.CreateItemAsync(new { id = "1", tenant = "t1", user = "u1" }, Full("t1", "u1"));
+await container.CreateItemAsync(new { id = "2", tenant = "t1", user = "u2" }, Full("t1", "u2"));
+await container.CreateItemAsync(new { id = "3", tenant = "t2", user = "u1" }, Full("t2", "u1"));
+
+// Prefix partition key "t1" should match only the 2 t1 documents.
+var prefix = new PartitionKeyBuilder().Add("t1").Build();
+var it = container.GetItemQueryIterator<Dictionary<string, object>>(
+    "SELECT VALUE c FROM root c",
+    requestOptions: new QueryRequestOptions { PartitionKey = prefix });
+int count = 0;
+while (it.HasMoreResults) count += (await it.ReadNextAsync()).Count;
+// Emulator returns 3 (all documents); expected 2.
+```
+
+---
+
+## Group 4 — `ORDER BY` by expression / composite / shadow property is not rejected (14)
+
+**Symptom.** Real Azure Cosmos DB rejects `ORDER BY` over multiple properties without
+a composite index (and over certain computed expressions) with a `CosmosException`.
+The emulator accepts these queries, so tests that do
+`await Assert.ThrowsAsync<CosmosException>(...)` fail with
+`No exception was thrown`.
+
+**Failing classes:** `NorthwindMiscellaneousQueryCosmosTest` (8),
+`OwnedQueryCosmosTest` (5), `NorthwindSelectQueryCosmosTest` (1).
+
+**Non-EF repro:**
+
+```csharp
+var container = (await db.CreateContainerIfNotExistsAsync("g4", "/pk")).Container;
+await container.CreateItemAsync(new { id = "1", pk = "1", City = "B", Region = "X" }, new PartitionKey("1"));
+await container.CreateItemAsync(new { id = "2", pk = "1", City = "A", Region = "Y" }, new PartitionKey("1"));
+
+// Real Cosmos: BadRequest (composite index required). Emulator: succeeds.
+var it = container.GetItemQueryIterator<Dictionary<string, object>>(
+    "SELECT VALUE c FROM root c ORDER BY c[\"City\"], c[\"Region\"]");
+await it.ReadNextAsync();   // no exception on the emulator
+```
+
+---
+
+## Group 5 — Full-text default-language validation is not enforced (4)
+
+**Symptom.** Configuring an unsupported / mismatched full-text default language should
+produce a `CosmosException` at container-create / index time. The emulator accepts the
+configuration, so `Assert.ThrowsAsync<CosmosException>(...)` fails with
+`No exception was thrown`.
+
+**Failing class:** `AdHocFullTextSearchCosmosTest` (4).
+
+**Non-EF repro:**
+
+```csharp
+// Real Cosmos rejects an unsupported full-text language; emulator accepts it.
+var props = new ContainerProperties("g5", "/pk")
+{
+    FullTextPolicy = new FullTextPolicy
+    {
+        DefaultLanguage = "xx-XX", // unsupported language
+        FullTextPaths = new List<FullTextPath>
+        {
+            new FullTextPath { Path = "/text", Language = "xx-XX" }
+        }
+    }
+};
+// Expected to throw on the service; succeeds on the emulator.
+await db.CreateContainerIfNotExistsAsync(props);
+```
+
+---
+
+## Group 6 — `MaxItemCount` / paging is not honored (2)
+
+**Symptom.** A query executed with `MaxItemCount = 1` (EF `ToPageAsync`) should return a
+single-item page plus a continuation token. The emulator returns **all 91** matching
+documents in the first page. Tests assert `Assert.Single` and fail with
+`The collection contained 91 items`.
+
+**Failing class:** `NorthwindMiscellaneousQueryCosmosTest` (2).
+
+**Non-EF repro:**
+
+```csharp
+var container = (await db.CreateContainerIfNotExistsAsync("g6", "/pk")).Container;
+for (var i = 0; i < 91; i++)
+    await container.CreateItemAsync(new { id = $"c{i}", pk = "1" }, new PartitionKey("1"));
+
+var it = container.GetItemQueryIterator<Dictionary<string, object>>(
+    "SELECT VALUE c[\"id\"] FROM root c ORDER BY c[\"id\"]",
+    requestOptions: new QueryRequestOptions { MaxItemCount = 1 });
+var page = await it.ReadNextAsync();
+// Emulator: page.Count == 91 (expected 1, with a continuation token).
+```
+
+---
+
+## Non-deterministic failures (not emulator bugs)
+
+When the **entire** suite runs against one local emulator, ~140 extra tests fail with
+`CosmosException: NotFound (404) — Collection 'Customers'/'Employees' not found in
+database 'Northwind'` (122 in `NorthwindWhereQueryCosmosTest`) or transient
+write errors (`CosmosBulkEndToEndTest`, etc.). **Every one of these passes when its
+class is run in isolation** (e.g. `NorthwindWhereQueryCosmosTest`: 421/421 passing).
+These are caused by shared-container contention / throttling under concurrent load on a
+single emulator instance, not by a deterministic emulator behavior difference, so they
+are excluded from the groups above.
+
+## Pre-existing failures (present before this change)
+
+Running the suite at `HEAD~1` produces 37 failures that are unrelated to the
+Linux-emulator skips (they fail regardless of this change) because the emulator does
+not implement these index features:
+
+- **Full-text search (20)** — `FullTextScore' is not a recognized built-in function
+  name (SC2005)`.
+- **Vector search (14) and hybrid search (3)** — container creation fails with
+  `vectorIndexes[0]: only 'diskANN' type is currently supported` for `flat` /
+  `quantizedFlat` vector index types.
+
+These should be tracked against the emulator's full-text / vector feature support and
+are out of scope for the un-skipping change.