Add Linux workflow for testing with Cosmos DB emulator docker container (current and vnext)

[Copilot]

This PR implements a new GitHub Actions workflow to run Cosmos DB unit tests on Linux using the official Cosmos DB emulator docker containers, enabling Linux developers to perform local testing and development with both current and preview emulator versions.

Changes Made

New Workflow: .github/workflows/linux-cosmos-emulator-test.yml

Key Features:

• Platform: Runs on ubuntu-latest
• Matrix Strategy: Tests against both current stable and vnext preview emulator versions with protocol-specific configurations
• Container Images:
  • Current: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest (HTTPS)
  • Vnext: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview (HTTP)
• Port Configuration:
  • Endpoint port: 8081 (both versions)
  • Explorer port: 8081 (current), 1234 (vnext)
  • Additional ports: 10251-10254 for both versions
• Health Checks: Protocol-agnostic TCP port check that works for both HTTP and HTTPS emulators
• SSL Certificates: Conditional certificate import - only for current/HTTPS emulator, skipped for vnext HTTP-only emulator
• Environment Setup: Dynamic endpoint configuration based on protocol
  • Current: https://127.0.0.1:8081/
  • Vnext: http://127.0.0.1:8081/
• Failure Tolerance: Vnext emulator tests are allowed to fail without breaking the workflow

Protocol-Specific Handling:

• Current Emulator (HTTPS):
  • Uses HTTPS connections with certificate validation
  • Downloads and imports PEM certificate for secure connections
  • Health check waits for PEM endpoint availability
• Vnext Emulator (HTTP):
  • Uses HTTP-only connections (no HTTPS support with .NET)
  • Skips certificate download and import steps
  • Health check uses simple HTTP endpoint validation
  • Explorer accessible on separate port 1234

Workflow Triggers:

• Manual execution via workflow_dispatch
• Automatic on PRs affecting Cosmos extension code (Extensions/Cosmos/**)
• Self-triggering when workflow file changes

Test Execution:

• Targets specifically the Cosmos extension unit tests: ./Extensions/Cosmos/Cosmos.DataTransfer.CosmosExtension.UnitTests/
• Tests detect emulator availability through environment variables and skip gracefully if not present
• Includes connectivity validation and detailed logging
• Uses the same environment variable pattern as existing Windows emulator workflow

Performance Optimizations:

• Extended health check configuration with 120s start period and 20 retries
• 10-minute maximum wait time for emulator initialization
• Checks every 20 seconds for faster detection when ready

Benefits

1. Cross-Platform Development: Enables Linux developers to run full Cosmos DB integration tests locally
2. CI/CD Coverage: Verifies DMT functionality works correctly with Cosmos DB emulator on Linux
3. Preview Testing: Tests against vnext emulator in preview without breaking builds if unstable
4. Protocol Flexibility: Handles both HTTPS (current) and HTTP-only (vnext) emulator configurations
5. Consistency: Uses same test patterns and environment setup as existing Windows workflow
6. Minimal Impact: Focused only on Cosmos extension tests, doesn't affect other workflows

Testing

• ✅ Verified environment variable detection works correctly
• ✅ Confirmed tests behave appropriately (skip when no emulator, run when available)
• ✅ Validated YAML syntax and GitHub Actions compatibility
• ✅ Tested against existing repository patterns and conventions
• ✅ Added matrix strategy for testing both current and vnext emulator versions
• ✅ Verified HTTP-only configuration works for vnext emulator
• ✅ Confirmed certificate import skips correctly for vnext

The workflow complements the existing cosmos_db_emulator.yml (Windows) workflow and provides the same testing capabilities for Linux environments with additional coverage for preview emulator versions and protocol-specific handling.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[philnach]

@copilot , where do the tests target the emulator?

[philnach]

Change looks good and verified works by running the action.

[philnach]

@copilot, the vnext Cosmos DB emulator is in preview see: https://learn.microsoft.com/en-us/azure/cosmos-db/emulator-linux. Can you update the PR to also run against the vnet emulator, but don't fail if the tests don't past?

[Copilot]

@copilot , where do the tests target the emulator?

The tests target the emulator through environment variables:

1. Environment Variables: The workflow sets Cosmos_Endpoint=https://127.0.0.1:8081/ and Cosmos_Key to the default emulator key
2. Test Logic: In CosmosDataSourceExtensionTests.cs, the static constructor checks for these environment variables:
   • If missing → skips tests with "Connection details for Cosmos DB not found. Ignoring tests..."
   • If present → creates a CosmosClient using those connection details to connect to the emulator
3. Service Configuration: The workflow runs the emulator container with ports 8081, 10251-10254 exposed and waits for it to be ready before running tests

This follows the same pattern as the existing Windows workflow (cosmos_db_emulator.yml) which sets identical environment variables.

@copilot, the vnext Cosmos DB emulator is in preview see: https://learn.microsoft.com/en-us/azure/cosmos-db/emulator-linux. Can you update the PR to also run against the vnet emulator, but don't fail...

Updated the workflow to test against both emulator versions using a matrix strategy:

• Current: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest (must pass)
• Vnext: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview (allowed to fail)

The vnext tests use continue-on-error: true and fail-fast: false so they won't break the workflow if they fail. Each job shows which emulator version it's running against in the job name.

Commit: 1718e27

[philnach]

@copilot, the test is failing because there isn't enough wait time for the container. Can you take a look at the failure and fix it?

[Copilot]

@copilot, the test is failing because there isn't enough wait time for the container. Can you take a look at the failure and fix it?

I've identified and fixed the container initialization timeout issue. The vnext emulator container was taking longer than 5 minutes to start, causing the health checks to fail.

Changes made in commit 77045dc:

• Added --health-start-period=120s to give the container 2 minutes before health checks start counting as failures
• Increased health check retries from 10 to 20
• Decreased health check interval from 30s to 20s (checks more frequently)
• Increased the "Wait for Cosmos Emulator" timeout from 300s to 600s (10 minutes total wait time)

This gives the container up to ~8.7 minutes to become healthy (120s start period + 20 retries × 20s intervals), which should be sufficient for the vnext emulator to fully initialize.

[philnach]

@copilot, the vnext emulator doesn't easily support HTTPS with .net. Can you use http:// protocol when connecting to the vnext emulator. You also cannot download the pem file. This means don't attempt to try and pull down the pem url for the vnext emulator. Also, the vnext emulator has two port configurations the endpoint which is on port 8081 and the explorer port which is on 1234. Can you update the test to target the vnext emulator in this fashion? See this page for more information on the vnext emulator: https://learn.microsoft.com/en-us/azure/cosmos-db/emulator-linux

[Copilot]

@copilot, the vnext emulator doesn't easily support HTTPS with .net. Can you use http:// protocol when connecting to the vnext emulator. You also cannot download the pem file. This means don't attem...

Updated the workflow to support vnext emulator's HTTP-only configuration in commit 2314467:

Key changes:

• Vnext emulator now uses http://127.0.0.1:8081/ instead of HTTPS
• Added explorer port 1234 to service ports
• Certificate import step now skips for vnext (only runs for current/HTTPS emulator)
• Health check changed to simple HTTP endpoint check (works for both versions)
• Wait step conditionally checks HTTPS PEM endpoint for current or HTTP root for vnext
• Connectivity test uses appropriate protocol based on emulator version
• Matrix configuration differentiates protocol, ports, and HTTPS usage between versions