Skip to content

FOUNDATIONS: Add matrix strategy and service-container components for acceptance-test workflows #175

Description

@SlimAhmad

Summary

Create the components that ADotNet is currently missing so that it can generate GitHub Actions
workflow YAML containing job-level strategy (matrix) and services (service containers).
These are required to generate acceptance-test workflows that run the same test suite across a build
matrix (e.g. sqlserver | postgres) with backing service containers.

The new components must be exposed through the existing GitHubPipelineBuilder / JobBuilder
fluent API (this is the convention used to build pipelines).

Hard requirement: the generated components MUST represent the full functionality offered by
GitHub Actions
for strategy and servicesnot just the subset shown in the reference
example below. Model every documented key (see the "Full GitHub surface" checklist), even where the
reference workflow does not use it.


Background

A downstream project needs an acceptance-test workflow shaped like this (abbreviated):

acceptance-tests:
  name: Acceptance Tests — ${{ matrix.provider }}
  runs-on: ubuntu-latest
  needs: unit-tests
  strategy:
    fail-fast: false
    matrix:
      include:
        - provider: sqlserver
          connection_string: >-
            Server=localhost,1433;Database=EventHighwayDB;User Id=sa;Password=***;TrustServerCertificate=True
        - provider: postgres
          connection_string: >-
            Host=localhost;Port=5432;Database=eventhighway;Username=postgres;Password=postgres
  services:
    sqlserver:
      image: mcr.microsoft.com/mssql/server:2022-latest
      env:
        SA_PASSWORD: "***"
        ACCEPT_EULA: "Y"
        MSSQL_PID: "Developer"
      ports:
        - "1433:1433"
      options: >-
        --health-cmd "..." --health-interval 10s --health-timeout 5s --health-retries 10
    postgres:
      image: postgres:17
      env:
        POSTGRES_PASSWORD: postgres
        POSTGRES_DB: eventhighway
        POSTGRES_USER: postgres
      ports:
        - "5432:5432"
      options: >-
        --health-cmd "pg_isready -U postgres" --health-interval 5s --health-timeout 3s --health-retries 10

ADotNet cannot currently produce the strategy (with include) or services sections.


Current state

Piece Today Gap
Strategy Exists — only matrix as Dictionary<string, List<string>> No include, exclude, fail-fast, max-parallel
Job.Strategy Wired (Order = 8) Depends on the limited Strategy above
services Missing entirely No Job.Services, no Service, no Credentials
Builder support JobBuilder has no strategy/service methods Need AddService(...) + strategy config on the existing builders

Scope — components to create

1. Models (ADotNet/Models/Pipelines/GithubPipelines/DotNets/)

  • Service — a service container definition.
  • Credentials — registry credentials for a service/container image.
  • Extend Strategy to add fail-fast, max-parallel, and matrix include / exclude.
  • (Recommended) a Matrix type to represent axis variables + include + exclude.
  • Add Services to Job as Dictionary<string, Service> (service id → definition).

2. Builders (ADotNet/Clients/Builders/)

Extend the existing GitHubPipelineBuilder / JobBuilder flow only — no new builder types
(no ServiceBuilder). Service and strategy configuration is added to the current builders:

  • JobBuilder.AddService(string id, Service service) — attach a service container to the job
    (the caller constructs the Service model; add convenience overloads if helpful).
  • JobBuilder strategy helpers — e.g. WithMatrix(...), AddMatrixInclude(...),
    WithFailFast(bool), WithMaxParallel(int).

3. Tests (AdoNet.Tests.Unit/)

  • Unit tests for the new models' YAML serialization.
  • Tests for the new GitHubPipelineBuilder / JobBuilder additions (service + strategy methods).
  • End-to-end GitHubPipelineBuilder test proving a full matrix + services workflow serializes
    to the expected YAML (see AdoNet.Tests.Unit/Clients/Builders/).

Full GitHub surface (must be represented — not limited to the example)

jobs.<job_id>.strategy

  • matrix — arbitrary axis variables: key: [values]
  • matrix.include — list of maps (combinations to add / extend, may add keys not in the axes)
  • matrix.exclude — list of maps (combinations to remove)
  • fail-fast — boolean (GitHub default is true)
  • max-parallel — integer

jobs.<job_id>.services.<service_id> (same schema as jobs.<job_id>.container)

  • image — string
  • credentials{ username, password }
  • env — map
  • ports — list of strings
  • volumes — list of strings ← missing from the reference example; still required
  • options — string

Because container and services.<id> share the identical schema, consider designing Service
so the same shape can be reused for a future job-level container component.


Reference component examples

These are illustrative of the shape/conventions expected (match existing YamlMember ordering and
OmitDefaults usage). Extend as needed to reach full GitHub coverage.

public class Service
{
    [YamlMember(Order = 0, Alias = "image", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public string Image { get; set; }

    [YamlMember(Order = 1, Alias = "credentials", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public Credentials Credentials { get; set; }

    [YamlMember(Order = 2, Alias = "env", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public Dictionary<string, string> Environment { get; set; }

    [YamlMember(Order = 3, Alias = "ports", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public List<string> Ports { get; set; }

    // Required for full GitHub coverage even though the reference workflow omits it:
    [YamlMember(Order = 4, Alias = "volumes", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public List<string> Volumes { get; set; }

    [YamlMember(Order = 5, Alias = "options", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public string Options { get; set; }
}
public class Credentials
{
    [YamlMember(Order = 0, Alias = "username", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public string Username { get; set; }

    [YamlMember(Order = 1, Alias = "password", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public string Password { get; set; }
}
// Extended Strategy — additive to the existing matrix support.
public class Strategy
{
    [YamlMember(Order = 0, Alias = "fail-fast", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public bool? FailFast { get; set; }   // nullable: GitHub defaults to true, so only emit when set

    [YamlMember(Order = 1, Alias = "max-parallel", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public int? MaxParallel { get; set; }

    [YamlMember(Order = 2, Alias = "matrix", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public Matrix Matrix { get; set; }
}
// Matrix — axis variables plus include/exclude combinations.
public class Matrix
{
    // Axis variables, e.g. { "provider": ["sqlserver", "postgres"] }.
    // NOTE: in YAML these keys must appear as SIBLINGS of include/exclude (see considerations).
    public Dictionary<string, List<string>> Variables { get; set; }

    [YamlMember(Alias = "include", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public List<Dictionary<string, string>> Include { get; set; }

    [YamlMember(Alias = "exclude", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
    public List<Dictionary<string, string>> Exclude { get; set; }
}
// Job — add the services map (service id -> definition).
[YamlMember(Order = 13, Alias = "services", DefaultValuesHandling = DefaultValuesHandling.OmitDefaults)]
public virtual Dictionary<string, Service> Services { get; set; }

Example builder usage (the API we want)

GitHubPipelineBuilder.CreateNewPipeline()
    .SetName("Build and Test")
    .OnPush("main")
    .AddJob("acceptance-tests", job => job
        .WithName("Acceptance Tests — ${{ matrix.provider }}")
        .RunsOn(BuildMachines.UbuntuLatest)
        .WithFailFast(false)
        .AddMatrixInclude(new() { ["provider"] = "sqlserver", ["connection_string"] = "..." })
        .AddMatrixInclude(new() { ["provider"] = "postgres",  ["connection_string"] = "..." })
        .AddService("sqlserver", new Service
        {
            Image = "mcr.microsoft.com/mssql/server:2022-latest",
            Environment = new() { ["ACCEPT_EULA"] = "Y" },
            Ports = new() { "1433:1433" },
            Options = "--health-cmd \"...\" --health-interval 10s"
        })
        .AddService("postgres", new Service
        {
            Image = "postgres:17",
            Environment = new() { ["POSTGRES_USER"] = "postgres" },
            Ports = new() { "5432:5432" }
        }))
    .SaveToFile(".github/workflows/build.yml");

Design considerations / open questions

  1. fail-fast default is true. The meaningful override is false. Use bool? (nullable) so an
    unset value is omitted and an explicit false is emitted — a non-nullable bool + OmitDefaults
    would silently drop false.
  2. Matrix serialization. Base axis variables are dynamic keys that must be emitted as siblings
    of include/exclude. YamlDotNet will not flatten a nested dictionary property automatically —
    a custom representation (e.g. build a single Dictionary<string, object> where values are either
    List<string> for axes or List<Dictionary<string,string>> for include/exclude) or a custom
    converter is likely needed.
  3. Backward compatibility. The existing Strategy.Matrix is Dictionary<string, List<string>>,
    which is public API. Changing its type is a breaking change — decide between (a) evolving Strategy
    additively while preserving the current property, or (b) a versioned successor consistent with the
    repo's V2/V3 component pattern.
  4. Shared shape with container. Design Service so it can be reused for a future job-level
    container component (identical GitHub schema).

Acceptance criteria

  • Service and Credentials models created, covering the full GitHub services.<id> schema
    (image, credentials, env, ports, volumes, options).
  • Strategy supports fail-fast, max-parallel, and matrix include / exclude (in addition to axes).
  • Job exposes a services map that serializes as service id -> definition.
  • Existing GitHubPipelineBuilder / JobBuilder extended with methods to add services and configure the strategy — no new builder types.
  • All new properties omit themselves from YAML when unset (OmitDefaults), consistent with existing models.
  • Unit tests: model serialization, the new builder methods, and an end-to-end GitHubPipelineBuilder test that reproduces the matrix + services workflow above.
  • Components represent the full GitHub feature set, not only the reference example.

Out of scope

  • Azure DevOps pipeline models.

Metadata

Metadata

Assignees

Labels

CODE RUBThe code rub category is for small changes (not for bug fixes)

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions