Skip to content

Add resource: github_repository_files#3456

Open
novucs wants to merge 1 commit into
integrations:mainfrom
novucs:add-resource-github-repository-files
Open

Add resource: github_repository_files#3456
novucs wants to merge 1 commit into
integrations:mainfrom
novucs:add-resource-github-repository-files

Conversation

@novucs
Copy link
Copy Markdown

@novucs novucs commented May 28, 2026

Resolves #2909


Before the change?

When parallel_requests = true is set on the provider, managing many files in a single repository via github_repository_file with for_each fails frequently with HTTP 409 errors:

Error: PUT https://api.github.com/repos/xxx/yyy/contents/tenants/test01/myfile.yaml: 409 is at 30e15e8485d2808d7f9cdf27cecbb145252e1427 but expected 46affa9e9f78ca99714ed0e12a9c44e94e1e9055 []
│
│   with github_repository_file.tenant["myfile"],
│   on fluxcd.tf line 204, in resource "github_repository_file" "tenant":
│  204: resource "github_repository_file" "tenant" {

Cause: each github_repository_file instance issues an independent PUT /contents/{path} (Contents API), which creates one commit per file. Under Terraform's parallelism, the second concurrent write sees a branch SHA that has already advanced past the SHA it captured, and GitHub rejects it via
optimistic-concurrency check. Setting parallel_requests = false serializes HTTP at the transport layer but is documented as experimental, slows down all provider work globally, and still produces N commits when the user really wanted one.

The underlying mismatch is semantic: users want one commit containing N files, but the resource model is one resource per file.

After the change?

Adds a new resource, github_repository_files, that manages a set of files within a branch and writes them all in a single commit per apply via the Git Data API:

  1. Git.GetRefGit.GetCommit to capture the base commit + tree.
  2. Git.CreateBlob for each file content (bounded parallelism).
  3. Git.CreateTree(baseTree, entries) — a partial tree merged onto the current base, so unmanaged files in the repo are preserved.
  4. Git.CreateCommit + Git.UpdateRef.
  5. On 409/422 from UpdateRef (branch advanced under us), re-read HEAD, rebase the change onto the new base, and retry. Bounded to 3 attempts with exponential backoff. Blobs are reused across retries because they're content-addressed.

Highlights:

  • One commit per apply, regardless of how many file {} blocks the resource owns. Removes the 409-conflict failure mode entirely for users converting for_each patterns over a single repo.
  • Adopt-and-overwrite by default — a batch resource is a desired-state declaration. No overwrite_on_create flag.
  • Files outside the declared file {} blocks are never touched — the merged-tree commit only adds/updates listed paths and deletes paths that were removed from the configuration.
  • Self-healing against concurrent writers via the rebase retry loop, so the issue reporter's parallel_requests = true use case works.
  • Existing github_repository_file is unchanged — users opt in by switching.

Example, applied directly to the reporter's config:

resource "github_repository_files" "tenant" {
  repository     = data.github_repository.cluster.name
  branch         = local.cluster_branch
  commit_message = "chore: sync tenants"

  dynamic "file" {
    for_each = { for ns in local.tenant_namespaces : ns.name => ns if ns.flux.enabled }
    content {
      path    = "tenants/${local.cluster_id}/${file.key}.yaml"
      content = yamlencode(file.value.flux)
    }
  }
}

Mirrors the conventions of the existing resource: repository_id + CustomizeDiff: diffRepository for repo-rename detection, computed ref / commit_sha / tree_sha / per-file sha, commit_author/commit_email pairing, deleteResourceOn404AndSwallow304OtherwiseReturnError for ETag-cached reads,
and handleArchivedRepoDelete for archived-repo destroys.

Docs include a "Differences from github_repository_file" section (intentionally-omitted attributes) and a "Migrating from github_repository_file" guide with the terraform state rm + terraform import workflow.

Pull request checklist

  • Schema migrations have been created if needed (example)
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been reviewed and added / updated if needed (for bug fixes / features)

Does this introduce a breaking change?

Please see our docs on breaking changes to help!

  • Yes
  • No

@github-actions
Copy link
Copy Markdown

👋 Hi! Thank you for this contribution! Just to let you know, our GitHub SDK team does a round of issue and PR reviews twice a week, every Monday and Friday! We have a process in place for prioritizing and responding to your input. Because you are a part of this community please feel free to comment, add to, or pick up any issues/PRs that are labeled with Status: Up for grabs. You & others like you are the reason all of this works! So thank you & happy coding! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[MAINT]: Improve UX for github_repository_file when parallel_requests = true

1 participant