Bug | repo-server-api fails to render multi-source apps with external $ref sources when primary source matches --repo

[Amoolaa]

Describe the bug

(Apologies for wordy title)

For context, we have some Applications targetRevisions pointing to main for our internal helm charts in dev for testing. When we raise changes to the helm charts I'd like to preview those changes across dev to get an idea on what the blast radius of those changes are.

When using --render-method repo-server-api with --redirect-target-revisions, multi-source Applications where the primary source (path:) matches --repo but a $ref source points to a different repo fail to render.

Expected behavior

When any ref source's repoURL does not match --repo, fall back to the remote RPC path (returning streamDir="") and populate request.RefSources with credentials for all ref sources. The repo server then git-fetches the primary source at source.TargetRevision and resolves all $ref sources.

Standard output (with --debug flag)

DBG Rendering application source via repo server (App: "my-app [t|...]") (multiSource: true) (remoteChart: false) (sourceIndex: 0) (streamDir: /var/folders/.../argocd-diff-preview-XXXXXXXXXX)
DBG Compressing application directory for repo server (app: my-app) (dir: /var/folders/.../argocd-diff-preview-XXXXXXXXXX)
DBG Opening GenerateManifestWithFiles stream (app: my-app)
ERR ❌ Failed to render application via repo server: error="failed to render app my-app [...]:
repo server returned error for content source 0: failed to receive manifest response:
rpc error: code = Unknown desc = failed to execute helm template command:
helm template . --values /tmp/.../.refs/helm-values-repo/values.yaml ...
Error: open /tmp/.../.refs/helm-values-repo/values.yaml: no such file or directory"

Application causing problems (if applicable)

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  namespace: argocd
spec:
  sources:
    - repoURL: https://github.com/org/helm-charts.git  # matches --repo
      path: charts/my-chart
      targetRevision: main
      helm:
        valueFiles:
          - $helm-values-repo/values.yaml
    - repoURL: https://github.com/org/app-values.git  # external ref
      targetRevision: main
      ref: helm-values-repo

Parameters

✨ Running v0.2.6 with:
✨ - using cluster with Argo CD pre-installed
✨ - render-method: repo-server-api
✨ - base-branch: main
✨ - target-branch: refs/pull/1/merge
✨ - argocd-namespace: argocd
✨ - repo: org/helm-charts
✨ - redirect-target-revisions: [main]
✨ - output-app-manifests: true

Your pipeline (if applicable)

Root cause

(Disclaimer: I have used an LLM to guide me to this conclusion)

In pkg/reposerverextract/extract.go, buildManifestRequestForSource reaches the streaming slow path when the primary source matches --repo and ref sources are present. It copies each ref source from branchFolder:

srcRefDir := filepath.Join(branchFolder, ref.Path)
if ref.Path == "" {
    // Ref-only source with no path points at the repo root.
    srcRefDir = branchFolder   // ← wrong when ref.RepoURL is a foreign repo
}
copyDir(srcRefDir, refDir)

There is no check for whether ref.RepoURL matches prRepo. When the ref source is external (ref.Path == ""), it silently copies branchFolder root instead of the actual ref repo content.

Proposed fix

Before creating the tempDir, check whether any ref source is external. If so, fall back to the remote RPC path with RefSources populated.

This is identical in structure to the fix in PR #359 (lines 594-615).

I have tested this locally and it works, so happy to raise a PR.

There is an easy workaround, which is to set --repo to the Applications repo and patch the targetRevisions in the target with the PR ref of the helm chart change. Perhaps this project isn't intended for previewing changes/cases like this, which I'm ok with.

[dag-andersen]

Hi @Amoolaa
Thanks for the detailed report! 🚀

I was able to confirm this is a valid bug. The repo-server-api path handled cross-repo primary sources, but it did not check whether $ref sources pointed to a different repo. In that case, the local streaming path could copy the PR repo root into .refs/<ref> instead of letting repo-server fetch the external ref repo.

I opened a fix here: #428

The fix makes multi-source apps use the remote repo-server RPC path when any ref source is external, and populates RefSources so repo-server can resolve the $ref value files itself. I also added a regression test for the case you reported.

It will be part of the next release ✌🏻

[Amoolaa]

wow that was quick! cheers

[noamyehudai]

hey @dag-andersen , when will the next version will be released?

[noamyehudai]

Hi @dag-andersen, thanks for shipping PR #428. We tested v0.2.7 against our setup with --render-method repo-server-api and the multi-source slow-path fix is not activating for us due to a substring false-positive in repoURLContains.

Setup

• App repo (--repo): org/helm-charts (chart code, ApplicationSets, env values).
• Ref repo: org/helm-charts-deploy (only imageTag.yaml files, separate repo, also private).
• Apps are multi-source. The chart source has path: and uses $ref/.../imageTag.yaml for the image tag value file.
• The ref repo's name shares a prefix with --repo (helm-charts vs helm-charts-deploy). This is what triggers the bug.

The bug

PR #428's slow-path is gated by:

// extract.go
if prRepo != "" && (!repoURLContains(primarySource.RepoURL, prRepo) || hasExternalRefSource(refSources, prRepo)) {
    // Slow path: remote RPC (the fix in PR #428)
}

// repocreds.go
func repoURLContains(repoURL, substr string) bool {
    return strings.Contains(normalizeRepoURL(repoURL), normalizeRepoURL(substr))
}

With our repos:

• --repo normalized: github.com/org/helm-charts
• Ref source URL normalized: github.com/org/helm-charts-deploy
• strings.Contains("github.com/org/helm-charts-deploy", "github.com/org/helm-charts") returns true because helm-charts is a prefix of helm-charts-deploy.

So hasExternalRefSource returns false for our genuinely-external ref source. The condition is not met, the remote-RPC slow path never activates, and we fall through to the local-stream path which only ships the chart's own subdirectory. Sibling charts referenced by file://../common_service Helm dependencies are not in that tarball, and helm dependency build fails inside argocd-repo-server:

rpc error: code = Unknown desc = error building helm chart dependencies:
  failed to build helm dependencies: ...
  Error: directory ../common_service not found

Suggested fix

Match full owner/repo segments instead of substring. Something like:

func repoURLEqual(a, b string) bool {
    return splitRepoURL(normalizeRepoURL(a)) == splitRepoURL(normalizeRepoURL(b))
}

That keeps the comparison provider-agnostic (still works against GitHub, GitLab, Bitbucket, self-hosted, etc.) but disambiguates org/foo from org/foo-deploy. The substring check was probably chosen for that provider-agnosticism (the comment in repocreds.go says so), but it produces false positives whenever --repo's name is a substring of any ref source's repo name.

Minimal repro

prRepo: org/helm-charts
ref source URL: https://github.com/org/helm-charts-deploy.git
expected: hasExternalRefSource = true
actual:   hasExternalRefSource = false

This will hit anyone whose --repo name is a prefix of any ref-source repo name. foo / foo-deploy, helm-charts / helm-charts-staging, monorepo / monorepo-values, etc.

Happy to test against a fix branch, or to open a PR if useful.

[noamyehudai]

hey @dag-andersen did you manage to have a look at the previous comment by any chance?

[dag-andersen]

Hi @noamyehudai! - thank for the detailed write-up! 🚀

I looked into this again, including the history around this behavior. The tricky part is that the current substring matching is not just an accidental implementation detail - some users rely on it today.
There are older discussions, for example #205, that explain why --repo matching ended up being loose. It helps with cases like optional .git suffixes, different URL formats, and templated Argo CD repo URLs.

That said, I agree that strict matching is the right direction going forward. --repo org/helm-charts should not accidentally match org/helm-charts-deploy.
The reason I cannot just apply the suggested fix directly is that it would be a breaking change for the existing use cases that depend on relaxed matching. So since your comment, I have been working on a branch that does two things together:

1. Make --repo matching stricter, so it matches complete repo path segments instead of plain substrings.
2. Add a new --repo-regex option for users who need the current more flexible/relaxed matching behavior, including templated repo URLs.

The idea is that --repo becomes the safer/default option, while --repo-regex is the explicit escape hatch for more complex setups.

That said, I will not have time to look further into it until the weekend. I mainly develop this project in my free time on weekends.

[noamyehudai]

@dag-andersen Thank you so much!
would love to help if needed 😄

[noamyehudai]

@dag-andersen hey, thanks for the quick response, I saw you merged a fix, when will it be released? 🤔