Skip to content

ArtifactService Twirp routes 404 instead of falling through to DEFAULT_ACTIONS_RESULTS_URL catch-all #237

@yonibaciu

Description

@yonibaciu

Summary

When a runner is patched (via falcondev-oss/runner or equivalent) so that CUSTOM_ACTIONS_RESULTS_URL sticks at job start, the runner sends both cache and artifact Twirp traffic to the cache server (because GitHub multiplexes them onto a single ACTIONS_RESULTS_URL). The cache server's routes/[...path].ts catch-all is supposed to proxy unknown paths to DEFAULT_ACTIONS_RESULTS_URL (results-receiver.actions.githubusercontent.com), but it does not fire for unmatched paths under routes/twirp/. Result: every actions/upload-artifact / actions/download-artifact call lands on the cache server's own 404 response, which the @actions/artifact client tries to JSON-parse and fails:

```
Failed to CreateArtifact: Failed to make request after 5 attempts:
Unexpected token '<', "<!DOCTYPE "... is not valid JSON
```

Confirmed on `:5`, `:9.4.7`, and current `main`.

Reproduction

  1. Run the cache server with default config (`DEFAULT_ACTIONS_RESULTS_URL` left at its default).
  2. Set `ACTIONS_RESULTS_URL` on a self-hosted runner pointing to the cache server (assume patched runner so the env var sticks).
  3. Run a workflow with `actions/upload-artifact@v4` (or `v6`).
  4. Upload fails as above.

Direct curl probe also reproduces the routing miss:

```
$ curl -i -X POST
-H 'Content-Type: application/json'
--data '{}'
http://cache-server:3000/twirp/github.actions.results.api.v1.ArtifactService/CreateArtifact

HTTP/1.1 404 Not Found
{"statusCode":404,"statusMessage":"Cannot find any route matching
/twirp/github.actions.results.api.v1.ArtifactService/CreateArtifact."}
```

The same path against a hypothetical `routes/twirp/github.actions.results.api.v1.ArtifactService/CreateArtifact.post.ts` would be routed, so the issue is specifically that nitro's route matching consumes the `/twirp/` segment with the existing `routes/twirp/github.actions.results.api.v1.CacheService/` directory and never falls back to `routes/[...path].ts` for sibling-but-unmatched paths.

Expected

Either (a) the root catch-all fires for any path that no specific route handles, or (b) an explicit catch-all under `routes/twirp/[...path].ts` proxies the same way the root one already does.

Proposed fix

Add `routes/twirp/[...path].ts` mirroring the root catch-all:

```typescript
import { env } from '/lib/env'
import { logger } from '
/lib/logger'

export default defineEventHandler(async (event) => {
logger.debug('proxying unknown twirp path', event.path, 'to', env.DEFAULT_ACTIONS_RESULTS_URL)
return proxyRequest(event, `${env.DEFAULT_ACTIONS_RESULTS_URL}${event.path}`)
})
```

(Or rework the root-level catch-all so nitro lets it match nested paths.)

Happy to send a PR if a maintainer signs off on the approach.

Workaround

For users blocked on this today: front the cache server with a small reverse proxy that explicitly routes `/twirp/.../ArtifactService/*` to `results-receiver.actions.githubusercontent.com` and everything else to the cache server.

Minimal nginx config:

```nginx
resolver 1.1.1.1 1.0.0.1 valid=300s ipv6=off;

upstream cache_server {
server cache-server.gha-cache.svc.cluster.local:3000;
}

server {
listen 3000;

location /twirp/github.actions.results.api.v1.ArtifactService/ {
set $upstream "results-receiver.actions.githubusercontent.com";
proxy_pass https://$upstream;
proxy_set_header Host $upstream;
proxy_ssl_server_name on;
proxy_ssl_protocols TLSv1.2 TLSv1.3;
proxy_http_version 1.1;
proxy_pass_request_headers on;
proxy_read_timeout 600s;
proxy_send_timeout 600s;
proxy_request_buffering off;
client_max_body_size 0;
}

location / {
proxy_pass http://cache_server;
proxy_http_version 1.1;
proxy_pass_request_headers on;
proxy_read_timeout 600s;
proxy_send_timeout 600s;
proxy_request_buffering off;
client_max_body_size 0;
}
}
```

We've been running this in production (self-hosted ARC cluster, three Ubuntu nodes, kubernetes container mode) since this morning — `actions/upload-artifact` round-trips fully through the proxy with healthy 200s from results-receiver, cache restore/save still hit the in-cluster falcondev server at LAN speed (~500 MB/s for our pnpm store).

Environment

  • `ghcr.io/falcondev-oss/github-actions-cache-server:9.4.7` (also reproduces on `:5`)
  • `ghcr.io/falcondev-oss/actions-runner:2.334.0`
  • actions-runner-controller chart `gha-runner-scale-set` v0.14.1, kubernetes container mode
  • `@actions/artifact` versions seen: 2.3.2, 4.0.0, 5.0.1 — all hit the same path

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions