Skip to content

nixos/forgejo.runner: initialize module#496325

Open
adamcstephens wants to merge 3 commits into
NixOS:masterfrom
adamcstephens:push-kpxuoykptuty
Open

nixos/forgejo.runner: initialize module#496325
adamcstephens wants to merge 3 commits into
NixOS:masterfrom
adamcstephens:push-kpxuoykptuty

Conversation

@adamcstephens
Copy link
Copy Markdown
Contributor

@adamcstephens adamcstephens commented Mar 3, 2026

I couldn't remember exactly where we left the discussion, and couldn't find the context looking at github/matrix, so I went ahead with an initial implementation. This is based off the gitea-actions-runner, but using a template service and applying maximum hardening. It's possible this is too much hardening and will need to be tuned back, but the basic tests at least are passing.

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

@tebriel
Copy link
Copy Markdown
Contributor

tebriel commented Mar 3, 2026

this is very exciting, I'll attempt to replace my manual implementation with this tonight.

@nixpkgs-ci nixpkgs-ci Bot added 2.status: merge conflict This PR has merge conflicts with the target branch 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog This PR adds or changes release notes 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: documentation This PR adds or changes documentation labels Mar 3, 2026
Comment thread nixos/modules/services/continuous-integration/forgejo-runner.nix Outdated
@tebriel
Copy link
Copy Markdown
Contributor

tebriel commented Mar 5, 2026

Ok I've moved my config to this module in https://frodux.dev/Frodux/nixos/pulls/772, it has properly registered, picked up a job, and is working on it. First test with docker as a runner working as expected.

Comment thread nixos/modules/services/continuous-integration/forgejo-runner.nix Outdated
@adamcstephens adamcstephens force-pushed the push-kpxuoykptuty branch 2 times, most recently from 615d282 to 41f14ec Compare March 5, 2026 21:09
@nixpkgs-ci nixpkgs-ci Bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 5, 2026
Comment thread nixos/modules/services/continuous-integration/forgejo-runner.nix Outdated
@ofborg ofborg Bot added the ofborg-internal-error Ofborg encountered an error label Mar 5, 2026
@adamcstephens adamcstephens force-pushed the push-kpxuoykptuty branch 2 times, most recently from a0b1988 to dc3b027 Compare March 9, 2026 15:48
@adamcstephens
Copy link
Copy Markdown
Contributor Author

This is looking pretty good to me here in nixpkgs. I still need to test this with my local exec runner to see if the hardening is ok.

@ofborg ofborg Bot removed the ofborg-internal-error Ofborg encountered an error label Mar 9, 2026
Comment thread nixos/modules/services/continuous-integration/forgejo-runner.nix Outdated
Copy link
Copy Markdown
Contributor

@tebriel tebriel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still only running it in docker mode but the update to use LoadCredential is working as expected.

@nixpkgs-ci nixpkgs-ci Bot added the 12.approvals: 1 This PR was reviewed and approved by one person. label Mar 11, 2026
@adamcstephens
Copy link
Copy Markdown
Contributor Author

Hmm, we may want to rethink this structure slightly. Runner 12.7.0 added multiple server connections https://forgejo.org/2026-02-monthly-report/#forgejo-runner-v1270

@emilylange
Copy link
Copy Markdown
Member

Hmm, we may want to rethink this structure slightly. Runner 12.7.0 added multiple server connections forgejo.org/2026-02-monthly-report#forgejo-runner-v1270

Yes! This is something I wanted to point out ever since we merged #492254. Unfortunately, I had no time to review this PR here yet. But there are a lot of things we can and should absolutely leverage and do differently.

I am hoping I have some time for this within the next two weeks.

@adamcstephens
Copy link
Copy Markdown
Contributor Author

I'd definitely appreciate your feedback @emilylange. I'm not in any hurry so when you get a chance, let me know your thoughts. I'd like to get this in before 26.05 but that's still months away.

@adamcstephens
Copy link
Copy Markdown
Contributor Author

adamcstephens commented Mar 12, 2026

Another thing to consider would be adding support for ephemeral runners, similar to services.github-runners..ephemeral . They use systemd to wipe the runner directory and re-register on start. The forgejo-runner does support ephmeral runners too. But I'd need to do some research to make sure this is actually valuable. It does add time for runner cycling in my experience with the github runner.

Comment thread nixos/modules/services/continuous-integration/forgejo-runner.nix
@adamcstephens adamcstephens marked this pull request as ready for review April 7, 2026 14:22
@adamcstephens adamcstephens marked this pull request as draft April 7, 2026 15:19
@adamcstephens
Copy link
Copy Markdown
Contributor Author

oops, sorry for the noise. forgot about the multiple connections.

@adamcstephens adamcstephens force-pushed the push-kpxuoykptuty branch 3 times, most recently from 982f04a to 75e99ac Compare May 13, 2026 03:59
@adamcstephens adamcstephens marked this pull request as ready for review May 13, 2026 04:02
Comment on lines +105 to +110
credentials = lib.mkOption {
type = attrsOf lib.types.path;
default = { };
example = {
WORKER1_TOKEN = "/run/keys/worker1";
};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can we use these credentials properly?

I've tried this example for the first time, my full config:

{
  services.forgejo.runner.instances.test = {
    enable = true;
    url = "https://git.tchfoo.com";
    labels = [ "native:host" ];
    settings.server.connections.gepbird = {
      uuid = "634c3257-0e5a-4d35-b5bb-8620f1b70212";
      token_url = "file:$WORKER1_TOKEN";
    };
    credentials.WORKER1_TOKEN = "/run/keys/worker1";
  };
}

System log:

forgejo-runner[1849855]: Error: invalid configuration: invalid `server` settings: connection "gepbird" is invalid: invalid `token_url`: cannot read secret "file:$WORKER1_TOKEN": open : no such file or directory

According to this comment in the example config, only $CREDENTIALS_DIRECTORY will be resolved. After modifying this part of the configuration:

{
  services.forgejo.runner.instances.test = {
    settings.server.connections.gepbird.token_url = "file:$CREDENTIALS_DIRECTORY/worker1";
    credentials.CREDENTIALS_DIRECTORY = "/run/keys";
  };
}

It still fails, but the variable is resolved:

forgejo-runner[1867313]: Error: invalid configuration: invalid `server` settings: connection "gepbird" is invalid: invalid `token_url`: cannot read secret "file:/run/credentials/forgejo-runner@test.service/worker1": open /run/credentials/forgejo-runner@test.service/worker1: no such file or directory

This configuration works (if the permissions are broad enough):

{
  services.forgejo.runner.instances.test = {
    settings.server.connections.gepbird.token_url = "file:/run/keys/worker1";
    # no credentials option set
  };
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CREDENTIALS_DIRECTORY is a built in env var for LoadCredential. You should probably choose a different attr name.

{
  services.forgejo.runner.instances.test = {
    settings.server.connections.gepbird.token_url = "file:$CREDENTIALS_DIRECTORY/my_token"; # my_token needs to match the credential *name*
    credentials.my_token = "/run/keys/my_token";
  };
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Works well, I think an example for the settings option would help many people :)

@gepbird
Copy link
Copy Markdown
Contributor

gepbird commented May 14, 2026

There seems to be a problem with escaping dashes in the service name.

Creating a test instance works well:

$ systemctl list-units | grep forgejo-runner
  forgejo-runner@test.service             loaded active running   Forgejo Actions Runner (test)
  system-forgejo\x2drunner.slice          loaded active active    Slice /system/forgejo-runner

But creating test-test instance makes the service fail:

$ systemctl list-units | grep forgejo-runner
● forgejo-runner@test\x2dtest.service     loaded activating auto-restart Forgejo Actions Runner (test-test)
  system-forgejo\x2drunner.slice          loaded active     active       Slice /system/forgejo-runner
$ systemctl status forgejo-runner@test\x2dtest.service     
○ forgejo-runner@testx2dtest.service - Forgejo Actions Runner (testx2dtest)
     Loaded: bad-setting (Reason: Unit forgejo-runner@testx2dtest.service has a bad unit file setting.)
     Active: inactive (dead)

May 14 21:24:49 geptop-xmg systemd[1]: forgejo-runner@testx2dtest.service: Service has no ExecStart=, ExecStop=, or SuccessAction=. Refusing.

@adamcstephens
Copy link
Copy Markdown
Contributor Author

$ systemctl status forgejo-runner@test\x2dtest.service     
○ forgejo-runner@testx2dtest.service - Forgejo Actions Runner (testx2dtest)

It appears your shell may have removed the backslash here?

@gepbird
Copy link
Copy Markdown
Contributor

gepbird commented May 14, 2026

It appears your shell may have removed the backslash here?

You're right, here are the correct error logs:

$ systemctl status 'forgejo-runner@test\x2dtest.service'
● forgejo-runner@test\x2dtest.service - Forgejo Actions Runner (test-test)
     Loaded: loaded (/etc/systemd/system/forgejo-runner@.service; enabled; preset: ignored)
    Drop-In: /nix/store/92p4ad7mrdgms6zfkkmaqxzhp5jl1nc1-system-units/forgejo-runner@test\x2dtest.service.d
             └─overrides.conf
     Active: activating (auto-restart) (Result: resources) since Thu 2026-05-14 21:35:06 CEST; 1s ago
 Invocation: ad28e3541489454398b200c28e54d792
         IP: 0B in, 0B out
         IO: 0B read, 0B written
   Mem peak: 0B
        CPU: 0
$ journalctl -eu 'forgejo-runner@test\x2dtest.service' | tail
May 14 21:35:30 geptop-xmg systemd[1]: forgejo-runner@test\x2dtest.service: Failed with result 'resources'.
May 14 21:35:30 geptop-xmg systemd[1]: Failed to start Forgejo Actions Runner (test-test).
May 14 21:35:32 geptop-xmg systemd[1]: forgejo-runner@test\x2dtest.service: Scheduled restart job, restart counter is at 321.
May 14 21:35:32 geptop-xmg systemd[1]: forgejo-runner@test\x2dtest.service: Failed to spawn 'start' task: Invalid argument
May 14 21:35:32 geptop-xmg systemd[1]: forgejo-runner@test\x2dtest.service: Failed with result 'resources'.
May 14 21:35:32 geptop-xmg systemd[1]: Failed to start Forgejo Actions Runner (test-test).
May 14 21:35:34 geptop-xmg systemd[1]: forgejo-runner@test\x2dtest.service: Scheduled restart job, restart counter is at 322.
May 14 21:35:34 geptop-xmg systemd[1]: forgejo-runner@test\x2dtest.service: Failed to spawn 'start' task: Invalid argument
May 14 21:35:34 geptop-xmg systemd[1]: forgejo-runner@test\x2dtest.service: Failed with result 'resources'.
May 14 21:35:34 geptop-xmg systemd[1]: Failed to start Forgejo Actions Runner (test-test).

(I was expecting a "Unit X.service could not be found." error if the service name doesn't exist, but turns out @ services behave differently)

@mweinelt mweinelt added this to the 26.05 milestone May 14, 2026
@icewind1991
Copy link
Copy Markdown
Contributor

icewind1991 commented May 16, 2026

It seems required to specify the runner labels under settings.runner.labels, which is unexpected given that the top level labels setting exists.

Looking at the code I expect this is because of switching from registrationTokenFile to settings.server.connections, when the registrationTokenFile is used, the explicit forgejo-runner register step does use the top level labels.

Simply defaulting settings.runner.labels to labels seems to be sufficient.

Works great otherwise

@adamcstephens
Copy link
Copy Markdown
Contributor Author

Yes, there are probably some more assertions we should add. There’s really two mutually exclusive ways of configuring, url/registrationTokenFile/labels and server.connections. At a minimum, I think I’ll add one to force users to pick one path or the other.

@tebriel
Copy link
Copy Markdown
Contributor

tebriel commented May 17, 2026

Also note: if you use server.connections (and the hard to discover token_url) with the .credentials [thank you for including the loadcredential bit made my life easier] you need to put labels in both the labels and settings.runner.labels otherwise the runner doesn't get access to the docker socket.

@adamcstephens
Copy link
Copy Markdown
Contributor Author

you need to put labels in both the labels and settings.runner.labels otherwise the runner doesn't get access to the docker socket.

that sounds like a bug, the labels should get merged for the runner instances logic around this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog This PR adds or changes release notes 8.has: documentation This PR adds or changes documentation 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 12.approvals: 1 This PR was reviewed and approved by one person.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants