Skip to content
This repository was archived by the owner on Jul 10, 2025. It is now read-only.

Commit af9d82e

Browse files
authored
docs: README.md improvements (#116)
* docs: update readme to reflect updatated quota name form 'Low Priority' to 'Spot' * docs(readme): expand architecture details and component docs * Added “How It Works” section outlining event flow from GitHub to VM teardown * Documented roles of Service Bus, App Configuration, both Key Vaults, and Shared Image Gallery * Clarified Runner-Controller responsibilities and queue consumption behavior Closes #109 * docs: Add information notes in the docs for the different azure keyvault authentication methods depending RBAC vs Access Policy. Also includes a simple example to help get users started.
1 parent bd1a9d3 commit af9d82e

1 file changed

Lines changed: 131 additions & 17 deletions

File tree

README.md

Lines changed: 131 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,19 @@ This project includes all necessary components to spin up the infrastructure for
1616
## Architecture Diagram
1717
![Terraform Azure GitHub Runners](https://user-images.githubusercontent.com/100593043/194669700-4cd851ab-b047-4dd4-87bd-81cb4e572e24.png)
1818

19+
### How It Works
20+
1. **GitHub Actions event ➜ Event-Handler** – A `workflow_job` event is emitted by GitHub Enterprise/Cloud and forwarded by a GitHub App to the Event-Handler Azure Function.
21+
2. **Event-Handler ➜ Service Bus** – The function validates the request, filters on runner labels, and publishes a message to an Azure Service Bus queue.
22+
3. **Service Bus ➜ Runner-Controller** – The Runner-Controller Azure App Service continuously listens to the queue. Based on settings stored in Azure App Configuration it decides whether to spin up a new runner or rely on the warm-pool.
23+
4. **Runner-Controller ➜ Azure VM** – When a new runner is required the controller:
24+
• Fetches the latest Packer-built image from an Azure Shared Image Gallery.
25+
• Retrieves a one-time registration token from GitHub, storing it in the Registration Key Vault.
26+
• Creates a spot/on-demand VM, injecting the token via cloud-init so the VM registers itself with GitHub on first boot.
27+
5. **Runner lifecycle** – The VM processes exactly one job, then a shutdown hook notifies the controller which de-registers the runner and deletes the VM, keeping the warm-pool at the desired size.
28+
6. **Secrets & config** – GitHub App credentials and webhook secret live in the App Key Vault; operational parameters are stored in Azure App Configuration. All resources use Managed Identities for least-privilege access.
29+
7. **Provisioning** – This entire architecture is provisioned reproducibly by the Terraform module contained in this repository.
30+
31+
1932
## Components
2033

2134
### Terraform Module
@@ -25,15 +38,31 @@ This [Terraform](https://www.terraform.io/) module generates the infrastructure
2538
The event-handler will receive messages from the GitHub App during workflow run events. It will act as a filter to ensure they are from GitHub with labels that match what is provided in the module.
2639

2740
### Runner-Controller (Azure App Service)
28-
This application will act as the controller for the warm pool and ensure that the pool size adheres to the parameters specified in the Terraform module. It will consume events from the queue as necessary to create VMs and ensure a healthy number of VMs are always ready to process new workflow jobs.
41+
This application will act as the controller for the warm pool and ensure that the pool size adheres to the parameters specified in the Terraform module. It consumes Service Bus messages to create or delete VMs so that a healthy number of runners are always ready to process workflow jobs.
42+
43+
### Service Bus (Azure Service Bus)
44+
A reliable message queue that buffers `workflow_job` events between the Event-Handler and Runner-Controller, ensuring no job is lost and enabling smooth scale-out.
45+
46+
### App Configuration (Azure App Configuration)
47+
Central store for runtime settings such as warm-pool size, VM SKU, image version, and spot/on-demand preferences. Runner-Controller reads these values on startup and on a configurable interval.
48+
49+
### App Key Vault (Azure Key Vault)
50+
Holds long-lived secrets used by the Event-Handler and Runner-Controller (GitHub App private key, webhook secret, etc.). Accessed via Managed Identity with the minimum required permissions.
51+
52+
### Registration Key Vault (Azure Key Vault)
53+
Stores the short-lived registration tokens generated by Runner-Controller. Each token is valid for a single VM and is deleted once the runner registers with GitHub.
54+
55+
### Shared Image Gallery
56+
Contains HashiCorp Packer-built VM images pre-loaded with the toolchain your organization needs. Runner-Controller always provisions the latest image version.
57+
2958

3059
## Getting Started
3160

3261
### Pre-requisites
3362
- GitHub App for Organization (owner access)
3463
- Azure
3564
- Subscription
36-
- *Note: Subscription quota for "Total Regional Low-priority vCPUs" should be increased to allow multiple spot instances*
65+
- *Note: Subscription quota for "Total Regional Spot vCPUs" should be increased to allow multiple spot instances*
3766
- Resource Group
3867
- Subnet with internet access
3968
- KeyVault for GitHub App Credential
@@ -71,11 +100,11 @@ The GitHub App serves as the foundation for sending webhook events to App A and
71100
| Homepage URL | {insert-any-url} |
72101
| Webhook Active | False |
73102
| Webhook URL | |
74-
| Subscribe to events | Workflow job |
103+
| Subscribe to events* | Workflow job |
75104
| Where can this GitHub App be installed? | Only on this account |
76105

77106
*Note: You will need one GitHub App per org. Allowing installation to "Any account" makes it difficult to change access if installed on orgs outside your control.
78-
107+
*Note: Initially the webhook is disabled, but will be enabled in the next step. You will only see 'Subscribed to events' after the webhook is enabled.
79108
#### **Add secrets to Azure KeyVault**
80109

81110
(optional) Set Key Vault name variable:
@@ -108,34 +137,119 @@ Webhook Secret:
108137
az keyvault secret set --name github-webhook-secret --value $(uuidgen) --vault-name $KEYVAULT_NAME
109138
```
110139

111-
*Note: The private key must be added via the [AZ CLI](https://learn.microsoft.com/en-us/cli/azure/), all other secrets can be added manually via the portal if you choose to do so.
140+
*Note: The private key must be added via the [AZ CLI](https://learn.microsoft.com/en-us/cli/azure/), all other secrets can be added manually via the portal if you choose to do so.*
141+
142+
> **Key Vault authorization models**
143+
>
144+
> • If your Key Vault was created with **RBAC authorization** (`enableRbacAuthorization = true` or `az keyvault create --enable-rbac-authorization true`), you must grant the **Key Vault Secrets Officer** (or Administrator) role to **both** the Event-Handler Function **and** the Runner-Controller App Service managed identities. Terraform cannot assign this role unless you opt into the upcoming RBAC support variable.
145+
>
146+
> • If your vault uses the older **access-policy model** (`enableRbacAuthorization = false`, the module default), Terraform will automatically create an access policy that lets the Function and Controller read secrets.
147+
>
148+
> **Default behaviour:** The Azure portal and recent versions of the Azure CLI (*az* ≥ 2.51) create Key Vaults with **RBAC enabled by default**. In contrast, the Terraform `azurerm_key_vault` resource keeps **RBAC disabled by default**—you must set `enable_rbac_authorization = true` to opt-in. Keep this in mind when mixing manual and IaC-provisioned environments.
149+
112150

113151
### Setup Terraform Module
114152

115-
Consume this `azure_github_runner` module with inputs required for your GitHub Enterprise Cloud or GitHub Enterprise server configuration. Example of how to consume the module are coming soon.
153+
Below is a **minimal working example** you can copy-paste into a new `main.tf`. Replace the placeholder values with your own IDs.
154+
155+
```hcl
156+
terraform {
157+
required_providers {
158+
azurerm = {
159+
source = "hashicorp/azurerm"
160+
version = ">= 3.100"
161+
}
162+
azuread = {
163+
source = "hashicorp/azuread"
164+
version = ">= 2.45"
165+
}
166+
}
167+
}
168+
169+
provider "azurerm" {
170+
features {}
171+
tenant_id = "00000000-0000-0000-0000-000000000000" # Azure AD tenant
172+
subscription_id = "11111111-1111-1111-1111-111111111111" # Azure subscription
173+
}
174+
175+
provider "azuread" {}
176+
177+
module "github_runners" {
178+
source = "git::https://github.com/liatrio/terraform-azure-github-runner.git?ref=vX.Y.Z" # pin a release tag
179+
180+
# ---- Azure settings ----
181+
azure_tenant_id = "00000000-0000-0000-0000-000000000000"
182+
azure_subscription_id = "11111111-1111-1111-1111-111111111111"
183+
azure_resource_group_name = "rg-github-runners"
184+
azure_resource_group_location = "eastus"
185+
azure_subnet_id = "/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/vnet-github-runners/subnets/subnet-runners"
186+
187+
# ---- GitHub App details ----
188+
github_organization = "my-org"
189+
github_app_id = "123456"
190+
github_client_id = "Iv1.abcdef123456"
191+
github_installation_id = "7891011"
192+
193+
# ---- Key Vault secret IDs (Key Vault *ID* not value) ----
194+
azure_secrets_key_vault_resource_id = "/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.KeyVault/vaults/kv-gh-secrets"
195+
azure_runner_default_password_key_vault_id = "/.../secrets/azure-runner-default-password"
196+
github_client_secret_key_vault_id = "/.../secrets/github-client-secret"
197+
github_webhook_secret_key_vault_id = "/.../secrets/github-webhook-secret"
198+
github_private_key_key_vault_id = "/.../secrets/github-private-key"
199+
200+
# ---- Misc ----
201+
owners = ["00000000-0000-0000-0000-000000000000"] # Azure AD object IDs
202+
}
203+
204+
output "function_webhook_url" {
205+
value = module.github_runners.function_webhook_url
206+
description = "URL to paste into the GitHub App webhook settings"
207+
sensitive = true
208+
}
209+
```
116210

117-
Run terraform by using the following commands
211+
Run Terraform:
118212

119-
```zsh
213+
```bash
120214
terraform init
121-
terraform apply
215+
terraform plan -out tfplan
216+
terraform apply tfplan
217+
```
218+
219+
After `terraform apply` completes, run:
220+
221+
```bash
222+
terraform output -raw function_webhook_url
122223
```
123224

124-
The terraform output displays the Azure Function endpoint and secret, which you need in the next step.
225+
This returns **the full Azure Function URL**, including a `?code=` query-string. Copy the **entire string** – the `code` value is the Function *host key* and must stay in the URL.
226+
227+
You will also need the **GitHub webhook secret** that you stored in Key Vault earlier:
228+
229+
```bash
230+
az keyvault secret show \
231+
--vault-name <your-kv-name> \
232+
--name github-webhook-secret \
233+
--query value -o tsv
234+
```
235+
236+
Keep both values handy for the next step.
125237

126238
### Deploy Function App and App Service
127239

128240
This terraform module is set up by default to use the latest version of both apps and deploy them on `terraform apply`. Specific versions found in our public [GitHub Packages](https://github.com/orgs/liatrio/packages?repo_name=terraform-azure-github-runner) and set in the terraform module inputs. If you choose to publish your own images, functionality to do so will be implemented soon™.
129241

130-
### Setup the webhook and install the GitHub App
242+
### Configure the GitHub App webhook and install the app
243+
244+
In the GitHub UI navigate to **Settings → Developer settings → GitHub Apps → _Your App_** and:
131245

132-
Go back to the GitHub App and update the following settings
246+
1. **Activate** the webhook toggle.
247+
2. **Webhook URL** – paste the Function URL you copied from `terraform output` (it already contains the `code` host key).
248+
3. **Webhook secret** – paste the value of `github-webhook-secret` you retrieved from Key Vault.
249+
4. **Save**.
250+
5. Open **Install App**, select the gear icon next to your organization, under 'Repository access' select 'All repositories' and click **Save**.
133251

134-
1. Activate the webhook
135-
2. Provide the webhook url, which should be part of the terraform output
136-
3. Provide the webhook secret
137-
4. Save changes and navigate to the Install App tab
138-
5. Next to your GitHub App, select Install next to your org and select 'All Repositories'
252+
The system is now wired together: GitHub sends signed workflow-job events to your Function, the Function enqueues work, and the Runner Controller spins up VMs on demand.
139253

140254
## Required Inputs
141255

0 commit comments

Comments
 (0)