Skip to content

Commit 7e499a1

Browse files
CoderCococlaude
andauthored
feat(terraform): declarative EFS file seeding via file_seeds in game_servers (#42)
## Summary Closes #38 Adds a declarative file seeding system that pre-populates files on EFS volumes before game servers start. This allows users to define configuration files, mods, and other assets directly in Terraform without manual setup. ## Key Changes - **New EFS Seeder Lambda** (`terraform/efs-seeder.tf`): - Per-game Lambda function that mounts the first volume's EFS access point - Receives file seed declarations and writes them to the mounted EFS volume - Invocation is content-addressed (re-triggers only when seed content changes) - Shared security group for all seeder Lambdas with NFS egress to EFS - **Seeder Handler** (`app/packages/lambda/efs-seeder/src/handler.ts`): - Validates seed paths against the container's `container_path` prefix to prevent path traversal - Supports both UTF-8 text (`content`) and binary (`content_base64`) file content - Creates parent directories as needed and sets file permissions via optional `mode` field - Comprehensive logging for debugging - **Build Configuration** (`app/packages/lambda/efs-seeder/`): - esbuild config for bundling TypeScript handler to CommonJS - Package.json with build/clean scripts - TypeScript configuration extending base tsconfig - **Terraform Integration**: - Added `file_seeds` optional field to `game_servers` variable schema - Updated EFS security group to allow NFS ingress from seeder Lambdas - Updated documentation with usage examples and troubleshooting - **Documentation**: - Added `file_seeds` configuration guide with examples (text and binary files) - Included troubleshooting section for common seeder issues - Example tfvars showing Palworld config and mod seeding patterns ## Notable Implementation Details - **Content-Addressed Invocation**: Uses SHA256 hash of seed content as trigger, making re-applies idempotent when seeds haven't changed - **Path Safety**: Validates that all seed paths start with the volume's `container_path` and resolves safely within the mount point to prevent directory traversal attacks - **No Secret Storage**: Documentation explicitly warns against storing secrets in `file_seeds` since content is persisted in Terraform state - **Removed Entries Not Deleted**: Seeder only writes files; removed seed entries are not cleaned up from EFS (users must manually delete via FileBrowser) - **Binary File Support**: Base64 encoding allows seeding of non-UTF-8 files like game mods https://claude.ai/code/session_01VTBPgzdvwxeEw6nBscSwn6 --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 2c9b5ac commit 7e499a1

13 files changed

Lines changed: 574 additions & 2 deletions

File tree

app/package-lock.json

Lines changed: 11 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

app/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
"dev": "concurrently -n server,client -c cyan,magenta \"npm run dev -w @gsd/server\" \"npm run dev -w @gsd/web\"",
1515
"prebuild": "node scripts/embed-tfstate.mjs",
1616
"build": "npm run build -w @gsd/shared && npm run build -w @gsd/server && npm run build -w @gsd/web",
17-
"build:lambdas": "npm run build -w @gsd/shared && npm run build -w @gsd/lambda-interactions -w @gsd/lambda-followup -w @gsd/lambda-update-dns -w @gsd/lambda-watchdog",
17+
"build:lambdas": "npm run build -w @gsd/shared && npm run build -w @gsd/lambda-interactions -w @gsd/lambda-followup -w @gsd/lambda-update-dns -w @gsd/lambda-watchdog -w @gsd/lambda-efs-seeder",
1818
"start": "node packages/server/dist/main.js",
1919
"test": "vitest run",
2020
"test:watch": "vitest",
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
import { build } from 'esbuild';
2+
import { mkdirSync } from 'fs';
3+
4+
mkdirSync('dist', { recursive: true });
5+
6+
await build({
7+
entryPoints: ['src/handler.ts'],
8+
outfile: 'dist/handler.cjs',
9+
bundle: true,
10+
platform: 'node',
11+
target: 'node20',
12+
format: 'cjs',
13+
minify: true,
14+
sourcemap: true,
15+
logLevel: 'info',
16+
});
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"name": "@gsd/lambda-efs-seeder",
3+
"version": "1.0.0",
4+
"private": true,
5+
"type": "module",
6+
"main": "./dist/handler.cjs",
7+
"scripts": {
8+
"build": "node esbuild.config.mjs",
9+
"clean": "rm -rf dist"
10+
},
11+
"devDependencies": {
12+
"@types/node": "^20.14.0"
13+
}
14+
}
Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
/**
2+
* Tests for the EFS seeder Lambda handler.
3+
*
4+
* The handler receives a list of file seeds and writes them to the EFS
5+
* access point mounted at /mnt/efs, resolving in-container paths by stripping
6+
* the first volume's container_path prefix.
7+
*/
8+
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
9+
10+
const mkdirSyncMock = vi.fn();
11+
const writeFileSyncMock = vi.fn();
12+
13+
vi.mock('fs', () => ({
14+
mkdirSync: (...args: unknown[]) => mkdirSyncMock(...args),
15+
writeFileSync: (...args: unknown[]) => writeFileSyncMock(...args),
16+
}));
17+
18+
/** Import after mocks are registered so the handler uses the mocked fs. */
19+
const { handler } = await import('./handler.js');
20+
21+
const GAME = 'palworld';
22+
const CONTAINER_PATH = '/palworld';
23+
24+
describe('efs-seeder handler', () => {
25+
beforeEach(() => {
26+
vi.clearAllMocks();
27+
});
28+
29+
afterEach(() => {
30+
vi.restoreAllMocks();
31+
});
32+
33+
it('should write a UTF-8 text seed to the correct EFS path', async () => {
34+
await handler({
35+
game: GAME,
36+
seeds: [{ path: '/palworld/Pal/Saved/Config/settings.ini', content: '[Settings]\nkey=value' }],
37+
container_path: CONTAINER_PATH,
38+
});
39+
40+
expect(mkdirSyncMock).toHaveBeenCalledWith(
41+
'/mnt/efs/Pal/Saved/Config',
42+
{ recursive: true },
43+
);
44+
expect(writeFileSyncMock).toHaveBeenCalledWith(
45+
'/mnt/efs/Pal/Saved/Config/settings.ini',
46+
Buffer.from('[Settings]\nkey=value', 'utf8'),
47+
{ flag: 'w', mode: 0o644 },
48+
);
49+
});
50+
51+
it('should write a binary seed decoded from base64', async () => {
52+
const binaryContent = Buffer.from([0x50, 0x4b, 0x03, 0x04]);
53+
const b64 = binaryContent.toString('base64');
54+
55+
await handler({
56+
game: GAME,
57+
seeds: [{ path: '/palworld/Pal/Content/Paks/MyMod.pak', content_base64: b64 }],
58+
container_path: CONTAINER_PATH,
59+
});
60+
61+
expect(writeFileSyncMock).toHaveBeenCalledWith(
62+
'/mnt/efs/Pal/Content/Paks/MyMod.pak',
63+
binaryContent,
64+
{ flag: 'w', mode: 0o644 },
65+
);
66+
});
67+
68+
it('should apply a custom mode when provided', async () => {
69+
await handler({
70+
game: GAME,
71+
seeds: [{ path: '/palworld/server.sh', content: '#!/bin/sh', mode: '0755' }],
72+
container_path: CONTAINER_PATH,
73+
});
74+
75+
expect(writeFileSyncMock).toHaveBeenCalledWith(
76+
'/mnt/efs/server.sh',
77+
expect.any(Buffer),
78+
{ flag: 'w', mode: 0o755 },
79+
);
80+
});
81+
82+
it('should write multiple seeds in a single invocation', async () => {
83+
await handler({
84+
game: GAME,
85+
seeds: [
86+
{ path: '/palworld/a.ini', content: 'a=1' },
87+
{ path: '/palworld/b.ini', content: 'b=2' },
88+
],
89+
container_path: CONTAINER_PATH,
90+
});
91+
92+
expect(writeFileSyncMock).toHaveBeenCalledTimes(2);
93+
});
94+
95+
it('should throw when a seed path does not start with container_path', async () => {
96+
await expect(
97+
handler({
98+
game: GAME,
99+
seeds: [{ path: '/other/path/config.ini', content: 'x=1' }],
100+
container_path: CONTAINER_PATH,
101+
}),
102+
).rejects.toThrow('does not start with container_path');
103+
});
104+
105+
it('should throw when the seed path equals container_path with no file component', async () => {
106+
await expect(
107+
handler({
108+
game: GAME,
109+
seeds: [{ path: '/palworld', content: 'x=1' }],
110+
container_path: CONTAINER_PATH,
111+
}),
112+
).rejects.toThrow('no file component after container_path');
113+
});
114+
115+
it('should throw on path traversal attempts', async () => {
116+
await expect(
117+
handler({
118+
game: GAME,
119+
seeds: [{ path: '/palworld/../../../etc/passwd', content: 'evil' }],
120+
container_path: CONTAINER_PATH,
121+
}),
122+
).rejects.toThrow();
123+
});
124+
125+
it('should throw when a seed has neither content nor content_base64', async () => {
126+
await expect(
127+
handler({
128+
game: GAME,
129+
seeds: [{ path: '/palworld/empty.txt' }],
130+
container_path: CONTAINER_PATH,
131+
}),
132+
).rejects.toThrow('neither content nor content_base64');
133+
});
134+
135+
it('should throw when a seed sets both content and content_base64', async () => {
136+
await expect(
137+
handler({
138+
game: GAME,
139+
seeds: [{ path: '/palworld/a.ini', content: 'x=1', content_base64: 'eD0x' }],
140+
container_path: CONTAINER_PATH,
141+
}),
142+
).rejects.toThrow('sets both content and content_base64');
143+
});
144+
145+
it('should throw when mode is not a valid octal string', async () => {
146+
await expect(
147+
handler({
148+
game: GAME,
149+
seeds: [{ path: '/palworld/a.ini', content: 'x=1', mode: 'rwxr-xr-x' }],
150+
container_path: CONTAINER_PATH,
151+
}),
152+
).rejects.toThrow('invalid mode');
153+
});
154+
155+
it('should handle an empty seeds list without errors', async () => {
156+
await expect(
157+
handler({ game: GAME, seeds: [], container_path: CONTAINER_PATH }),
158+
).resolves.toBeUndefined();
159+
160+
expect(writeFileSyncMock).not.toHaveBeenCalled();
161+
});
162+
});
Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
import { writeFileSync, mkdirSync } from 'fs';
2+
import { dirname, resolve, join } from 'path';
3+
4+
interface FileSeed {
5+
path: string;
6+
content?: string;
7+
content_base64?: string;
8+
mode?: string;
9+
}
10+
11+
/** Payload sent by `aws_lambda_invocation.efs_seeder` in Terraform. */
12+
interface SeederEvent {
13+
game: string;
14+
seeds: FileSeed[];
15+
/** The first volume's `container_path`, used to resolve in-container paths to EFS-relative paths. */
16+
container_path: string;
17+
}
18+
19+
const MOUNT_POINT = '/mnt/efs';
20+
21+
/**
22+
* Strips the container_path prefix from a seed path and resolves it to an
23+
* absolute destination under MOUNT_POINT. Throws on path-traversal attempts
24+
* or if the path resolves to the mount root (i.e. no file name was given).
25+
*/
26+
function resolveDestination(seedPath: string, containerPath: string): string {
27+
const normalizedContainer = containerPath.replace(/\/$/, '');
28+
29+
if (seedPath === normalizedContainer) {
30+
throw new Error(`Seed path "${seedPath}" has no file component after container_path`);
31+
}
32+
33+
if (!seedPath.startsWith(normalizedContainer + '/')) {
34+
throw new Error(
35+
`Seed path "${seedPath}" does not start with container_path "${normalizedContainer}"`,
36+
);
37+
}
38+
39+
const relative = seedPath.slice(normalizedContainer.length).replace(/^\//, '');
40+
41+
const dest = resolve(join(MOUNT_POINT, relative));
42+
43+
if (!dest.startsWith(MOUNT_POINT + '/')) {
44+
throw new Error(`Path traversal detected: "${seedPath}" resolves outside mount point`);
45+
}
46+
47+
return dest;
48+
}
49+
50+
/**
51+
* Writes each `file_seeds` entry to the EFS access point mounted at
52+
* `/mnt/efs`. Invoked synchronously by `aws_lambda_invocation` during
53+
* `terraform apply`; throws on any error so Terraform surfaces the failure.
54+
*/
55+
export const handler = async (event: SeederEvent): Promise<void> => {
56+
const { game, seeds, container_path: containerPath } = event;
57+
58+
console.log(`Seeding ${seeds.length} file(s) for game "${game}" (mount: ${MOUNT_POINT})`);
59+
60+
for (const seed of seeds) {
61+
const dest = resolveDestination(seed.path, containerPath);
62+
63+
if (seed.content !== undefined && seed.content_base64 !== undefined) {
64+
throw new Error(
65+
`Seed at path "${seed.path}" sets both content and content_base64 — use one or the other`,
66+
);
67+
}
68+
69+
let content: Buffer;
70+
if (seed.content_base64 !== undefined) {
71+
content = Buffer.from(seed.content_base64, 'base64');
72+
} else if (seed.content !== undefined) {
73+
content = Buffer.from(seed.content, 'utf8');
74+
} else {
75+
throw new Error(`Seed at path "${seed.path}" has neither content nor content_base64`);
76+
}
77+
78+
const modeStr = seed.mode ?? '0644';
79+
if (!/^0?[0-7]{3,4}$/.test(modeStr)) {
80+
throw new Error(`Seed at path "${seed.path}" has invalid mode "${modeStr}" — expected an octal string such as "0644"`);
81+
}
82+
const mode = parseInt(modeStr, 8);
83+
84+
mkdirSync(dirname(dest), { recursive: true });
85+
writeFileSync(dest, content, { flag: 'w', mode });
86+
87+
console.log(`Wrote ${dest} (${content.length} bytes, mode ${seed.mode ?? '0644'})`);
88+
}
89+
90+
console.log(`Done — ${seeds.length} file(s) seeded for game "${game}"`);
91+
};
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"extends": "../../../tsconfig.base.json",
3+
"compilerOptions": {
4+
"outDir": "./dist",
5+
"rootDir": "./src",
6+
"moduleResolution": "node",
7+
"module": "ESNext",
8+
"noEmit": true,
9+
"composite": false
10+
},
11+
"include": ["src/**/*"],
12+
"exclude": ["src/**/*.test.ts", "dist", "node_modules"]
13+
}

docs/docs/components/terraform.md

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ step 3 of the [setup guide](/setup) for details.
1717
| `alb.tf` | Conditional on any game having `https = true`: ACM certificate (DNS-validated), ALB + target groups per HTTPS game, HTTPS listener + HTTP→HTTPS redirect, Route 53 ALIAS records. |
1818
| `route53.tf` | Route 53 zone **data source** (zone must exist); the `update-dns` Lambda with its IAM, EventBridge rule on `ECS Task State Change`. |
1919
| `watchdog.tf` | `watchdog` Lambda with its IAM, EventBridge schedule at `rate(${watchdog_interval_minutes} minute(s))`. |
20+
| `efs-seeder.tf` | Conditional on any game having `file_seeds`: shared seeder SG, per-game IAM role + policy, CloudWatch log group, Lambda (VPC + EFS mount), and `aws_lambda_invocation` that re-triggers only when seed content changes. |
2021
| `interactions.tf` | `interactions` Lambda with IAM + Function URL (`auth_type = NONE`, CORS for `https://discord.com`). Exposes `interactions_invoke_url`. |
2122
| `followup.tf` | `followup` Lambda with IAM (`ecs:RunTask`, `StopTask`, `DescribeTasks`, `iam:PassRole`, `dynamodb:GetItem`/`PutItem`, `ec2:DescribeNetworkInterfaces`). Async-invoked by interactions. |
2223
| `discord_store.tf` | DynamoDB table (pk+sk, TTL on `expiresAt`), two Secrets Manager secrets (`${project_name}/discord/bot-token`, `/discord/public-key`) with `recovery_window_in_days = 0` and `lifecycle.ignore_changes` on seeded secret values. Optional `CONFIG#discord` DynamoDB item seeded from tfvars. Optional `BASE#discord` item holding the Terraform-managed base allowlist/admins (see `base_allowed_guilds` / `base_admin_*` variables). When `discord_bot_token`, `discord_application_id`, and at least one `base_allowed_guilds` entry are set, a `null_resource` runs `curl` to register slash commands in each base guild during apply; re-runs on token rotation or command-descriptor changes. |
@@ -31,7 +32,7 @@ step 3 of the [setup guide](/setup) for details.
3132
| `aws_region` | `string` | `us-east-1` | AWS region for all resources. |
3233
| `project_name` | `string` | `game-servers` | Prefix for named resources and the Secrets Manager paths. |
3334
| `vpc_cidr` | `string` | `10.0.0.0/16` | Parent CIDR; subnets are /24s within it. |
34-
| `game_servers` | `map(object)` || The single source of truth. Per-game: `image`, `cpu`, `memory`, `ports[]`, `environment[]`, `volumes[]` (`name` + `container_path`), `https`. Each `volumes` entry creates its own EFS access point rooted at `/${game}/${name}`. |
35+
| `game_servers` | `map(object)` || The single source of truth. Per-game: `image`, `cpu`, `memory`, `ports[]`, `environment[]`, `volumes[]` (`name` + `container_path`), `https`, `file_seeds[]` (optional). Each `volumes` entry creates its own EFS access point rooted at `/${game}/${name}`. See `game_servers[].file_seeds` below. |
3536
| `hosted_zone_name` | `string` | _(required)_ | Existing Route 53 zone looked up as a data source (e.g. `example.com`). |
3637
| `acm_certificate_domain` | `string` | `null``*.{hosted_zone_name}` | Wildcard ACM cert for the ALB listener. |
3738
| `dns_ttl` | `number` | `30` | TTL on Route 53 A records the update-dns Lambda writes. Keep low for fast task churn. |
@@ -46,6 +47,22 @@ step 3 of the [setup guide](/setup) for details.
4647
| `base_admin_role_ids` | `list(string)` | `[]` | Discord role IDs with permanent server-wide admin rights. Same Terraform-managed floor as above. |
4748
| `tags` | `map(string)` | defaults | Merged into `default_tags` for cost allocation (`Project`). |
4849

50+
### `game_servers[].file_seeds` (optional)
51+
52+
Declare files to be written to a game's EFS volume during `terraform apply`.
53+
Each entry in the list is:
54+
55+
| Field | Type | Default | Description |
56+
|---|---|---|---|
57+
| `path` | `string` | _(required)_ | In-container path (e.g. `/palworld/Pal/Saved/Config/LinuxServer/PalWorldSettings.ini`). The first volume's `container_path` is stripped to resolve the EFS-relative destination. |
58+
| `content` | `string` | `null` | UTF-8 text content. Mutually exclusive with `content_base64`. |
59+
| `content_base64` | `string` | `null` | Base64-encoded binary content — use for non-UTF-8 files such as mod `.pak` files (`base64 -w0 MyMod.pak`). |
60+
| `mode` | `string` | `"0644"` | chmod octal string applied to the written file. |
61+
62+
When `file_seeds` is non-empty, `efs-seeder.tf` creates a seeder Lambda for the game and invokes it immediately. The invocation re-runs only when the sha256 of `file_seeds` changes, making re-applies with unchanged seeds a no-op. Removed seed entries are **not** deleted from EFS — clean them up via FileBrowser.
63+
64+
> **Do not store secrets in `file_seeds`** — content is written verbatim into Terraform state.
65+
4966
## Outputs
5067

5168
| Output | Consumer |
@@ -95,6 +112,14 @@ step 3 of the [setup guide](/setup) for details.
95112
- **`events:TagResource` / `UntagResource` / `ListTagsForResource`** aren't
96113
in any AWS-managed policy — you need `events:*` (or at least those three)
97114
on the deploy user. The setup guide's inline policy already covers this.
115+
- **`file_seeds` targets the first volume only.** The seeder Lambda mounts the
116+
EFS access point for `volumes[0]`, so all seed `path` values must use that
117+
volume's `container_path` as a prefix. Multi-volume games with seeds across
118+
different volumes are not supported in this release.
119+
- **`file_seeds` content lives in Terraform state.** Suitable for config files
120+
and small binary assets (mods). Do not put passwords or tokens here.
121+
- **Removed seed entries are not deleted from EFS.** They are simply no longer
122+
managed. Delete stale files via the FileBrowser task.
98123
- **Removing a game from the map deletes its task definition** but does not
99124
stop running tasks. Stop the game from the dashboard first, then remove
100125
the key.

0 commit comments

Comments
 (0)