Skip to content

Commit 753d6f7

Browse files
Cap PGagneum
Cap PG
authored and
agneum
committed
feat: add dblab teleport serve CLI command + hostssl cert in default pg_hba
1 parent 8c3ef6f commit 753d6f7

10 files changed

Lines changed: 1774 additions & 4 deletions

File tree

‎docs/plans/20260319-teleport-setup-command.md‎

Lines changed: 273 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,273 @@
1+
# `dblab teleport setup` CLI command
2+
3+
## Overview
4+
5+
Add a `dblab teleport setup` command that automates the Teleport integration
6+
setup described in `engine/cmd/cli/commands/teleport/SETUP.md`. Currently, users must
7+
manually create Teleport roles, generate bot identities, create SSL certificates,
8+
export CA certs, write pg_hba.conf, and configure the Teleport DB agent — a multi-step
9+
process that is error-prone and time-consuming.
10+
11+
The command runs 8 idempotent steps in sequence, skipping any that are already done.
12+
It is flag-driven (no interactive prompts), consistent with the existing CLI pattern.
13+
For server.yml changes, the command prints YAML snippets for the user to apply manually
14+
rather than editing the file directly (to avoid YAML anchor/comment corruption).
15+
16+
**Acceptance criteria:** Running `dblab teleport setup` on a fresh host with Teleport
17+
access produces all required files and Teleport resources. After following the printed
18+
next-steps (DLE restart, data refresh), the Teleport integration is fully operational.
19+
20+
## Context (from discovery)
21+
22+
- files/components involved:
23+
- `engine/cmd/cli/commands/teleport/serve.go` — existing command pattern to follow
24+
- `engine/cmd/cli/commands/teleport/tctl.go` — existing tctl execution helpers
25+
- `engine/cmd/cli/commands/teleport/SETUP.md` — reference for all setup steps
26+
- `engine/cmd/cli/main.go:43` — where `teleport.CommandList()` is registered
27+
- related patterns found:
28+
- CLI uses `urfave/cli/v2`, all config via flags/env vars, no interactive prompts
29+
- `tctl` called via `exec.CommandContext()` with identity + auth-server args
30+
- existing `runTctl()` helper in `tctl.go` requires identity file (not usable for pre-identity steps)
31+
- dependencies identified:
32+
- external binaries: `tctl`, `tbot`, `openssl`
33+
- DBLab API client for edition check
34+
35+
## Development Approach
36+
37+
- **testing approach**: Regular (code first, then tests)
38+
- complete each task fully before moving to the next
39+
- make small, focused changes
40+
- use a `cmdRunner` interface for external command execution to enable mocking in tests
41+
- **CRITICAL: every task MUST include new/updated tests**
42+
- **CRITICAL: all tests must pass before starting next task**
43+
- **CRITICAL: update this plan file when scope changes during implementation**
44+
- run tests after each change
45+
- maintain backward compatibility
46+
47+
## Testing Strategy
48+
49+
- **unit tests**: required for every task
50+
- use `cmdRunner` interface to mock all external command calls (tctl/tbot/openssl)
51+
- test idempotency checks (skip-if-done logic)
52+
- test pg_hba.conf and teleport.yaml content generation
53+
- test flag validation and defaults
54+
- test token parsing with real output samples from tctl/tbot
55+
- no e2e tests (CLI command, tested via unit tests + manual verification on demo server)
56+
57+
## Progress Tracking
58+
59+
- mark completed items with `[x]` immediately when done
60+
- add newly discovered tasks with ➕ prefix
61+
- document issues/blockers with ⚠️ prefix
62+
- update plan if implementation deviates from original scope
63+
64+
## What Goes Where
65+
66+
- **Implementation Steps** (`[ ]` checkboxes): code changes, tests, documentation
67+
- **Post-Completion** (no checkboxes): manual testing on demo server, SETUP.md updates
68+
69+
## Implementation Steps
70+
71+
### Task 1: Command registration, config, and command runner interface
72+
73+
**Files:**
74+
- Create: `engine/cmd/cli/commands/teleport/setup.go`
75+
76+
- [ ] add `setup` subcommand to the existing `teleport` parent command in `CommandList()`
77+
- [ ] define all flags with defaults: `--teleport-proxy` (required), `--cert-dir` (default: `/var/lib/dblab/cert`), `--pg-hba-path` (default: `/var/lib/dblab/pg_hba.conf`), `--teleport-yaml` (default: `/etc/teleport.yaml`), `--identity-dir` (default: `/etc/teleport/bot-dest`), `--environment-id` (required), `--webhook-listen-addr` (default: `172.17.0.1:9876`), `--dblab-url` (default: `http://localhost:2345`), `--dblab-token` (env: `DBLAB_TOKEN`), `--webhook-secret` (env: `WEBHOOK_SECRET`)
78+
- [ ] define `SetupConfig` struct and `setupAction` that parses flags, validates required fields
79+
- [ ] define `cmdRunner` interface (`Run(ctx, name, args) ([]byte, error)`) and real implementation using `exec.CommandContext`
80+
- [ ] implement `checkTools()` to verify `tctl`, `tbot`, `openssl` are in PATH
81+
- [ ] write tests for flag validation and `checkTools()` using mock `cmdRunner`
82+
- [ ] run tests — must pass before task 2
83+
84+
### Task 2: Teleport role creation (steps 1-2)
85+
86+
**Files:**
87+
- Create: `engine/cmd/cli/commands/teleport/setup_steps.go`
88+
89+
- [ ] implement `ensureRole(ctx, runner, name, yamlContent)` — runs `tctl get role/<name>` (without --identity, uses user's tctl credentials from tsh login), creates via `tctl create -f -` if not found
90+
- [ ] define `dblabBotRoleYAML` constant (db/db_server/app/app_server permissions)
91+
- [ ] define `dblabUserRoleYAML` constant (db_labels: dblab=true)
92+
- [ ] wire steps 1-2 into `setupAction` with `[1/8]` / `[2/8]` log output
93+
- [ ] write tests for `ensureRole` — role exists (skip), role missing (create), tctl error
94+
- [ ] run tests — must pass before task 3
95+
96+
### Task 3: Bot identity generation (step 3)
97+
98+
**Files:**
99+
- Modify: `engine/cmd/cli/commands/teleport/setup_steps.go`
100+
101+
- [ ] implement `ensureBotIdentity(ctx, runner, cfg)` — check if identity file exists in `--identity-dir`, skip if present
102+
- [ ] check if bot already exists with `tctl bots ls`, handle "bot exists but identity missing" case
103+
- [ ] run `tctl bots add dblab-sidecar --roles=dblab-bot --format=json` (use --format=json for stable token parsing; fall back to regex if --format not supported)
104+
- [ ] run `tbot start --oneshot` with the parsed token
105+
- [ ] wire step 3 into `setupAction` with `[3/8]` log output
106+
- [ ] write tests: identity exists (skip), bot missing (create + tbot), bot exists but identity missing (recreate identity), token parsing from JSON and fallback regex
107+
- [ ] run tests — must pass before task 4
108+
109+
### Task 4: SSL certificate and Teleport CA generation (steps 4-5)
110+
111+
**Files:**
112+
- Modify: `engine/cmd/cli/commands/teleport/setup_steps.go`
113+
114+
- [ ] implement `ensureSSLCerts(ctx, runner, certDir)` — check if `server.crt` + `server.key` exist, skip if present; mkdir certDir if needed
115+
- [ ] run `openssl req -new -x509 ...` then `chown 999:999`; if chown fails, log warning instead of failing (user may not be root)
116+
- [ ] implement `ensureTeleportCA(ctx, runner, certDir)` — check if `teleport-ca.crt` exists, skip if present
117+
- [ ] run `tctl auth export --type=db-client` then `chown 999:999` (same warning approach)
118+
- [ ] wire steps 4-5 into `setupAction` with `[4/8]` / `[5/8]` log output
119+
- [ ] write tests for cert existence checks, command construction, chown warning on failure
120+
- [ ] run tests — must pass before task 5
121+
122+
### Task 5: pg_hba.conf generation (step 6)
123+
124+
**Files:**
125+
- Modify: `engine/cmd/cli/commands/teleport/setup_steps.go`
126+
127+
- [ ] implement `ensurePgHba(path)` — check if file exists, skip if present
128+
- [ ] write pg_hba.conf with comment header and 3 rules: `local trust`, `hostssl cert`, `host md5`
129+
- [ ] wire step 6 into `setupAction` with `[6/8]` log output
130+
- [ ] write tests for content generation and skip-if-exists
131+
- [ ] run tests — must pass before task 6
132+
133+
### Task 6: Print server.yml configuration snippets (step 7)
134+
135+
**Files:**
136+
- Modify: `engine/cmd/cli/commands/teleport/setup_steps.go`
137+
138+
- [ ] implement `printServerYmlSnippets(cfg)` — generates and prints YAML blocks for the user to add to server.yml
139+
- [ ] print `databaseConfigs` snippet with ssl, ssl_cert_file, ssl_key_file, ssl_ca_file (using paths from cfg.CertDir)
140+
- [ ] print `databaseContainer.containerConfig.volume` snippet with cert mount path
141+
- [ ] print `webhooks` snippet with URL (using cfg.WebhookAddr), secret, and triggers
142+
- [ ] wire step 7 into `setupAction` with `[7/8]` log output
143+
- [ ] write tests for snippet content (correct paths, correct webhook URL)
144+
- [ ] run tests — must pass before task 7
145+
146+
### Task 7: teleport.yaml generation (step 8)
147+
148+
**Files:**
149+
- Modify: `engine/cmd/cli/commands/teleport/setup_steps.go`
150+
151+
- [ ] implement `ensureTeleportYaml(ctx, runner, path, cfg)` — check if file exists, skip if present
152+
- [ ] create join token via `tctl tokens add --type=db --ttl=8760h --format=json` (fall back to regex parsing)
153+
- [ ] write teleport.yaml with proxy_server, join_params, db_service with `dblab: "true"` label matcher
154+
- [ ] log a note about token expiration (1 year TTL) in the output
155+
- [ ] wire step 8 into `setupAction` with `[8/8]` log output
156+
- [ ] write tests for teleport.yaml content generation and token parsing
157+
- [ ] run tests — must pass before task 8
158+
159+
### Task 8: Post-setup summary and next-steps output
160+
161+
**Files:**
162+
- Modify: `engine/cmd/cli/commands/teleport/setup.go`
163+
164+
- [ ] implement `printNextSteps(cfg)` — print summary of what was created/skipped
165+
- [ ] print concrete next-steps commands: (1) add server.yml snippets from step 7, (2) restart DLE with `-v pg_hba.conf:/home/dblab/standard/...`, (3) trigger data refresh, (4) start teleport agent, (5) start sidecar with exact flags
166+
- [ ] wire summary into `setupAction` after all steps complete
167+
- [ ] write test for next-steps output containing correct paths from config
168+
- [ ] run tests — must pass before task 9
169+
170+
### Task 9: Verify acceptance criteria
171+
172+
- [ ] verify all 8 steps are implemented and idempotent
173+
- [ ] verify external tool checks fail fast with clear messages
174+
- [ ] verify flag defaults are sensible
175+
- [ ] run full test suite: `make test`
176+
- [ ] run linter: `make run-lint`
177+
- [ ] run formatter: `make fmt`
178+
179+
### Task 10: [Final] Update documentation
180+
181+
- [ ] update SETUP.md to add "Quick Start" section at the top referencing `dblab teleport setup`
182+
- [ ] keep existing manual steps as reference for users who prefer step-by-step
183+
- [ ] move this plan to `docs/plans/completed/`
184+
185+
## Technical Details
186+
187+
### SetupConfig struct
188+
189+
```go
190+
type SetupConfig struct {
191+
TeleportProxy string
192+
CertDir string
193+
PgHbaPath string
194+
TeleportYaml string
195+
IdentityDir string
196+
EnvironmentID string
197+
WebhookAddr string
198+
DblabURL string
199+
DblabToken string
200+
WebhookSecret string
201+
}
202+
```
203+
204+
### cmdRunner interface (for testability)
205+
206+
```go
207+
type cmdRunner interface {
208+
Run(ctx context.Context, name string, args ...string) ([]byte, error)
209+
RunWithStdin(ctx context.Context, stdin string, name string, args ...string) ([]byte, error)
210+
}
211+
```
212+
213+
Real implementation wraps `exec.CommandContext`. Tests provide a mock that records
214+
calls and returns configured output.
215+
216+
### tctl authentication model
217+
218+
Steps 1-3 use `tctl` with the **user's own credentials** (from `tsh login`).
219+
This means `tctl` is called WITHOUT `--identity` or `--auth-server` flags —
220+
it uses the default Teleport profile from `~/.tsh/`.
221+
222+
Steps 5 (Teleport CA export) and 7 (token creation) also use user credentials.
223+
224+
The `--identity` flag from serve.go's `runTctl()` is NOT reused for setup,
225+
because the bot identity does not exist yet during setup.
226+
227+
### Step execution flow
228+
229+
```
230+
setupAction(c *cli.Context)
231+
├── parse flags → SetupConfig
232+
├── checkTools(tctl, tbot, openssl)
233+
├── [1/8] ensureRole("dblab-bot", dblabBotRoleYAML)
234+
├── [2/8] ensureRole("dblab-user", dblabUserRoleYAML)
235+
├── [3/8] ensureBotIdentity(cfg)
236+
├── [4/8] ensureSSLCerts(cfg.CertDir)
237+
├── [5/8] ensureTeleportCA(cfg.CertDir)
238+
├── [6/8] ensurePgHba(cfg.PgHbaPath)
239+
├── [7/8] printServerYmlSnippets(cfg) // prints, does not edit
240+
├── [8/8] ensureTeleportYaml(cfg)
241+
└── printNextSteps(cfg)
242+
```
243+
244+
### File layout
245+
246+
- `setup.go` — command registration, flags, SetupConfig, setupAction, cmdRunner interface, checkTools
247+
- `setup_steps.go` — step implementations (ensureRole, ensureBotIdentity, ensureSSLCerts, etc.)
248+
- `setup_test.go` — all tests with mock cmdRunner
249+
250+
### Token parsing strategy
251+
252+
Use `--format=json` flag for `tctl bots add` and `tctl tokens add` when available.
253+
If the command fails with `--format=json` (older Teleport versions), fall back to
254+
regex parsing of human-readable output:
255+
- Bot token: `The bot token: ([a-f0-9]+)`
256+
- Join token: `The invite token: ([a-f0-9]+)`
257+
258+
### Role YAML templates
259+
260+
Embedded as Go string constants, matching the YAML from SETUP.md §1-§2.
261+
262+
## Post-Completion
263+
264+
**Manual verification:**
265+
- test full setup flow on a test server with a Teleport cluster
266+
- test idempotency by running the command twice
267+
- test with missing external tools
268+
- test with partially completed setup (some steps done, some not)
269+
- test as non-root user (verify chown warnings)
270+
271+
**SETUP.md update:**
272+
- add "Quick Start" section at the top referencing `dblab teleport setup`
273+
- keep manual steps as reference for users who prefer step-by-step

‎engine/.gitlab-ci.yml‎

Lines changed: 27 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,30 @@ build-binary-client-dev:
9191
- cd engine
9292
- make build-client
9393

94+
build-binary-client-branch:
95+
rules:
96+
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_SOURCE_PROJECT_PATH == $CI_PROJECT_PATH'
97+
changes:
98+
- engine/**/*
99+
stage: build-binary
100+
artifacts:
101+
paths:
102+
- engine/bin
103+
script:
104+
- cd engine
105+
- make build-client
106+
107+
# Install google-cloud-sdk.
108+
- curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
109+
- echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
110+
- apt-get update && apt-get install -y google-cloud-sdk
111+
112+
# Authenticate.
113+
- GCP_KEY_FILE=$(mktemp) && trap "rm -f \"$GCP_KEY_FILE\"" EXIT && printf '%s' "${GCP_SERVICE_KEY}" > "$GCP_KEY_FILE" && gcloud auth activate-service-account --key-file="$GCP_KEY_FILE" && rm -f "$GCP_KEY_FILE"
114+
115+
# Upload artifacts using MR IID to prevent cross-branch overwrites.
116+
- gsutil -m cp -r bin/cli/* gs://database-lab-cli/mr-${CI_MERGE_REQUEST_IID}/
117+
94118
build-binary-client-master:
95119
<<: *only_master
96120
stage: build-binary
@@ -107,7 +131,7 @@ build-binary-client-master:
107131
- apt-get update && apt-get install -y google-cloud-sdk
108132

109133
# Authenticate.
110-
- echo $GCP_SERVICE_KEY | gcloud auth activate-service-account --key-file=-
134+
- GCP_KEY_FILE=$(mktemp) && trap "rm -f \"$GCP_KEY_FILE\"" EXIT && printf '%s' "${GCP_SERVICE_KEY}" > "$GCP_KEY_FILE" && gcloud auth activate-service-account --key-file="$GCP_KEY_FILE" && rm -f "$GCP_KEY_FILE"
111135

112136
# Upload artifacts.
113137
- gsutil -m cp -r bin/cli/* gs://database-lab-cli/master/
@@ -127,7 +151,7 @@ build-binary-client:
127151
- apt-get update && apt-get install -y google-cloud-sdk
128152

129153
# Authenticate.
130-
- echo $GCP_SERVICE_KEY | gcloud auth activate-service-account --key-file=-
154+
- GCP_KEY_FILE=$(mktemp) && trap "rm -f \"$GCP_KEY_FILE\"" EXIT && printf '%s' "${GCP_SERVICE_KEY}" > "$GCP_KEY_FILE" && gcloud auth activate-service-account --key-file="$GCP_KEY_FILE" && rm -f "$GCP_KEY_FILE"
131155

132156
# Upload artifacts.
133157
- gsutil -m cp -r bin/cli/* gs://database-lab-cli/${CLEAN_TAG}/
@@ -148,7 +172,7 @@ build-binary-client-rc:
148172
- apt-get update && apt-get install -y google-cloud-sdk
149173

150174
# Authenticate.
151-
- echo $GCP_SERVICE_KEY | gcloud auth activate-service-account --key-file=-
175+
- GCP_KEY_FILE=$(mktemp) && trap "rm -f \"$GCP_KEY_FILE\"" EXIT && printf '%s' "${GCP_SERVICE_KEY}" > "$GCP_KEY_FILE" && gcloud auth activate-service-account --key-file="$GCP_KEY_FILE" && rm -f "$GCP_KEY_FILE"
152176

153177
# Upload artifacts.
154178
- gsutil -m cp -r bin/cli/* gs://database-lab-cli/${CLEAN_TAG}/

0 commit comments

Comments
 (0)