Skip to content

Commit ae02539

Browse files
authored
[minor] Document compose reconcile startup (#15)
1 parent e3d728e commit ae02539

24 files changed

Lines changed: 581 additions & 55 deletions

apps.mdx

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
---
2+
title: Managed applications
3+
description: How sitectl-managed application templates use Docker Compose, local image builds, and development overrides.
4+
---
5+
6+
import { Compose } from "/snippets/compose-tooltip.mdx";
7+
8+
`sitectl` application templates are normal <Compose /> projects with a shared operating contract. The repository owns the application workload, and `sitectl` supplies context-aware lifecycle commands, component updates, health checks, image overrides, and service helpers.
9+
10+
## Compose Files
11+
12+
`docker-compose.yml` or `docker-compose.yaml` is the production contract. It should describe the stack that runs on the host: services, volumes, secrets, Traefik ingress, health checks, build definitions, and the default published ports.
13+
14+
Local-only changes belong in `docker-compose.override.yml`. That file is intentionally ignored by template repositories. Developers can use it for temporary bind mounts, image overrides, secrets used only on their workstation, and port swaps. Production should not rely on a Compose override file; production behavior should be visible in the tracked Compose file and tracked component changes.
15+
16+
When an application service requires MariaDB during startup, declare that relationship in the tracked Compose file with `depends_on` and `condition: service_healthy`. The application plugins' standard healthcheck runner verifies that policy for MariaDB-backed web apps.
17+
18+
## Local Image Builds
19+
20+
Every application template includes a Dockerfile for the local application image. The Dockerfile starts from the matching LibOps base image, copies lockfiles and downloaded dependencies before copying local modules, plugins, themes, config, or other customization points, and leaves runtime configuration to the Compose environment where possible.
21+
22+
This keeps local builds fast:
23+
24+
- Docker can reuse package-manager and download cache layers when only local custom code changed.
25+
- The build uses the platform selected by the Docker CLI on the host.
26+
- Local builds do not push images. Publishing images belongs in CI or another explicit release workflow.
27+
28+
The LibOps base images provide the default PHP and nginx configuration. When a template needs to tune upload size, timeouts, trusted proxy handling, or similar runtime behavior, prefer environment values in the Compose service over rebuilding nginx or PHP config into the downstream template image.
29+
30+
Use [`sitectl image set`](/commands/image) when you need a local context to run a different image, tag, or build argument without editing the tracked template files.
31+
32+
## Development Ports
33+
34+
Application templates should publish standard ingress ports in the tracked Compose file:
35+
36+
```yaml
37+
services:
38+
traefik:
39+
ports:
40+
- "80:80"
41+
```
42+
43+
If the stack serves HTTPS directly, publish `443:443` in the tracked Compose file as well. `sitectl` infers `URI_SCHEME=https` when the Compose project publishes target port `443` or Traefik declares an HTTPS entrypoint on `:443`; otherwise it uses `URI_SCHEME=http`.
44+
45+
For local development contexts, [`sitectl compose up`](/commands/compose) checks whether `80` or `443` is already occupied. If another process owns a needed port, it writes `docker-compose.override.yml` with a Compose `ports: !override` entry such as:
46+
47+
```yaml
48+
services:
49+
traefik:
50+
ports: !override
51+
- "8443:443"
52+
```
53+
54+
That smart port allocation is a development convenience only. It should not be committed and should not be part of production deployment.
55+
56+
## Local HTTPS
57+
58+
If production runs HTTPS and a developer wants the local site to behave the same way, use mkcert-backed Traefik TLS:
59+
60+
```bash
61+
sitectl traefik tls mkcert --domain app.localhost
62+
sitectl compose up --remove-orphans -d
63+
```
64+
65+
See [Traefik service commands](/plugins/traefik) for TLS modes and [Compose commands](/commands/compose) for the local port behavior.

commands/compose.mdx

Lines changed: 47 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,61 @@ description: Run Docker Compose commands through the active sitectl context.
55

66
import { Compose } from "/snippets/compose-tooltip.mdx";
77
import ComposeCmd from "/snippets/commands/sitectl-compose.mdx";
8+
import ComposeCleanCmd from "/snippets/commands/sitectl-compose-clean.mdx";
9+
import ComposeReconcileCmd from "/snippets/commands/sitectl-compose-reconcile.mdx";
810

911
# <Compose /> command
1012

1113
The `compose` command is a core `sitectl` command that runs <Compose /> commands against the active `sitectl` context.
1214

1315
Use it when you want the same <Compose /> operation to respect the site and environment wiring already stored in the context.
1416

15-
For local development contexts, `sitectl compose up` checks whether the default HTTP host port is already in use. If another process owns it, sitectl passes `HOST_INSECURE_PORT` to Docker Compose with the next available development port. It only checks and sets `HOST_SECURE_PORT` when the active Compose files publish target port `443` or reference `HOST_SECURE_PORT`, so HTTP-only stacks do not bind HTTPS. This behavior only applies to local dev contexts, not local production workloads.
17+
For local development contexts, `sitectl compose up` checks whether the default HTTP or HTTPS host port is already in use. If another process owns a needed port, sitectl writes `docker-compose.override.yml` with a Compose `ports: !override` entry for the next available development port before invoking Docker Compose.
18+
19+
HTTP-only stacks publish target port `80` and use `URI_SCHEME=http`. Stacks that publish target port `443`, or whose Traefik command declares an HTTPS entrypoint on `:443`, use `URI_SCHEME=https`. This behavior only applies to local dev contexts, not local production workloads.
20+
21+
See [Managed applications](/apps) for the application template Compose contract.
22+
23+
## Reconcile
24+
25+
For local plugin-owned contexts, `sitectl compose up` can perform a reconcile before handing off to Docker Compose. It reads the plugin's default create definition, checks init artifacts, declared volumes, and local image metadata, then runs the plugin's init and build commands when they are still needed.
26+
27+
This lets a setup-only checkout start cleanly:
28+
29+
```bash
30+
sitectl create ojs/default --setup-only --path ./ojs --type local --checkout-source template
31+
sitectl compose up
32+
```
33+
34+
You can also run the same workflow explicitly:
35+
36+
```bash
37+
sitectl compose reconcile
38+
sitectl compose reconcile --force
39+
sitectl compose reconcile --reset-init
40+
```
41+
42+
The reconcile cache is local to the workstation and expires after 7 days. `--force` ignores the cache and reruns build/up. `--reset-init` removes plugin-declared init artifacts and init volumes, clears the cache entry, then reconciles again.
43+
44+
The reconcile step is a synchronous CLI workflow for one local <Compose /> project. It borrows condition names such as `Initialized` and `ImagesAvailable`, but it is not a Kubernetes controller or background loop.
45+
46+
## Clean
47+
48+
Use `sitectl compose clean` when you need to throw away local runtime state and return to a fresh init/build flow:
49+
50+
```bash
51+
sitectl compose clean
52+
sitectl compose clean --yes
53+
```
54+
55+
`clean` runs `docker compose down -v`, removes plugin-declared init artifacts such as generated secrets, certs, UID markers, and env files, and clears the reconcile cache entry. It is destructive and prompts for a typed confirmation unless `--yes` is passed.
56+
57+
See [Compose reconcile contract](/contributing/compose-reconcile) for the contributor-facing contract.
1658

1759
## Reference
1860

1961
<ComposeCmd />
62+
63+
<ComposeReconcileCmd />
64+
65+
<ComposeCleanCmd />

commands/create.mdx

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,3 +50,26 @@ sitectl create drupal/default --path ./drupal --context staging
5050
Plugin-specific create flags (for example `--path`, `--type`, `--checkout-source`,
5151
`--setup-only`, or ISLE's `--fcrepo` and `--blazegraph`) still go after the target,
5252
because they belong to the plugin's create runner.
53+
54+
## Setup-Only and First Start
55+
56+
`--setup-only` creates the local checkout and context without starting the stack. A fresh template checkout may still run clone-time init. For plugin-owned local contexts, the first later `sitectl compose up` checks the plugin's declared init artifacts and images, then runs any remaining init or build commands before starting Compose:
57+
58+
```bash
59+
sitectl create ojs/default --path ./ojs --type local --checkout-source template --setup-only
60+
sitectl compose up
61+
```
62+
63+
That handoff uses the plugin create definition as desired lifecycle metadata. See [Compose reconcile contract](/contributing/compose-reconcile) for contributor details.
64+
65+
## Image and Build Overrides
66+
67+
Compose template create runners can accept image override flags:
68+
69+
```bash
70+
sitectl create wp/default --path ./wp --type local --tag wp=nginx-1.30.3-php84
71+
sitectl create ojs/default --path ./ojs --type local --image ojs=ghcr.io/example/ojs:pr-123
72+
sitectl create drupal/default --path ./drupal --type local --build-arg drupal.BASE_IMAGE=libops/drupal:php84
73+
```
74+
75+
These flags write the same local `docker-compose.override.yml` file used by [`sitectl image set`](/commands/image). Use `sitectl image set` later when you need to change image or build-arg overrides for an existing local context.

commands/healthcheck.mdx

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,13 @@ context's plugin registers a healthcheck handler, its runtime checks run too and
1212
are merged into the report. The command exits non-zero if any check still fails
1313
after the timeout, so it is safe to use in scripts and CI.
1414

15+
Application plugins use shared helpers for common web stacks: route checks, MariaDB
16+
ping checks, optional HTTP service checks, and the expected `depends_on:
17+
condition: service_healthy` relationship between the app container and MariaDB.
18+
For route checks, localhost-style URLs are tested from the host first; app-domain
19+
URLs can be checked from inside the app container with the correct HTTP `Host`
20+
header.
21+
1522
## Reference
1623

1724
<HealthcheckCmd />

commands/image.mdx

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
title: image
3+
description: Manage local Docker Compose image and build-arg overrides.
4+
---
5+
6+
import { Compose } from "/snippets/compose-tooltip.mdx";
7+
import ImageCmd from "/snippets/commands/sitectl-image.mdx";
8+
import ImageClearCmd from "/snippets/commands/sitectl-image-clear.mdx";
9+
import ImageSetCmd from "/snippets/commands/sitectl-image-set.mdx";
10+
11+
The `image` command manages local <Compose /> image overrides for the active `sitectl` context.
12+
13+
Use `sitectl image set` when you want a local site to run a different image tag, image reference, or build argument without editing the upstream template files directly.
14+
15+
```bash
16+
sitectl image set --tag wp=nginx-1.30.3-php84
17+
sitectl image set --image app=ghcr.io/example/app:pr-123
18+
sitectl image set --build-arg app.BASE_IMAGE=libops/app:php84
19+
```
20+
21+
`sitectl image set` currently requires a local context with a project directory. It writes `docker-compose.override.yml` in that project.
22+
23+
Use `sitectl image clear` to remove image and build-arg override keys without deleting unrelated local override content:
24+
25+
```bash
26+
sitectl image clear
27+
sitectl image clear wp
28+
```
29+
30+
With no service arguments, `clear` removes image/build-arg overrides for every service in the override file. With service arguments, it clears only those services.
31+
32+
## Flags
33+
34+
| Flag | Format | Use |
35+
|---|---|---|
36+
| `--tag` | `SERVICE=TAG` | Set a known LibOps service to a published LibOps tag. |
37+
| `--image` | `SERVICE=IMAGE` | Set any Compose service to an explicit image reference. |
38+
| `--build-arg` | `SERVICE.ARG=VALUE` | Set a Compose build argument for a service. |
39+
40+
`--tag` only works for services known to sitectl's shared LibOps image map. For app-specific services, use `--image` or `--build-arg`.
41+
42+
## Compose Reconcile
43+
44+
Image overrides participate in the first-run `sitectl compose up` reconcile contract:
45+
46+
- an explicit image override for a service means core does not require the plugin's default image to exist locally
47+
- a build-arg override triggers the build phase so the new arguments are applied
48+
- the override file fingerprint is part of the local reconcile cache key
49+
50+
After changing overrides, run:
51+
52+
```bash
53+
sitectl compose up
54+
```
55+
56+
If you clear a build-arg override and want to rebuild with the default image metadata immediately, run:
57+
58+
```bash
59+
sitectl image clear wp
60+
sitectl compose reconcile --force
61+
```
62+
63+
See [Compose reconcile contract](/contributing/compose-reconcile) for the contributor-facing contract.
64+
65+
## Reference
66+
67+
<ImageCmd />
68+
69+
<ImageSetCmd />
70+
71+
<ImageClearCmd />

contributing/app-template.mdx

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
---
2+
title: Application plugin template
3+
description: How to build a new Compose-backed sitectl application plugin from sitectl-app-tmpl.
4+
---
5+
6+
import { Compose } from "/snippets/compose-tooltip.mdx";
7+
8+
Use [`sitectl-app-tmpl`](https://github.com/libops/sitectl-app-tmpl) when adding a new application plugin for a standalone <Compose /> template repository.
9+
10+
The template is intentionally thin. The application repository owns the Compose workload. Core `sitectl` owns shared lifecycle commands. The plugin contributes app-specific metadata and helpers.
11+
12+
## Command Ownership
13+
14+
| Concern | Owner | Examples |
15+
|---|---|---|
16+
| Compose lifecycle | Core `sitectl` | `sitectl compose up`, `sitectl compose down`, `sitectl compose logs`, `sitectl compose ps` |
17+
| Create flow metadata | App plugin | `CreateSpec`, template repo, init artifacts, image specs |
18+
| App-specific commands | App plugin | `sitectl <app> exec ...`, API wrappers, application maintenance helpers |
19+
| Shared service operations | Core `sitectl` | `sitectl mariadb backup`, `sitectl solr info`, `sitectl traefik tls` |
20+
| Cross-cutting reports | Core command plus plugin runner | `sitectl debug`, `sitectl validate`, `sitectl healthcheck` |
21+
22+
New app plugins should usually call `RegisterComposeTemplateCreateRunner`, not `RegisterStandardComposeTemplate`. The latter still exists for plugins that deliberately want direct namespace lifecycle commands, but first-class app plugins should let core `sitectl compose` provide the shared lifecycle surface.
23+
24+
## Create Definition Contract
25+
26+
The create definition is more than a clone recipe. It is also the desired state metadata used by core `sitectl compose up` to decide whether a local checkout needs first-run init or image build work. See [Compose reconcile contract](/contributing/compose-reconcile) for the shared core behavior.
27+
28+
```go
29+
func createDefinition() plugin.CreateSpec {
30+
return plugin.CreateSpec{
31+
Name: "default",
32+
Default: true,
33+
DockerComposeRepo: TemplateRepo,
34+
DockerComposeBranch: TemplateBranch,
35+
DockerComposeInit: []string{
36+
"if [ ! -f .env ]; then cp sample.env .env; fi",
37+
"docker compose run --rm init",
38+
},
39+
InitArtifacts: []plugin.InitArtifact{
40+
{Path: ".env"},
41+
{Path: "secrets/DB_ROOT_PASSWORD"},
42+
{Path: "secrets/APP_DB_PASSWORD"},
43+
},
44+
InitVolumes: []plugin.InitVolume{
45+
{Name: "mariadb-data"},
46+
},
47+
DockerComposeBuild: []string{
48+
"docker compose pull --ignore-buildable",
49+
"docker compose build --pull",
50+
},
51+
Images: []plugin.ComposeImageSpec{
52+
{
53+
Service: "app",
54+
Image: "libops/app:local",
55+
BuildPolicy: plugin.BuildPolicyIfNotPresent,
56+
},
57+
},
58+
DockerComposeUp: []string{
59+
"docker compose up --remove-orphans -d",
60+
},
61+
DockerComposeDown: []string{"docker compose down"},
62+
DockerComposeRollout: []string{"./scripts/rollout.sh"},
63+
}
64+
}
65+
```
66+
67+
`DockerComposeInit` should be idempotent. It may create `.env`, run an `init` service, and write deterministic secret files. It must not destroy existing application data when run again.
68+
69+
`InitArtifacts` should list files that prove init has completed. Prefer explicit files over probing container state. Use `ValueFrom: plugin.InitArtifactValueFromHostUID` for a UID marker file when generated files need to match the local host user.
70+
71+
`InitVolumes` should list named Compose volumes that prove first-start runtime state exists, such as database volumes and application file/upload volumes. Core resolves those names through `docker compose config`, so custom Compose project names and explicit volume names still work.
72+
73+
`Images` should list local images the template expects to build. Use `BuildPolicyIfNotPresent` for normal local images, `BuildPolicyAlways` only when every `compose up` should rebuild, and `BuildPolicyNever` for services that should never trigger the build phase.
74+
75+
## Image Overrides
76+
77+
Core `sitectl image set` writes `docker-compose.override.yml` for local contexts:
78+
79+
```bash
80+
sitectl image set --tag mariadb=11.4
81+
sitectl image set --image app=ghcr.io/example/app:pr-123
82+
sitectl image set --build-arg app.BASE_IMAGE=libops/app:php84
83+
```
84+
85+
Image overrides affect reconcile:
86+
87+
- an explicit image override for a service means core does not require the plugin's default image to exist locally
88+
- a build-arg override triggers the build phase so the new arguments are applied
89+
- the override file fingerprint is part of the reconcile cache key
90+
91+
Use `--image` or `--build-arg` for app-specific services that are not known to the shared `--tag` map.
92+
93+
## Template Checklist
94+
95+
After creating a repository from `sitectl-app-tmpl`:
96+
97+
1. Rename the Go module, binary, plugin name, display name, and release workflow outputs.
98+
2. Set `TemplateRepo`, `TemplateBranch`, `DefaultPath`, `AppService`, `AppImage`, `DatabaseService`, `DatabaseName`, and codebase rootfs constants.
99+
3. Update `CreateSpec` lifecycle commands, `InitArtifacts`, `InitVolumes`, and `Images` to match the real template repository.
100+
4. Tune project discovery in `SetComposeProjectDiscovery`. Use service names and durable codebase markers that identify the real project.
101+
5. Keep app-specific commands in the plugin namespace. Avoid registering `build`, `init`, `up`, `down`, `status`, `logs`, or `rollout` unless the app has a deliberate compatibility need.
102+
6. Use core service commands for shared operations. For example, document `sitectl mariadb backup app` rather than adding a thin app plugin alias.
103+
7. Implement debug, validate, and healthcheck runners through the SDK. Results should be structured; progress and diagnostics should go to stderr.
104+
8. Keep local workspace wiring in `go.work`; do not add sibling-module `replace` directives to `go.mod`.
105+
9. Update the integration script to create with `--setup-only`, start with `sitectl compose up`, and then run `sitectl healthcheck`.
106+
10. Add a plugin page to `sitectl-docs` for app-specific operations.
107+
108+
## Local Verification
109+
110+
Run the standard local workflow:
111+
112+
```bash
113+
make work
114+
make test
115+
make install
116+
```
117+
118+
Then verify the installed plugin can be discovered and used through core commands:
119+
120+
```bash
121+
sitectl create app-tmpl/default --path ./app --type local --checkout-source template --setup-only
122+
sitectl compose up
123+
sitectl healthcheck
124+
sitectl app-tmpl exec -- printenv
125+
```

0 commit comments

Comments
 (0)