Skip to content
Open
68 changes: 56 additions & 12 deletions docs/oabctl.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,11 @@ spec:
securityGroups: [sg-xxx]
```

`metadata.namespace` must be hyphen-free (`[a-z0-9]+`); `metadata.name` may
contain `-`. Physical resources are named `oab-{namespace}-{name}`, and a
hyphen-free namespace keeps that identity unambiguous (see the teardown note
below for the legacy-namespace policy).

### Ingress — inbound webhooks (Telegram / LINE)

Discord bots are outbound-only and need no ingress. Webhook platforms (Telegram,
Expand Down Expand Up @@ -365,18 +370,57 @@ must still be registered manually in the LINE Developers console.
> reminded, not verified).
>
> **Teardown:** `oabctl delete oabservice <name>` (or `oabctl delete -f <manifest>`,
> using the same file `apply -f` deployed it from) permanently removes the bot's
> per-bot ingress resources — its exact Cloud Map service (resolved by the ECS
> service's own registry ARN, not a name search, so same-named bots in different
> VPCs/environments can't collide) and its HTTP API (`oab-webhook-<ns>-<name>`,
> including the API resource itself this time, since the bot is gone for good) —
> on a best-effort basis (it never blocks service deletion). If you instead edit
> a manifest to remove `spec.ingress` while keeping the bot, `apply` runs the same
> Cloud Map + routes/integration/stage cleanup automatically, but **keeps the HTTP
> API** in case ingress is re-added later. The **shared** VPC Link and the
> security-group inbound rule are always left in place for other bots. If the
> Cloud Map service still has registered instances, teardown retries for ~25s
> before falling back to a warning with the manual cleanup command.
> using the same file `apply -f` deployed it from) first captures the exact ECS
> cluster/service ARNs, ECS registry ARN, matching HTTP API ID, configured
> control-plane bucket, caller partition/account/region, and ECS service
> incarnation (`createdAt`) in `delete-checkpoints/<namespace>/<name>.json`. The
> checkpoint is written before ECS mutation and removed only after all
> dependent and S3 cleanup succeeds, so rerunning the same command safely
> resumes a partial delete. ECS HTTP-200 failures, ambiguous drain responses,
> empty service responses, and a missing service fail closed; a matching checkpoint
> authorizes retry only when ECS explicitly reports exactly one `MISSING`
> failure with zero services, or an `INACTIVE` response from the original
> service incarnation.
>
> API cleanup never selects the first same-named API: duplicate names fail
> closed, and the sole candidate is checkpointed only when its integration URI
> equals the ECS registry ARN. Cloud Map is deleted only by the service ID
> parsed from a structurally and boundary-validated Cloud Map service ARN.
> Exact-ID NotFound is idempotent; other cleanup errors retain the checkpoint.
> There is no name-only API, Cloud Map, or S3 orphan cleanup path. Apply and
> delete share one injective identity rule: namespaces must not contain `-`
> (names may contain it), so the physical `oab-{namespace}-{name}` identity
> always maps back to exactly one `namespace`/`name` pair. Manifest validation
> and programmatic delete reject hyphenated namespaces before any AWS call,
> and before mutating anything both apply and delete verify in the control
> plane that no other recorded logical pair (for example legacy `prod-team/bot`
> versus `prod/team-bot`) claims the same physical name — contested identities
> fail closed. Legacy deployments created under a hyphenated namespace cannot
> be re-applied, but `oabctl delete` still accepts them (with a warning) under
> that same ownership check; migrate by deleting and re-creating them under a
> hyphen-free namespace. If a per-target
> `ingress-teardown-checkpoints/<namespace>/<name>.json` record exists, delete
> fails before destructive mutation so it cannot discard unfinished exact
> cleanup identity. Re-run the ingress-free apply to completion, then retry
> delete.
>
> Programmatic apply and delete do not provide an internal same-target lock.
> Callers must serialize mutations for the same AWS account, Region,
> control-plane bucket, ECS cluster, namespace, and name. After an accidental
> overlap, stop concurrent writers, inspect retained checkpoints, then
> explicitly re-apply the desired state or retry delete.
>
> If you edit a manifest to remove `spec.ingress` while keeping the bot, `apply` first
> stores every exact ECS registry ARN only when the previously stored OAB
> manifest owned ingress (or resumes an already-valid checkpoint); an arbitrary
> attached registry is never enough.
> `ingress-teardown-checkpoints/<namespace>/<name>.json`, clears only API wiring
> bound to those ARNs, detaches all registries from ECS, waits until the detach
> is observable, and only then deletes each exact Cloud Map service. It keeps
> the HTTP API resource so its URL can survive re-enabling ingress and removes
> the checkpoint only after the full apply succeeds, so retries cannot lose
> cleanup identity. The **shared** VPC Link and security-group inbound rule are
> always left in place for other bots.
>
> **Changing `paths`:** `apply` prunes routes on the bot's API that are no longer
> in the manifest's `ingress.paths`, so renaming or removing a webhook path never
Expand Down
80 changes: 66 additions & 14 deletions operator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,16 +75,19 @@ commands reference.

## Library API

The crate exposes a deliberately narrow manifest + apply facade for control
planes that should not shell out to the CLI:
The crate exposes a deliberately narrow manifest + apply/delete facade for
control planes that should not shell out to the CLI:

```rust,no_run
use oabctl::{apply_manifests, ApplyOptions, OABServiceManifest};
use oabctl::{
apply_manifests, delete_services, ApplyOptions, DeleteOptions, DeleteTarget,
OABServiceManifest,
};

async fn deploy(
async fn reconcile(
aws: &aws_config::SdkConfig,
manifest: OABServiceManifest,
) -> Result<(), oabctl::ApplyError> {
) -> Result<(), Box<dyn std::error::Error>> {
let report = apply_manifests(
aws,
&[manifest],
Expand All @@ -94,20 +97,69 @@ async fn deploy(
for service in report.services {
println!("{}: {:?}", service.ecs_service_name, service.action);
}

delete_services(
aws,
&[DeleteTarget::new("prod", "bot")],
&DeleteOptions::new("production-cluster")
.with_control_plane_bucket("my-control-plane-bucket"),
)
.await?;
Ok(())
}
```

Programmatic apply emits no progress to process-global stdout/stderr. Success
returns per-service actions, webhook URLs, and warnings; reconciliation errors
identify the failed service and include the report completed before the failure.
Both CLI and programmatic apply verify that the target cluster exists and is
`ACTIVE` before mutation, so the caller identity requires
`ecs:DescribeClusters`. This ECS action does not support resource-level
Programmatic apply/delete emit no progress to process-global stdout/stderr.
Apply success returns per-service actions, webhook URLs, and warnings;
reconciliation errors identify the failed service and include the report
completed before the failure. Both CLI and programmatic apply verify that the
target cluster exists and is `ACTIVE` before mutation, so the caller identity
requires `ecs:DescribeClusters`. This ECS action does not support resource-level
permissions; its IAM statement must use `Resource: "*"`.
The library never reads `~/.oabctl/config.toml`: set
`with_control_plane_bucket(...)` explicitly when needed, otherwise bucket
resolution uses `OAB_CONTROL_PLANE_BUCKET` and then the caller's AWS account.

The library never reads `~/.oabctl/config.toml`. Apply and delete resolve the
control-plane bucket through the shared chain: an explicit
`with_control_plane_bucket(...)` override, then `OAB_CONTROL_PLANE_BUCKET`, then
`oab-control-plane-{account}` from the caller identity. Before deleting ECS,
`delete_services` stores the resolved bucket, caller partition/account/region,
canonical ECS cluster/service ARNs, the service `createdAt` incarnation, and
exact ingress IDs in `delete-checkpoints/<namespace>/<name>.json`. The checkpoint
is written before ECS mutation and removed last. Initial DescribeServices
responses must identify exactly one expected live service; missing, empty-success,
ambiguous, or mixed-failure responses fail closed. Retries use only checkpointed
IDs after ECS explicitly reports exactly one `MISSING` failure with zero services
or a matching original `INACTIVE` incarnation. Duplicate APIs fail closed,
and a sole same-name API is checkpointed only when an integration URI exactly
matches the ECS registry ARN.
There is no name-only API, Cloud Map, or S3 orphan cleanup fallback. If a
per-target apply-side `ingress-teardown-checkpoints/<namespace>/<name>.json`
record exists, delete fails before destructive mutation so it cannot discard
unfinished exact cleanup identity. Re-run the ingress-free apply to completion,
then retry delete.

Apply and delete share one injective logical-identity rule: namespaces must not
contain `-`; names may contain it, so the physical `oab-{namespace}-{name}`
identity always parses back to exactly one `namespace`/`name` pair. Manifest
validation (CLI apply, fleet expansion, and `apply_manifests`) and
`delete_services` all reject hyphenated namespaces before any AWS call. Before
mutating, apply and every delete entry point additionally verify in the
control plane that no *other* recorded logical pair (manifest, delete
checkpoint, or ingress-teardown checkpoint) claims the same physical name, and
fail closed with the colliding pair named in the error.

Legacy policy: deployments created under a hyphenated namespace before this
rule cannot be re-applied — apply fails with a migration message. They remain
deletable through `oabctl delete`, which accepts a hyphenated namespace for
teardown, prints a warning, and relies on the same control-plane ownership
check to refuse contested physical names. Migrate by deleting the legacy
deployment and re-creating it under a hyphen-free namespace.

Programmatic apply and delete do not provide an internal same-target lock.
Callers must serialize mutations for the same AWS account, Region, control-plane
bucket, ECS cluster, namespace, and name. If overlapping calls occur, stop
concurrent writers, inspect the retained checkpoint, then explicitly re-apply
the desired state or retry delete.

For `aws-sm://<secret-id>#<json-key>`, a non-ARN `<secret-id>` requires the
caller to have `secretsmanager:DescribeSecret`; full-ARN shorthand does not
need that lookup.
Expand Down
4 changes: 2 additions & 2 deletions operator/schema/oabservice-v2.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
"required": ["name", "namespace"],
"properties": {
"name": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*$" },
"namespace": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*$" },
"namespace": { "type": "string", "pattern": "^[a-z0-9]+$" },
"generation": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
Expand All @@ -45,7 +45,7 @@
"required": ["name", "namespace"],
"properties": {
"name": { "type": "string" },
"namespace": { "type": "string" }
"namespace": { "type": "string", "pattern": "^[a-z0-9]+$" }
},
"additionalProperties": false
},
Expand Down
Loading
Loading