Skip to content

Commit ff023e7

Browse files
committed
docs(plan): tiebreaker rule alignment + quorum auto-toggle
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
1 parent 4163d00 commit ff023e7

1 file changed

Lines changed: 4 additions & 4 deletions

File tree

PLAN.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -413,7 +413,7 @@ The phases above closed the MVP slice and the csi-sanity REST contract. A deep a
413413
- [x] **`linstor node evacuate` actually migrates replicas** (2026-05-09). New `internal/controller.NodeReconciler` watches Node CRDs and on EVICTED enumerates every Resource on the affected node, runs the shared `pkg/placer.Place` to create a replacement on a non-disabled peer (honouring the parent RG's topology constraints), and leaves the source replica in place — the operator decides when to remove it (typically once the replacement is UpToDate). The placer's "existing replicas" count now excludes EVICTED/LOST nodes so a 2-replica RD with one evicted source actually triggers the migration. Test: `TestNodeReconciler_EvictedTriggersMigration` pins the 3-node migration path.
414414
- [x] **`linstor node lost` recovery** (2026-05-09): the same NodeReconciler also detects LOST. Migration runs as for EVICTED, then the source Resource CRD is deleted via the K8s API path so the Resource controller's finalizer cleans up. The TCP-port/node-id allocations stored on the source Resource Status free naturally on delete (the per-node port allocator scans live Resources). Test: `TestNodeReconciler_LostDeletesSourceResource`. **e2e** (hard-kill a satellite pod) sits on the 8.8 checklist.
415415
- [x] **auto-diskful** (2026-05-09): the ResourceReconciler now promotes a DISKLESS replica to diskful when `Resource.Status.InUse=true` AND the hosting node has a viable storage pool. Removes the DISKLESS flag and stamps `StorPoolName` on Spec.Props; the satellite reconciler picks up the change on its next pass and creates the LV / runs `drbdadm attach`. TIE_BREAKER witnesses are exempted — promoting one would defeat the quorum-only purpose. Tests: `TestAutoDisklessPromoted`, `TestAutoDisklessSkipsTiebreaker`, `TestAutoDisklessSkipsWhenNoPool`. **`auto-diskful-cleanup`** (demote-on-idle) deferred — needs hysteresis to avoid flapping on transient opens; operators can demote manually via `linstor r d` until the access-pattern tracking lands.
416-
- [x] **Tiebreaker auto-creation** (2026-05-09). New `internal/controller.ResourceDefinitionReconciler` watches RDs; an RD with exactly 2 diskful replicas auto-gains a 3rd DISKLESS replica on a healthy non-replica node so DRBD's `quorum:majority` always has a majority on a 1-vs-1 partition. Idempotent (re-running on a 3-replica or already-witnessed RD is a no-op), respects EVICTED/LOST node flags, picks deterministically (lowest-name first) so reconcile races converge. The witness carries the `TIE_BREAKER` flag for cleanup tracking. Tests: `TestTiebreakerCreated`, `TestTiebreakerSkipsThreeReplicas`, `TestTiebreakerSkipsTwoNodeCluster`, `TestTiebreakerSkipsEvictedNode`.
416+
- [x] **Tiebreaker auto-creation + quorum auto-toggle** (2026-05-09, upstream-aligned). `internal/controller.ResourceDefinitionReconciler` watches RDs and Resource events (Watches+EnqueueRequestsFromMapFunc), mirrors upstream LINSTOR's two distinct rules: `CtrlRscAutoTieBreakerHelper.shouldTieBreakerExist` — create a TIE_BREAKER witness iff `diskful ≥ 2 ∧ diskful%2 == 0 ∧ non-witness-diskless == 0` — and `CtrlRscAutoQuorumHelper.isQuorumFeasible` — `(diskful == 2 ∧ diskless ≥ 1) ∨ diskful ≥ 3`. The reconciler stamps `DrbdOptions/Resource/quorum=majority` when feasible, `=off` otherwise (so a 50/50 split doesn't deadlock both halves). Idempotent (an already-witnessed RD is a no-op; a witness on an evicted node gets dropped + recreated elsewhere). Tests: `TestTiebreakerCreated`, `TestTiebreakerSkipsThreeReplicas`, `TestTiebreakerSkipsTwoNodeCluster`, `TestTiebreakerSkipsEvictedNode`, `TestTiebreakerEvenWithDiskless` (user-added diskless suppresses witness), `TestTiebreakerEvenAfterUserAdds4` (4-replica → witness lands), `TestTiebreakerRemovedWhenParityFlips`, `TestTiebreakerSinglesAreLeftAlone`. Stand-side e2e (`tests/e2e/tiebreaker.sh`) green.
417417
- [x] **Resource activate / deactivate** (2026-05-09): `POST /v1/resource-definitions/{rd}/resources/{node}/{activate,deactivate}` toggles the `INACTIVE` flag on the Resource. Idempotent. Satellite reconciler reads the flag and runs `drbdadm down` (deactivate) or normal apply (activate) — the .res file, port, and node-id allocations all stay intact, so flipping back doesn't lose state. Tests: `TestResourceDeactivate` (idempotent set + clear), `TestResourceActivateUnknown` (404 on missing replica).
418418
- [x] **Diskless replicas as first-class autoplace candidates** (2026-05-09): `AutoSelectFilter.DisklessOnRemaining` now actually does what the field name promises. After diskful place_count is satisfied, the placer creates DISKLESS replicas on every healthy node not already hosting a replica — the upstream "cluster-wide attachable" pattern useful for consumers that need to mount on any node. Test: `TestAutoplaceDisklessOnRemaining` (4-node cluster, place_count=2 → 2 diskful + 2 diskless witnesses).
419419

@@ -426,9 +426,9 @@ The phases above closed the MVP slice and the csi-sanity REST contract. A deep a
426426
### 8.5 Operator surface
427427

428428
- [x] **`linstor physical-storage` / `create-device-pool`** (2026-05-09): listed as **explicitly out-of-scope** for cozystack — pools are provisioned via Talos extensions / static node config, not at runtime. The list endpoints (cluster + per-node) return 200 with `[]`; the create endpoint returns 501 with a LINSTOR-shaped ApiCallRc explaining the boundary. Without the stubs, piraeus-operator's `LinstorSatelliteConfiguration.spec.storagePools` retry loop would 404 indefinitely. Tests: `TestPhysicalStorageList`, `TestPhysicalStorageCreateNotImplemented`.
429-
- [ ] **HA failover layer (drbd-reactor + piraeus-ha-controller)**. Two cooperating components, both required for cozystack's RWX (Ganesha) + Postgres failover (Patroni) flows; we conflated them in the original audit. Status:
430-
- `drbd-reactor` — local daemon on each satellite, listens to `drbdsetup events2` and triggers systemd promoter units on Primary-acquired (e.g. `nfs-ganesha@<rd>.service`). Cozystack already ships it via Talos / DaemonSet; the only blockstor-side concern is whether operators want the satellite to render `/etc/drbd-reactor.d/<rd>.toml` per-RD or keep that static. Decision needed; default is to leave reactor config out of the satellite's responsibility (operators ship it via ConfigMap / extension).
431-
- `piraeus-ha-controller` ([upstream](https://github.com/piraeusdatastore/piraeus-ha-controller))separate Kubernetes-aware controller. Watches Pod / Node / LINSTOR-resource state and does the cluster-aware actions: pod eviction on DRBD `StandAlone`, force-detach when a backing device fails, mount-node migration when the consumer Pod reschedules. It dials the LINSTOR REST API as a client. Verify against blockstor: end-to-end run with piraeus-ha-controller against our REST surface, document which endpoints it hits, fix any gaps. No code changes from blockstor expected unless an endpoint shape diverges from upstream; the audit needs to happen on the dev stand.
429+
- [x] **HA failover layer (drbd-reactor + piraeus-ha-controller)** (2026-05-09 audited on stand): two cooperating components, decoupled from blockstor's REST surface. No blockstor-side gaps surfaced by the audit.
430+
- `drbd-reactor` — local daemon on each satellite, listens to `drbdsetup events2` and triggers systemd promoter units (e.g. `nfs-ganesha@<rd>.service`) on Primary-acquired. Cozystack ships it via the same Talos extension that ships drbd-utils; blockstor renders `quorum:majority` by default into the .res file (DRBD-9 default, no override) so the reactor's quorum-loss path fires. Per-RD reactor toml stays the operator's responsibility (ConfigMap / Talos overlay), NOT something the satellite renders.
431+
- `piraeus-ha-controller` v1.3.1 (DaemonSet on every worker)verified on stand against blockstor: it does NOT call the LINSTOR REST API at all. Architecture: it watches Kubernetes Pod / Node objects + reads node taints (`drbd.linbit.com/force-io-error`, `drbd.linbit.com/lost-quorum`) that drbd-reactor stamps on quorum loss. On taint, it evicts pods that are using affected DRBD resources. Stand-side ha-controller logs show only `starting reconciliation` heartbeats — zero traffic to blockstor. Contract is purely K8s-state-based: as long as drbd-reactor + DRBD's quorum behave, blockstor doesn't need to do anything REST-side. No code changes required.
432432
- [x] **`linstor advise`** (2026-05-09): `GET /v1/view/advise/resources` and `GET /v1/resource-definitions/{rd}/advise` return per-RD recommendations (top-N pools by free capacity, sorted desc) without persisting anything. Surfaces a `Conflict` string when the request can't be satisfied so the CLI prints it. Tests: `TestAdviseRD`, `TestAdviseRDInsufficient`.
433433
- [x] **`linstor query-size-info` / spaceinfo** (2026-05-09): `POST /v1/resource-groups/{rg}/query-size-info` answers `max_vlm_size_in_kib = FreeCapacity of the n-th-largest pool` (n = place_count) — the cap that all replicas can fit at once, the value golinstor's pre-flight uses. `POST /v1/query-all-size-info` returns the per-RG map in one shot. EVICTED/LOST nodes excluded from capacity. Tests: 3 cases covering happy path, exhausted, and the cluster-wide aggregate.
434434
- [x] **shared LUN provider (EXOS / SHARED) architectural hooks** (2026-05-09): `StoragePool` (CRD spec + REST shape) gains an optional `SharedSpaceID` field (empty = local pool). `pkg/placer` tracks `sharedSeen` alongside the existing `diffSeen` and rejects any candidate whose `SharedSpaceID` matches an already-placed replica's pool — a 2-replica RD will never land on two pools that physically share the same backing LUN. `pkg/rest/query_size_info.dedupShared` collapses pool list to one representative per shared LUN before computing capacity totals and the n-th-largest free figure, so two satellites each "seeing" 1000 KiB of the same LUN contribute 1000, not 2000. `pkg/rest/advise` runs the same dedup so recommendations match what the placer would actually accept. Tests: `TestAutoplaceSharedLUNAntiAffinity`, `TestAutoplaceSharedLUNExhausted`, `TestQuerySizeInfoSharedLUN`. The actual SAN-attached-LUN provider implementation is still a follow-up — when it lands, it'll set `SharedSpaceID` on its `StoragePool` CRDs and the placer/advice/query-size-info paths will already do the right thing.

0 commit comments

Comments
 (0)