Skip to content

Commit 5cad2ca

Browse files
committed
feat(client): add Go SDK for hypercache-server clusters
Introduce pkg/client — a typed Go client for hypercache-server that closes three operational gaps surfaced by the OIDC-client example: - Multi-endpoint HA: New() accepts a seed slice; each request picks an endpoint at random and falls back to the next on transport failure, 5xx, or 503 draining. 4xx answers are deterministic and do not trigger failover. - Optional topology refresh: WithTopologyRefresh(interval) runs a background loop that pulls /cluster/members and updates the in-memory endpoint view, so nodes added or removed after deploy become reachable without redeploying consumers. Seeds remain as a permanent fallback when the live view empties. - Four auth modes in one API: WithBearerAuth, WithBasicAuth, WithOIDCClientCredentials (auto-refreshing OAuth2 client-credentials flow), and WithHTTPClient (bring your own mTLS-configured client). Mutually exclusive; last applied wins. Typed error surface: ErrNotFound, ErrUnauthorized, ErrForbidden, ErrDraining, ErrBadRequest, ErrInternal, ErrAllEndpointsFailed, ErrNoEndpoints compose with errors.Is. *StatusError carries the canonical {code, error, details} envelope for callers that need finer discrimination via errors.As. Commands: Set, Get (raw bytes), GetItem (full envelope with version/owners), Delete, Identity (/v1/me canary), Endpoints, RefreshTopology, Close. Promote golang.org/x/oauth2 from indirect to direct dependency. Update cspell.config.yaml with new terms (errchkjson, httptest, mathrand, misrouted, unparseable). Full test coverage in pkg/client/client_test.go covering happy- path round-trip, every auth mode, 5xx failover, 4xx no-failover regression guard, exhaustive-failure wrapping, all sentinel errors.Is mappings, topology refresh, and constructor input validation.
1 parent 02f30c4 commit 5cad2ca

11 files changed

Lines changed: 2041 additions & 37 deletions

File tree

CHANGELOG.md

Lines changed: 53 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -8,51 +8,69 @@ All notable changes to HyperCache are recorded here. The format follows
88

99
### Added
1010

11-
- **HTTP Basic auth as a first-class credential class (Redis-style `AUTH user pass`).** New top-level
12-
`users:` block in `HYPERCACHE_AUTH_CONFIG` accepts bcrypt-hashed passwords. Each user resolves to the
13-
same `Identity{ID, Scopes}` shape as every other auth mode, so all four mechanisms (static bearer →
14-
Basic → mTLS → OIDC) coexist in one cluster with consistent downstream behavior. Fail-closed posture:
15-
Basic over plaintext is refused by default; operators opt into dev-only plaintext via
16-
`allow_basic_without_tls: true`. Implementation in
17-
[`pkg/httpauth/policy.go`](pkg/httpauth/policy.go) with bcrypt verification via
11+
- **`pkg/client` — Go SDK for hypercache-server clusters.** Closes the three operational gaps the OIDC-client
12+
example surfaced: - **Multi-endpoint HA without an external LB.** `client.New([]string{...}, opts...)`
13+
accepts a slice of seed endpoints. Each request picks one at random; on transport failure / 5xx / 503
14+
(draining) the client walks to the next. 4xx (auth, scope, not-found, bad-request) are deterministic and do
15+
NOT trigger failover. See [RFC 0003](docs/rfcs/0003-client-sdk-and-redis-style-affordances.md) for the
16+
failover policy rationale (F2 random with crypto-seeded math/rand). - **Optional topology refresh.**
17+
`WithTopologyRefresh(interval)` enables a background loop that pulls `/cluster/members` and updates the
18+
in-memory endpoint view, so nodes added or removed after deploy become visible without redeploying
19+
consumers. The original seeds remain as a permanent fallback when the live view ever empties. - **Four auth
20+
modes coexisting in one API.** `WithBearerAuth`, `WithBasicAuth`, `WithOIDCClientCredentials` (full OAuth2
21+
client-credentials flow with auto-refresh), and `WithHTTPClient` (bring your own mTLS-configured client).
22+
Mutually exclusive: the last applied wins. - **Stable, typed error surface.** Sentinels (`ErrNotFound`,
23+
`ErrUnauthorized`, `ErrForbidden`, `ErrDraining`, `ErrBadRequest`, `ErrInternal`, `ErrAllEndpointsFailed`,
24+
`ErrNoEndpoints`) compose with `errors.Is`. `*StatusError` carries the cache's canonical
25+
`{ code, error, details }` envelope for callers that need finer discrimination via `errors.As`. - **Typed
26+
command surface.** `Set`, `Get` (raw bytes), `GetItem` (full envelope with version/owners), `Delete`,
27+
`Identity` (the `/v1/me` canary including the new capabilities field), `Endpoints` (the current view),
28+
`RefreshTopology` (manual refresh for tests/operators), `Close`. - **Full test coverage** in
29+
[`pkg/client/client_test.go`](pkg/client/client_test.go): happy-path round-trip, JSON-envelope decode, every
30+
auth mode against httptest stubs, 5xx failover, 4xx no-failover (regression guard), exhaustive-failure
31+
wrapping, every sentinel's `errors.Is` mapping, topology refresh, partition-survives-empty-refresh failsafe,
32+
and constructor input validation.
33+
- **HTTP Basic auth as a first-class credential class (Redis-style `AUTH user pass`).** New top-level `users:`
34+
block in `HYPERCACHE_AUTH_CONFIG` accepts bcrypt-hashed passwords. Each user resolves to the same
35+
`Identity{ID, Scopes}` shape as every other auth mode, so all four mechanisms (static bearer → Basic → mTLS
36+
→ OIDC) coexist in one cluster with consistent downstream behavior. Fail-closed posture: Basic over
37+
plaintext is refused by default; operators opt into dev-only plaintext via `allow_basic_without_tls: true`.
38+
Implementation in [`pkg/httpauth/policy.go`](pkg/httpauth/policy.go) with bcrypt verification via
1839
`golang.org/x/crypto/bcrypt`. Threat note: bcrypt-per-request is CPU-bound; rate-limiting is left to a
1940
fronting LB (see [RFC 0003](docs/rfcs/0003-client-sdk-and-redis-style-affordances.md) open question 3).
20-
- **`/v1/me` now returns a `capabilities` field.** Stable capability strings derived 1:1 from scopes
21-
(`read``cache.read`, etc.). Clients should prefer `capabilities` over `scopes` for
22-
forward-compatibility: if a scope is later split into multiple capabilities, scope-keyed clients
23-
break but capability-keyed clients keep working. OpenAPI spec
24-
([`cmd/hypercache-server/openapi.yaml`](cmd/hypercache-server/openapi.yaml)) updated to reflect the
25-
new required field; the binary's embedded spec is the contract.
26-
- **Tests pinning the new auth contract.**
27-
[`pkg/httpauth/policy_test.go`](pkg/httpauth/policy_test.go) covers Basic resolves on correct
28-
credentials, rejects on wrong passwords/users/malformed headers, refuses plaintext by default, and
29-
documents the bearer-wins-over-Basic chain order via a Locals-introspection test.
41+
- **`/v1/me` now returns a `capabilities` field.** Stable capability strings derived 1:1 from scopes (`read`
42+
`cache.read`, etc.). Clients should prefer `capabilities` over `scopes` for forward-compatibility: if a
43+
scope is later split into multiple capabilities, scope-keyed clients break but capability-keyed clients keep
44+
working. OpenAPI spec ([`cmd/hypercache-server/openapi.yaml`](cmd/hypercache-server/openapi.yaml)) updated
45+
to reflect the new required field; the binary's embedded spec is the contract.
46+
- **Tests pinning the new auth contract.** [`pkg/httpauth/policy_test.go`](pkg/httpauth/policy_test.go) covers
47+
Basic resolves on correct credentials, rejects on wrong passwords/users/malformed headers, refuses plaintext
48+
by default, and documents the bearer-wins-over-Basic chain order via a Locals-introspection test.
3049
[`pkg/httpauth/loader_test.go`](pkg/httpauth/loader_test.go) covers the YAML round-trip plus the
3150
fail-loud-at-boot guards for malformed bcrypt hashes and empty usernames.
32-
- **Operator runbook updates.**
33-
[`docs/oncall.md`](docs/oncall.md) Auth failures section gains a Basic-auth debugging row covering
34-
the `curl -u user:pass /v1/me` canary and the plaintext-refused failure mode.
35-
- **Migration-source observability for the hint queue.** Hints produced by rebalance migrations are now
36-
tagged at queue time and tracked in a dedicated set of counters alongside the existing aggregate
37-
metrics. Five new OTel metrics: `dist.migration.queued`, `dist.migration.replayed`,
38-
`dist.migration.expired`, `dist.migration.dropped`, and `dist.migration.last_age_ns` (queue residency of
39-
the most-recently-replayed migration hint — direct signal of new-primary reachability during rolling
40-
deploys). Existing `dist.hinted.*` counters keep their meaning as the aggregate across both sources, so
41-
operators can derive replication-only as `aggregate - migration`. Implementation reuses the proven hint
42-
queue infrastructure (TTL, caps, replay, drop logic) — no second queue, no second drain loop.
43-
Tests in [`pkg/backend/dist_migration_hint_test.go`](pkg/backend/dist_migration_hint_test.go) cover
44-
source-tag preservation through queue→replay, per-source counter increments on every terminal path
45-
(replay success, expired, transport drop, global-cap drop), and the not-found keep-in-queue path.
51+
- **Operator runbook updates.** [`docs/oncall.md`](docs/oncall.md) Auth failures section gains a Basic-auth
52+
debugging row covering the `curl -u user:pass /v1/me` canary and the plaintext-refused failure mode.
53+
- **Migration-source observability for the hint queue.** Hints produced by rebalance migrations are now tagged
54+
at queue time and tracked in a dedicated set of counters alongside the existing aggregate metrics. Five new
55+
OTel metrics: `dist.migration.queued`, `dist.migration.replayed`, `dist.migration.expired`,
56+
`dist.migration.dropped`, and `dist.migration.last_age_ns` (queue residency of the most-recently-replayed
57+
migration hint — direct signal of new-primary reachability during rolling deploys). Existing `dist.hinted.*`
58+
counters keep their meaning as the aggregate across both sources, so operators can derive replication-only
59+
as `aggregate - migration`. Implementation reuses the proven hint queue infrastructure (TTL, caps, replay,
60+
drop logic) — no second queue, no second drain loop. Tests in
61+
[`pkg/backend/dist_migration_hint_test.go`](pkg/backend/dist_migration_hint_test.go) cover source-tag
62+
preservation through queue→replay, per-source counter increments on every terminal path (replay success,
63+
expired, transport drop, global-cap drop), and the not-found keep-in-queue path.
4664
- **Adaptive Merkle anti-entropy scheduling.** New
4765
[`backend.WithDistMerkleAdaptiveBackoff(maxFactor)`](pkg/backend/dist_memory.go) option lets the auto-sync
4866
loop double its sleep interval after each tick that finds zero divergence across every peer, capped at
4967
`maxFactor`. Any tick with at least one dirty peer snaps the factor back to 1× immediately — recovery is
50-
never lazy. Disabled by default (factor=0 or 1) so existing deployments see no behavior change. Two new
51-
OTel metrics expose the state: `dist.auto_sync.backoff_factor` (gauge) and `dist.auto_sync.clean_ticks`
68+
never lazy. Disabled by default (factor=0 or 1) so existing deployments see no behavior change. Two new OTel
69+
metrics expose the state: `dist.auto_sync.backoff_factor` (gauge) and `dist.auto_sync.clean_ticks`
5270
(counter). Each factor change is logged once at Info (`merkle auto-sync backoff factor changed`) — no
5371
per-tick log spam. Unit tests in
54-
[`pkg/backend/dist_adaptive_backoff_test.go`](pkg/backend/dist_adaptive_backoff_test.go) cover the ramp,
55-
the cap, the dirty-tick reset, and the disabled-by-default back-compat invariant.
72+
[`pkg/backend/dist_adaptive_backoff_test.go`](pkg/backend/dist_adaptive_backoff_test.go) cover the ramp, the
73+
cap, the dirty-tick reset, and the disabled-by-default back-compat invariant.
5674
- **Structured logging for background loops and cluster lifecycle.** HyperCache gained a
5775
`WithLogger(*slog.Logger)` option ([config.go](config.go)) that wires a structured logger through the
5876
wrapper. Previously the eviction loop, expiration loop, and HyperCache lifecycle ran fully silent —

cspell.config.yaml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,7 @@ words:
8989
- elif
9090
- Equalf
9191
- errcheck
92+
- errchkjson
9293
- errp
9394
- eventbus
9495
- ewrap
@@ -140,6 +141,7 @@ words:
140141
- hreq
141142
- htpasswd
142143
- httpauth
144+
- httptest
143145
- HTTPTLS
144146
- hypercache
145147
- Hyperd
@@ -165,6 +167,7 @@ words:
165167
- logrus
166168
- longbridgeapp
167169
- mailtos
170+
- mathrand
168171
- maxmemory
169172
- memprofile
170173
- Merkle
@@ -173,6 +176,7 @@ words:
173176
- mfinal
174177
- Mgmt
175178
- microbenchmark
179+
- misrouted
176180
- mkdocs
177181
- mrand
178182
- mset
@@ -220,8 +224,8 @@ words:
220224
- SFCKGPQ
221225
- shamaton
222226
- shellcheck
223-
- skeys
224227
- singleflight
228+
- skeys
225229
- SLRU
226230
- softprops
227231
- statefulset
@@ -253,6 +257,7 @@ words:
253257
- unmarshals
254258
- unpadded
255259
- unparam
260+
- unparseable
256261
- unsharded
257262
- unsub
258263
- unsubbed

go.mod

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ require (
1919
go.opentelemetry.io/otel/sdk/metric v1.43.0
2020
go.opentelemetry.io/otel/trace v1.43.0
2121
golang.org/x/crypto v0.51.0
22+
golang.org/x/oauth2 v0.36.0
2223
gopkg.in/yaml.v3 v3.0.1
2324
)
2425

@@ -41,7 +42,6 @@ require (
4142
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
4243
go.uber.org/atomic v1.11.0 // indirect
4344
golang.org/x/net v0.54.0 // indirect
44-
golang.org/x/oauth2 v0.36.0 // indirect
4545
golang.org/x/sys v0.44.0 // indirect
4646
golang.org/x/text v0.37.0 // indirect
4747
)

0 commit comments

Comments
 (0)