Skip to content

Commit 02f30c4

Browse files
committed
feat(httpauth): add HTTP Basic auth and capabilities to /v1/me
Introduce username/password authentication via `Authorization: Basic` as a first-class credential class alongside static bearer tokens, mTLS certs, and OIDC JWTs. The resolve chain is now: bearer → Basic → mTLS → OIDC. Changes: - pkg/httpauth/policy.go: new `BasicIdentity` struct (bcrypt-hashed passwords); `resolveBasic` resolver with fail-closed TLS posture (`AllowBasicWithoutTLS` defaults to false); `Capabilities()` on `Identity` derives stable `cache.<scope>` strings from scopes. - pkg/httpauth/loader.go: new `users:` YAML block in `fileSchema`; `AllowBasicWithoutTLS` config flag; boot-time `Validate()` rejects malformed bcrypt hashes and empty usernames loudly. - cmd/hypercache-server/main.go + openapi.yaml: `meResponse` and `IdentityResponse` gain a required `capabilities` field populated from `Identity.Capabilities()`; OpenAPI description updated to reflect all four auth modes. - __examples/distributed-oidc-client/: new runnable OIDC client-credentials demo (PUT/GET/DELETE/batch surface) with full README covering env-var setup, scope mapping patterns across Keycloak/Auth0/Okta, and production caveats. - docs/rfcs/0003-client-sdk-and-redis-style-affordances.md: new RFC proposing a Go client SDK with multi-endpoint HA, username/password auth, and structured typed errors; design options enumerated with recommended path. - Tests: Basic auth happy path, wrong-password/user, malformed header, and plaintext-refused-by-default cases in `policy_test.go`; YAML round-trip and boot-guard tests in `loader_test.go`; `me_test.go` updated for the new `capabilities` field. - go.mod: promote `golang.org/x/crypto` from indirect to direct; bump ewrap v1.5.0 → v1.5.1, sectools v1.2.5 → v1.2.6.
1 parent c7eae63 commit 02f30c4

22 files changed

Lines changed: 1893 additions & 49 deletions

CHANGELOG.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,30 @@ All notable changes to HyperCache are recorded here. The format follows
88

99
### Added
1010

11+
- **HTTP Basic auth as a first-class credential class (Redis-style `AUTH user pass`).** New top-level
12+
`users:` block in `HYPERCACHE_AUTH_CONFIG` accepts bcrypt-hashed passwords. Each user resolves to the
13+
same `Identity{ID, Scopes}` shape as every other auth mode, so all four mechanisms (static bearer →
14+
Basic → mTLS → OIDC) coexist in one cluster with consistent downstream behavior. Fail-closed posture:
15+
Basic over plaintext is refused by default; operators opt into dev-only plaintext via
16+
`allow_basic_without_tls: true`. Implementation in
17+
[`pkg/httpauth/policy.go`](pkg/httpauth/policy.go) with bcrypt verification via
18+
`golang.org/x/crypto/bcrypt`. Threat note: bcrypt-per-request is CPU-bound; rate-limiting is left to a
19+
fronting LB (see [RFC 0003](docs/rfcs/0003-client-sdk-and-redis-style-affordances.md) open question 3).
20+
- **`/v1/me` now returns a `capabilities` field.** Stable capability strings derived 1:1 from scopes
21+
(`read``cache.read`, etc.). Clients should prefer `capabilities` over `scopes` for
22+
forward-compatibility: if a scope is later split into multiple capabilities, scope-keyed clients
23+
break but capability-keyed clients keep working. OpenAPI spec
24+
([`cmd/hypercache-server/openapi.yaml`](cmd/hypercache-server/openapi.yaml)) updated to reflect the
25+
new required field; the binary's embedded spec is the contract.
26+
- **Tests pinning the new auth contract.**
27+
[`pkg/httpauth/policy_test.go`](pkg/httpauth/policy_test.go) covers Basic resolves on correct
28+
credentials, rejects on wrong passwords/users/malformed headers, refuses plaintext by default, and
29+
documents the bearer-wins-over-Basic chain order via a Locals-introspection test.
30+
[`pkg/httpauth/loader_test.go`](pkg/httpauth/loader_test.go) covers the YAML round-trip plus the
31+
fail-loud-at-boot guards for malformed bcrypt hashes and empty usernames.
32+
- **Operator runbook updates.**
33+
[`docs/oncall.md`](docs/oncall.md) Auth failures section gains a Basic-auth debugging row covering
34+
the `curl -u user:pass /v1/me` canary and the plaintext-refused failure mode.
1135
- **Migration-source observability for the hint queue.** Hints produced by rebalance migrations are now
1236
tagged at queue time and tracked in a dedicated set of counters alongside the existing aggregate
1337
metrics. Five new OTel metrics: `dist.migration.queued`, `dist.migration.replayed`,

__examples/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,3 +23,5 @@ All the code in this directory is for demonstration purposes only.
2323
1. [`Size`](./size/size.go) - An example of using the HyperCache package to store a list of items and limit the cache based on size.
2424

2525
1. [`Observability (OpenTelemetry)`](./observability/otel.go) - Demonstrates wrapping the service with tracing and metrics middleware using OpenTelemetry.
26+
27+
1. [`Distributed OIDC client`](./distributed-oidc-client/) - End-to-end demo of a backend service authenticating to a `hypercache-server` cluster via OIDC client credentials, then exercising the PUT/GET/DELETE/batch surface. See the example's [README](./distributed-oidc-client/README.md) for env-var setup and IdP integration patterns.
Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# Distributed cache client with OIDC auth
2+
3+
A runnable example showing a backend service connecting to a `hypercache-server` cluster, authenticating via
4+
OIDC client credentials, and exercising the full client API.
5+
6+
This is a **service-to-service** flow (no browser, no user redirect). The application proves its identity to
7+
the IdP with a client ID and secret, gets back a short-lived JWT, and presents that JWT to the cache. The
8+
cache validates the JWT against the same IdP and resolves the caller's identity + scopes. The same model fits
9+
Keycloak, Auth0, Okta, Entra ID, Google, and any RFC 6749 §4.4 compliant IdP.
10+
11+
## What the example does
12+
13+
1. Discovers the IdP's token endpoint from `$OIDC_ISSUER/.well-known/openid-configuration`.
14+
1. Uses `golang.org/x/oauth2/clientcredentials` to exchange the client ID + secret for an access token. The
15+
library caches the token in memory and transparently refreshes it before expiry — every cache call below is
16+
a plain `http.Client.Do` with no header bookkeeping.
17+
1. Hits `GET /v1/me` to verify the bound identity + scopes (canary: "is my token actually valid against this
18+
cluster?").
19+
1. Exercises `PUT /v1/cache/:key`, `GET` with raw bytes, `GET` with the JSON envelope (metadata view),
20+
`DELETE`, and `POST /v1/cache/batch/put`.
21+
22+
## Requirements
23+
24+
- Go 1.26+ (see [`go.mod`](../../go.mod))
25+
- A reachable `hypercache-server` running with OIDC enabled (i.e. `HYPERCACHE_OIDC_ISSUER` and
26+
`HYPERCACHE_OIDC_AUDIENCE` set on the server). See
27+
[`cmd/hypercache-server/README.md`](../../cmd/hypercache-server/README.md).
28+
- An OIDC client registered in your IdP with -. The **client_credentials** grant type enabled. -. A scope (or
29+
audience claim mapper) that produces the scopes the cache expects — see [Scope mapping](#scope-mapping)
30+
below.
31+
32+
## Environment variables
33+
34+
| Variable | Required | Default | Description |
35+
| --------------------- | -------- | ----------------------- | ------------------------------------------------------------------------ |
36+
| `HYPERCACHE_ENDPOINT` | no | `http://localhost:8080` | Cache server base URL (client API port). |
37+
| `OIDC_ISSUER` | **yes** || IdP base URL (no trailing `/.well-known`). |
38+
| `OIDC_AUDIENCE` | **yes** || Must match the server's `HYPERCACHE_OIDC_AUDIENCE`. |
39+
| `OIDC_CLIENT_ID` | **yes** || OAuth2 client ID registered for this service in the IdP. |
40+
| `OIDC_CLIENT_SECRET` | **yes** || OAuth2 client secret. Treat as a secret — never commit. |
41+
| `OIDC_SCOPES` | no | `openid` | Space-separated scope list. See [Scope mapping](#scope-mapping). |
42+
| `OIDC_TOKEN_INSECURE` | no | `0` | Set to `1` to skip TLS verification on the token endpoint. **Dev only.** |
43+
44+
## Run
45+
46+
```sh
47+
export HYPERCACHE_ENDPOINT=https://cache.example.com:8080
48+
export OIDC_ISSUER=https://keycloak.example.com/realms/cache
49+
export OIDC_AUDIENCE=hypercache-cluster
50+
export OIDC_CLIENT_ID=my-service
51+
export OIDC_CLIENT_SECRET=...
52+
export OIDC_SCOPES="openid cache:read cache:write"
53+
54+
go run ./__examples/distributed-oidc-client/
55+
```
56+
57+
Expected output:
58+
59+
```text
60+
== /v1/me (verify bound identity) ==
61+
resolved identity: my-service
62+
granted scopes: [read write]
63+
64+
== PUT /v1/cache/example-key (5 min TTL) ==
65+
stored
66+
67+
== GET /v1/cache/example-key (raw bytes) ==
68+
value: "hello from oidc client"
69+
70+
== GET /v1/cache/example-key (JSON envelope) ==
71+
key: example-key
72+
version: 1
73+
owners: [cache-0 cache-1 cache-2]
74+
encoding: base64
75+
76+
== DELETE /v1/cache/example-key ==
77+
deleted
78+
79+
== batch PUT /v1/cache/batch/put (3 keys) ==
80+
stored 3 keys
81+
```
82+
83+
## Scope mapping
84+
85+
The cache treats scopes as a closed set: `read`, `write`, `admin`. Your IdP's scope/claim values must map to
86+
those three strings for the cache to grant access.
87+
88+
Two configuration knobs on the server control this mapping:
89+
90+
- `HYPERCACHE_OIDC_SCOPE_CLAIM` (default `scope`) — which JWT claim to read. Standard OAuth2 uses `scope`
91+
(space-separated string); some IdPs use a custom array claim like `cache_scopes`.
92+
- The values inside that claim must be exactly `read`, `write`, or `admin`. Anything else is dropped silently.
93+
94+
**Pattern 1: OAuth2 standard scopes.** Register scopes `read` and `write` in your IdP, grant them to the
95+
service client, and request them via `OIDC_SCOPES="openid read write"`. The cache reads them from the standard
96+
`scope` claim.
97+
98+
**Pattern 2: Mapped scopes** (when your IdP namespaces scopes, e.g. `cache:read`). Use the IdP's claim-mapper
99+
feature to project the `cache:read` scope into a custom claim, then set
100+
`HYPERCACHE_OIDC_SCOPE_CLAIM=cache_scopes` server-side. Map `cache:read``read`, `cache:write``write` at
101+
the mapper level.
102+
103+
**Pattern 3: Role-based.** Map IdP roles (e.g. Keycloak realm roles `cache-reader`, `cache-writer`) into the
104+
custom claim. Same shape as Pattern 2.
105+
106+
## Coexistence with static bearer tokens
107+
108+
A cluster can run with both OIDC verification and static bearer tokens configured at the same time. The auth
109+
chain resolves in this order:
110+
111+
1. `Authorization: Bearer <token>` matched against `HYPERCACHE_AUTH_CONFIG`'s `tokens` list → static identity.
112+
1. If no static match, the OIDC verifier runs → OIDC identity.
113+
1. If neither matches and `AllowAnonymous: true`, request runs as anonymous. Otherwise 401.
114+
115+
This means a single deployment can have humans signing in via OIDC (through the monitor's redirect flow) and
116+
machine integrations using long-lived static bearers — both succeed against the same cache. See
117+
[`pkg/httpauth/policy.go`](../../pkg/httpauth/policy.go) for the implementation.
118+
119+
## Production caveats
120+
121+
This example is intentionally minimal. For real services:
122+
123+
- **Pool HTTP connections.** `oauth2.NewClient` returns a fresh `http.Client`; in production you want a
124+
configured `Transport` with `MaxIdleConnsPerHost`, keepalives, and connection-level timeouts.
125+
- **Retry policy.** This example does no retries; transient 5xx or network errors surface as failures. Wrap
126+
the cache calls in a bounded exponential-backoff retry for production.
127+
- **Observability.** The cache emits OpenTelemetry traces if the server is configured with a tracer provider;
128+
propagate the trace context by setting `traceparent` headers on your requests.
129+
- **Token caching across processes.** `clientcredentials` caches tokens per-process. If your service spawns
130+
many short-lived workers, consider a shared cache (e.g. Redis-backed) to avoid re-hitting the IdP on every
131+
process start.
132+
133+
## See also
134+
135+
- [`docs/auth.md`](../../docs/auth.md) — server-side auth surface (when present; otherwise
136+
[`cmd/hypercache-server/README.md`](../../cmd/hypercache-server/README.md) is the current source of truth).
137+
- [`docs/oncall.md`](../../docs/oncall.md#auth-failures) — debugging 401/403s when the client surface is
138+
misbehaving.
139+
- [`cmd/hypercache-server/oidc.go`](../../cmd/hypercache-server/oidc.go) — the cache's OIDC verifier closure,
140+
for reference on what's enforced server-side.

0 commit comments

Comments
 (0)