Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
87157da
feat(cloudflare): add telemetry collection and /metrics endpoint
vahidlazio May 7, 2026
30050f2
fix: await KV puts, guard malformed BucketSpan offsets, add tests
vahidlazio May 7, 2026
d71af3f
chore: sync WASM module for Go provider
vahidlazio May 7, 2026
d149fe5
fix: satisfy strict clippy lints in accumulate_delta
vahidlazio May 7, 2026
58cdfc9
style: fix rustfmt line length in accumulate_delta
vahidlazio May 7, 2026
d241fa7
chore: re-sync WASM module for Go provider after fmt fix
vahidlazio May 7, 2026
6d5f02b
fix: aggregate telemetry deltas in flag_logger::aggregate_batch
vahidlazio May 7, 2026
c686dd3
fix(cloudflare): address PR review feedback
vahidlazio May 8, 2026
f5a745e
fix: resolve clippy redundant closure warning
vahidlazio May 8, 2026
86fb301
perf(cloudflare): replace Reflect with web-sys typed bindings for per…
vahidlazio May 11, 2026
cc4ee3f
docs: explain why js_sys::global() cast is needed for WorkerGlobalScope
vahidlazio May 11, 2026
eb2183b
feat(cloudflare): source resolve latency from CF analytics API
vahidlazio May 11, 2026
3ee0396
fix(cloudflare): fix histogram bucket overflow and distribution rounding
vahidlazio May 11, 2026
b857fcc
fix(cloudflare): handle single-request analytics data points
vahidlazio May 11, 2026
b087861
feat(cloudflare): rate-limit CF analytics queries to minimum 10s inte…
vahidlazio May 12, 2026
ba0c117
refactor(cloudflare): use cursor datetime for rate-limiting instead o…
vahidlazio May 12, 2026
e724283
feat(cloudflare): include CF analytics latency in WriteFlagLogsRequest
vahidlazio May 12, 2026
096769c
feat: add SDK_ID_CLOUDFLARE_RESOLVER (25) to SdkId enum
vahidlazio May 12, 2026
17fcf25
fix(cloudflare): set resolver_version in telemetry data
vahidlazio May 12, 2026
75aeae4
fix(cloudflare): address PR review findings
vahidlazio May 12, 2026
c6e9676
feat(cloudflare): use cached percentiles for continuous latency telem…
vahidlazio May 12, 2026
545315d
feat(cloudflare): measure resolve latency inline via scheduler.wait(0)
vahidlazio May 12, 2026
84ca088
refactor(cloudflare): remove CF GraphQL analytics dependency
vahidlazio May 12, 2026
5903d4e
chore(cloudflare): clean up after GraphQL removal
vahidlazio May 12, 2026
6afd948
chore: re-sync WASM module for Go provider after rebase
vahidlazio May 12, 2026
3c46b13
fix(cloudflare): address review comments (#6-#9)
vahidlazio May 12, 2026
e5be018
docs(cloudflare): add telemetry docs and DISABLE_METRICS option
vahidlazio May 12, 2026
dd15df5
docs(cloudflare): clarify backend telemetry is independent of DISABLE…
vahidlazio May 12, 2026
e081a84
docs(cloudflare): remove Grafana references from deployer docs
vahidlazio May 12, 2026
aeadc9f
docs(cloudflare): link KV pricing in metrics docs
vahidlazio May 12, 2026
1fb034c
refactor(cloudflare): change /metrics from opt-out to opt-in
vahidlazio May 13, 2026
1334969
refactor(cloudflare): replace static loggers with per-request FLAG_LOG
andreas-karlsson May 13, 2026
9e4d07b
Update confidence-cloudflare-resolver/deployer/README.md
andreas-karlsson May 13, 2026
5ae3403
fixup! refactor(cloudflare): replace static loggers with per-request …
andreas-karlsson May 13, 2026
1461226
fix: formatting and linting
andreas-karlsson May 13, 2026
123add6
fix: wasm go....
andreas-karlsson May 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions confidence-cloudflare-resolver/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,7 @@ worker = { version= "0.6.1", features=['queue'] }
base64 = "0.22.1"
once_cell = "1.19"
prost = "0.13"
arc-swap = "1"
js-sys = "0.3"
serde = { version = "1.0.219" }
serde_json = "1.0.85"
28 changes: 28 additions & 0 deletions confidence-cloudflare-resolver/deployer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ The deployer automatically:
| `WRANGLER_DEPLOY_MESSAGE` | Value passed to `wrangler deploy --message` |
| `WRANGLER_DEPLOY_ARGS` | Additional newline-separated arguments passed to `wrangler deploy` |
| `WRANGLER_DEPLOY_ARGS_FILE` | Path to a file containing additional `wrangler deploy` arguments, one argument per line |
| `ENABLE_METRICS` | Set to create a KV namespace and enable the `/metrics` Prometheus endpoint. Requires a [KV store](https://developers.cloudflare.com/kv/platform/pricing/) |

### Extending Wrangler Configuration

Expand Down Expand Up @@ -120,6 +121,33 @@ When integrating with the Cloudflare resolver, you have two options:

For more details on integration, including code examples using the [`@spotify-confidence/sdk`](https://github.com/spotify/confidence-sdk-js), see the [Confidence documentation](https://confidence.spotify.com/docs/sdks/edge/cloudflare#cloudflare-workers).

## Telemetry & Metrics

The resolver collects telemetry and exposes a Prometheus-compatible `/metrics` endpoint using the same metric names as all other Confidence providers (`confidence_resolve_latency_microseconds`, `confidence_resolves_total`).

### How latency is measured

Cloudflare Workers freeze `Date.now()` and `performance.now()` during synchronous CPU work (Spectre mitigation). The resolver uses `scheduler.wait(0)` — a zero-delay yield to the runtime — to unfreeze the clock after each resolve. This provides 1ms resolution with no measurable overhead.

### `/metrics` endpoint

Requires authentication:

```bash
curl -H "Authorization: ClientSecret <your-client-secret>" \
https://<worker>.workers.dev/metrics
```

Returns Prometheus exposition format with:
- `confidence_resolve_latency_microseconds` — histogram (sum, count, cumulative `le` buckets)
- `confidence_resolves_total` — counter by resolve reason

Metrics are accumulated in a [KV namespace](https://developers.cloudflare.com/kv/platform/pricing/) (`CONFIDENCE_METRICS_KV`). Set `ENABLE_METRICS` to have the deployer create the KV namespace and bind it to the Worker. Without it, the `/metrics` endpoint returns empty and no KV writes occur.

### Backend telemetry

Resolve rates and latency are always sent to the Confidence backend via `WriteFlagLogsRequest`, regardless of the `ENABLE_METRICS` setting. The `/metrics` endpoint and KV store are only needed for direct Prometheus scraping — backend telemetry flows through the queue consumer independently.

## Limitations

* **Sticky assignments**: Not currently supported with the Cloudflare resolver. Flags with sticky assignment rules will return "flag not found".
58 changes: 58 additions & 0 deletions confidence-cloudflare-resolver/deployer/script.sh
Original file line number Diff line number Diff line change
Expand Up @@ -366,6 +366,63 @@ else
echo "⚠️ Could not check queue status (HTTP $QUEUE_STATUS)"
fi

# Create KV namespace for /metrics endpoint if it doesn't exist
if [ -n "$WORKER_NAME_PREFIX" ]; then
KV_NAMESPACE_TITLE="${WORKER_NAME_PREFIX}-resolver-metrics"
else
KV_NAMESPACE_TITLE="resolver-metrics"
fi

ENABLE_METRICS=${ENABLE_METRICS:=}
if [ -z "$ENABLE_METRICS" ]; then
echo "ℹ️ ENABLE_METRICS not set; skipping KV namespace creation (/metrics endpoint disabled)"
KV_NAMESPACE_ID=""
else

echo "🔍 Checking if KV namespace '$KV_NAMESPACE_TITLE' exists..."
KV_LIST=$(curl -sS -w "%{http_code}" \
-H "Authorization: Bearer ${CLOUDFLARE_API_TOKEN}" \
"https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/storage/kv/namespaces?per_page=100")
KV_LIST_STATUS="${KV_LIST: -3}"
KV_LIST_BODY="${KV_LIST%???}"

KV_NAMESPACE_ID=""
if [ "$KV_LIST_STATUS" = "200" ]; then
KV_NAMESPACE_ID=$(printf "%s" "$KV_LIST_BODY" | jq -r ".result[] | select(.title == \"${KV_NAMESPACE_TITLE}\") | .id" 2>/dev/null || true)
fi

if [ -z "$KV_NAMESPACE_ID" ]; then
echo "📦 KV namespace '$KV_NAMESPACE_TITLE' not found, creating..."
KV_CREATE_RESP=$(curl -sS -w "%{http_code}" -X POST \
-H "Authorization: Bearer ${CLOUDFLARE_API_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"title\": \"${KV_NAMESPACE_TITLE}\"}" \
"https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/storage/kv/namespaces")
KV_CREATE_STATUS="${KV_CREATE_RESP: -3}"
KV_CREATE_BODY="${KV_CREATE_RESP%???}"
if [ "$KV_CREATE_STATUS" = "200" ] || [ "$KV_CREATE_STATUS" = "201" ]; then
KV_NAMESPACE_ID=$(printf "%s" "$KV_CREATE_BODY" | jq -r '.result.id')
echo "✅ KV namespace '$KV_NAMESPACE_TITLE' created (id: $KV_NAMESPACE_ID)"
else
echo "⚠️ Failed to create KV namespace (HTTP $KV_CREATE_STATUS), /metrics will be unavailable"
fi
else
echo "✅ KV namespace '$KV_NAMESPACE_TITLE' already exists (id: $KV_NAMESPACE_ID)"
fi

# Append KV binding to wrangler.toml if namespace was created
if [ -n "$KV_NAMESPACE_ID" ]; then
cat >> wrangler.toml <<EOF

[[kv_namespaces]]
binding = "CONFIDENCE_METRICS_KV"
id = "$KV_NAMESPACE_ID"
EOF
echo "✅ Added CONFIDENCE_METRICS_KV binding to wrangler.toml"
fi

fi # end ENABLE_METRICS check

# Update worker name and queue name in wrangler.toml if using prefix
if [ -n "$WORKER_NAME_PREFIX" ]; then
sed -i.tmp "s/^name = .*/name = \"$WORKER_NAME\"/" wrangler.toml
Expand Down Expand Up @@ -479,6 +536,7 @@ add_wrangler_deploy_args_from_lines "WRANGLER_DEPLOY_ARGS" "$WRANGLER_DEPLOY_ARG
# only deploy if NO_DEPLOY is not set
if test -z "$NO_DEPLOY"; then
wrangler deploy "${WRANGLER_DEPLOY_ARGS_ARRAY[@]}"

else
echo "NO_DEPLOY is set, skipping deploy"
fi
Loading