Skip to content

Commit dec1c01

Browse files
committed
feat: add signal quality and onboarding surfaces
1 parent e4ab682 commit dec1c01

26 files changed

Lines changed: 1637 additions & 80 deletions

README.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,8 @@ go run ./cmd/bering discover \
8787

8888
Expected result: a deterministic `io.mb3r.bering.model@1.1.0` artifact.
8989

90+
`discover` also writes `out/bering-model.signal-quality.json` next to the model.
91+
9092
### 2) Validate an artifact
9193

9294
```bash
@@ -109,6 +111,8 @@ go run ./cmd/bering discover \
109111

110112
Expected result: a model artifact plus an `io.mb3r.bering.snapshot@1.1.0` envelope.
111113

114+
The snapshot gets its own `out/bering-snapshot.signal-quality.json` sidecar.
115+
112116
### 4) Run runtime mode
113117

114118
```bash
@@ -126,6 +130,8 @@ Useful endpoints:
126130
- `GET /healthz`
127131
- `GET /readyz`
128132
- `GET /metrics`
133+
- `GET /reconciliation/report`
134+
- `GET /reconciliation/summary`
129135

130136
### 5) Use the result with Sheaft
131137

@@ -160,6 +166,10 @@ This is the stable topology artifact intended for file-based consumers and tools
160166

161167
This wraps the model with runtime window metadata, ingest counts, coverage, provenance, topology diffs, and discovery-side copies of the same typed placement, resilience, and observed edge metadata.
162168

169+
### Signal quality report
170+
171+
Every batch model, batch snapshot, and runtime snapshot gets a separate `*.signal-quality.json` sidecar. The sidecar reports source mix, provenance, field coverage for typed enrichment fields, and warnings for low-coverage dimensions without changing the public model or snapshot schemas.
172+
163173
### Runtime reconciliation views
164174

165175
Runtime mode also keeps a conservative reconciliation layer for sparse traffic and telemetry gaps:
@@ -168,6 +178,7 @@ Runtime mode also keeps a conservative reconciliation layer for sparse traffic a
168178
- `stable_core`: the high-confidence retained view
169179
- `guardrail_union`: the conservative default downstream artifact
170180
- a machine-readable reconciliation report for operators
181+
- an operator-facing reconciliation summary in Markdown
171182

172183
These runtime views do not change the public `io.mb3r.bering.model@1.1.0` or `io.mb3r.bering.snapshot@1.1.0` contracts.
173184

@@ -203,6 +214,8 @@ Release outputs include:
203214
- Batch outputs: [examples/outputs/bering-model.normalized.sample.json](examples/outputs/bering-model.normalized.sample.json), [examples/outputs/bering-snapshot.normalized.sample.json](examples/outputs/bering-snapshot.normalized.sample.json), [examples/outputs/bering-model.topology-api.sample.json](examples/outputs/bering-model.topology-api.sample.json), [examples/outputs/bering-snapshot.topology-api.sample.json](examples/outputs/bering-snapshot.topology-api.sample.json)
204215
- Runtime config: [configs/serve.sample.yaml](configs/serve.sample.yaml)
205216
- Discovery overlay: [configs/discovery.overlay.sample.yaml](configs/discovery.overlay.sample.yaml)
217+
- Enrichment onboarding: [examples/enrichment](examples/enrichment/README.md)
218+
- Non-trace adapter: [examples/adapters/service-catalog](examples/adapters/service-catalog/README.md)
206219
- Collector sidecar: [examples/collector/otelcol.sidecar.yaml](examples/collector/otelcol.sidecar.yaml)
207220
- Prometheus scrape config: [examples/prometheus/bering.prometheus.yml](examples/prometheus/bering.prometheus.yml)
208221
- Grafana dashboard: [examples/grafana/bering-runtime-dashboard.json](examples/grafana/bering-runtime-dashboard.json)
@@ -216,6 +229,7 @@ Release outputs include:
216229
- [docs/schema-publishing.md](docs/schema-publishing.md)
217230
- [docs/trace-input-format.md](docs/trace-input-format.md)
218231
- [docs/topology-input-format.md](docs/topology-input-format.md)
232+
- [docs/enrichment-onboarding.md](docs/enrichment-onboarding.md)
219233
- [docs/mvp-scope-and-limits.md](docs/mvp-scope-and-limits.md)
220234
- [Delivery tracker](https://github.com/MB3R-Lab/Bering/issues/19)
221235
- [docs/architecture.md](docs/architecture.md)

configs/serve.sample.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ runtime:
1111
enabled: true
1212
state_path: out/reconciliation-state.json
1313
report_path: out/reconciliation-report.json
14+
summary_path: out/reconciliation-summary.md
1415
raw_window_path: out/latest-raw-window.json
1516
stable_core_path: out/latest-stable-core.json
1617
decay_half_life: 10m
@@ -30,6 +31,7 @@ runtime:
3031
sink:
3132
directory: out/snapshots
3233
latest_path: out/latest-snapshot.json
34+
signal_quality_path: out/latest-signal-quality.json
3335
logging:
3436
structured: true
3537
overlays:

docs/architecture.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ Bering now has two user-facing flows built on one normalized discovery core.
1111
3. The discovery engine or adapter-specific builder produces services, edges, endpoints, confidence, and overlay-driven metadata.
1212
4. Bering writes the stable `io.mb3r.bering.model` artifact.
1313
5. Optional `--snapshot-out` also writes a `io.mb3r.bering.snapshot` envelope.
14+
6. Batch and runtime artifacts get adjacent signal-quality sidecars that report source mix, provenance, and typed-field coverage without changing the public schemas.
1415

1516
### Runtime flow
1617

@@ -27,7 +28,7 @@ The runtime additions are intentionally interface-driven so future connectors ca
2728

2829
- source adapters: normalize external data into internal spans or overlays
2930
- overlay loaders: decode reusable discovery-side metadata files
30-
- snapshot sinks: write snapshot envelopes to one or more destinations
31+
- snapshot sinks: write snapshot envelopes and signal-quality sidecars to one or more destinations
3132
- clocks: make windowing deterministic in tests
3233
- storage boundary: one active in-memory tumbling window with an explicit cap
3334

docs/enrichment-onboarding.md

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Enrichment Onboarding
2+
3+
Bering treats traces, overlays, and `topology_api` inputs as different discovery evidence surfaces.
4+
5+
## Workflow
6+
7+
1. Start with trace-only discovery.
8+
2. Read the generated `*.signal-quality.json` sidecar.
9+
3. Add an overlay for curated metadata that traces cannot infer.
10+
4. Move to `topology_api` when a non-trace system is authoritative for topology, placement, resilience, or endpoint metadata.
11+
12+
## Trace-only baseline
13+
14+
```bash
15+
go run ./cmd/bering discover \
16+
--input examples/traces/normalized.sample.json \
17+
--out out/onboarding/trace-only/bering-model.json \
18+
--snapshot-out out/onboarding/trace-only/bering-snapshot.json \
19+
--discovered-at 2026-03-03T00:00:00Z
20+
```
21+
22+
The generated signal quality report calls out missing coverage for fields such as `services.placements`, `edges.request_timeout_ms`, `edges.retry_policy`, and `endpoints.custom_success_predicate`.
23+
24+
## Overlay upgrade
25+
26+
```bash
27+
go run ./cmd/bering discover \
28+
--input examples/traces/normalized.sample.json \
29+
--overlay examples/enrichment/checkout.overlay.yaml \
30+
--out out/onboarding/overlay/bering-model.json \
31+
--snapshot-out out/onboarding/overlay/bering-snapshot.json \
32+
--discovered-at 2026-03-03T00:00:00Z
33+
```
34+
35+
Use overlays when the trace-derived topology is acceptable but operational metadata comes from configuration, ownership catalogs, SLO catalogs, or curated policy files.
36+
37+
## topology_api upgrade
38+
39+
```bash
40+
go run ./cmd/bering discover \
41+
--input examples/enrichment/checkout.topology-api.yaml \
42+
--out out/onboarding/topology-api/bering-model.json \
43+
--snapshot-out out/onboarding/topology-api/bering-snapshot.json \
44+
--discovered-at 2026-03-03T00:00:00Z
45+
```
46+
47+
Use `topology_api` when Bering should publish an explicit topology from a non-trace adapter rather than infer the topology from spans.
48+
49+
Related examples live in [examples/enrichment](../examples/enrichment/README.md).

docs/runtime-config.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ Suggested fields:
7070
- `enabled`: toggle the reconciliation module on or off
7171
- `state_path`: JSON file used to persist reconciliation state between windows and restarts
7272
- `report_path`: machine-readable report file for the latest reconciliation result
73+
- `summary_path`: operator-facing Markdown summary for the latest reconciliation result
7374
- `decay_half_life`: controls how quickly older evidence loses weight
7475
- `minimum_opportunity_windows`: minimum opportunity evidence before misses can count
7576
- `telemetry_health_freeze_threshold`: freeze or heavily damp retirement below this score
@@ -95,6 +96,14 @@ Optional stable path that is rewritten on every successful snapshot.
9596

9697
Example: `out/latest-snapshot.json`
9798

99+
### `sink.signal_quality_path`
100+
101+
Stable path for the latest signal quality and field coverage report.
102+
103+
Example: `out/latest-signal-quality.json`
104+
105+
Each timestamped snapshot also gets an adjacent `*.signal-quality.json` sidecar.
106+
98107
### `logging.structured`
99108

100109
Emit JSON logs instead of text logs.
@@ -126,7 +135,9 @@ overlays:
126135
- `--reconciliation-enabled`
127136
- `--reconciliation-state-path`
128137
- `--reconciliation-report-path`
138+
- `--reconciliation-summary-path`
129139
- `--sink-dir`
130140
- `--latest-path`
141+
- `--signal-quality-path`
131142
- `--log-format text|json`
132143
- `--overlay` (repeatable)

docs/topology-input-format.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ Bering supports an explicit non-trace batch source: `topology_api`.
44

55
Use this when topology, placement, or resilience metadata already exists in a service registry, mesh export, inventory API, or curated file and you want Bering to publish the same stable model and snapshot artifacts without trace inference.
66

7+
A reference non-trace adapter lives at [examples/adapters/service-catalog](../examples/adapters/service-catalog/README.md). It converts a small service catalog JSON document into `topology_api` JSON before calling `bering discover`.
8+
79
## Contract line
810

911
- `bering discover` now emits `io.mb3r.bering.model@1.1.0` and `io.mb3r.bering.snapshot@1.1.0` by default.
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Service Catalog Adapter Example
2+
3+
This is a reference non-trace adapter. It reads a small service catalog JSON file and emits Bering `topology_api` JSON.
4+
5+
Generate topology input:
6+
7+
```bash
8+
go run ./examples/adapters/service-catalog \
9+
--input examples/adapters/service-catalog/catalog.sample.json \
10+
--out out/adapters/service-catalog/topology-api.json
11+
```
12+
13+
Run Bering discovery on the generated topology:
14+
15+
```bash
16+
go run ./cmd/bering discover \
17+
--input out/adapters/service-catalog/topology-api.json \
18+
--out out/adapters/service-catalog/bering-model.json \
19+
--snapshot-out out/adapters/service-catalog/bering-snapshot.json \
20+
--discovered-at 2026-03-03T00:00:00Z
21+
```
22+
23+
The adapter is intentionally plain Go and has no service-catalog dependency. Real adapters should map their source-specific concepts into the same `topology_api` fields shown here: services, placements, shared resources, dependencies, resilience policy, policy scope, and endpoints.
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
{
2+
"source_ref": "service-catalog://checkout-sample",
3+
"services": [
4+
{
5+
"id": "frontend",
6+
"replicas": 2,
7+
"team": "web",
8+
"tier": "edge"
9+
},
10+
{
11+
"id": "checkout",
12+
"replicas": 3,
13+
"team": "commerce",
14+
"tier": "backend",
15+
"failure_eligible": true,
16+
"placements": [
17+
{
18+
"replicas": 2,
19+
"labels": {
20+
"region": "us-east-1",
21+
"az": "us-east-1a"
22+
}
23+
},
24+
{
25+
"replicas": 1,
26+
"labels": {
27+
"region": "us-east-1",
28+
"az": "us-east-1b"
29+
}
30+
}
31+
],
32+
"shared_resources": [
33+
"kafka-orders",
34+
"payments-db"
35+
]
36+
},
37+
{
38+
"id": "payment",
39+
"replicas": 2,
40+
"team": "payments",
41+
"tier": "backend",
42+
"failure_eligible": true
43+
}
44+
],
45+
"dependencies": [
46+
{
47+
"from": "frontend",
48+
"to": "checkout",
49+
"mode": "sync",
50+
"blocking": true,
51+
"weight": 0.85,
52+
"request_timeout_ms": 2000,
53+
"retry_max_attempts": 3,
54+
"circuit_breaker_enabled": true,
55+
"source_endpoint_id": "frontend:GET /checkout",
56+
"method": "GET",
57+
"route": "/checkout"
58+
},
59+
{
60+
"from": "checkout",
61+
"to": "payment",
62+
"mode": "async",
63+
"blocking": false,
64+
"weight": 0.35
65+
}
66+
],
67+
"endpoints": [
68+
{
69+
"service": "frontend",
70+
"method": "GET",
71+
"path": "/checkout",
72+
"predicate_ref": "catalog.frontend.checkout.success",
73+
"weight": 0.7
74+
},
75+
{
76+
"service": "checkout",
77+
"method": "POST",
78+
"path": "/process",
79+
"predicate_ref": "catalog.checkout.process.success",
80+
"weight": 0.6
81+
}
82+
]
83+
}

0 commit comments

Comments
 (0)