Skip to content

Commit f4cf37f

Browse files
a-a-ka-a-k
committed
Enhance README and documentation to support new topology input format, allowing explicit non-trace batch sources. Update CLI commands and examples to reflect the inclusion of topology files. Improve test coverage for topology discovery and ensure proper handling of provenance and overlays in the discovery process.
1 parent 958faad commit f4cf37f

15 files changed

Lines changed: 1242 additions & 20 deletions

‎README.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Bering is a discovery and publishing layer for service topology and endpoint con
44

55
It supports two operating modes:
66

7-
- deterministic batch discovery from trace files and directories
7+
- deterministic batch discovery from trace files/directories or explicit topology input files
88
- long-running runtime discovery that accepts OTLP/HTTP and optional OTLP/gRPC spans and publishes rolling snapshot envelopes for observability consumers
99

1010
Bering owns discovery and discovery-side public contracts. It does not own simulation, gating, chaos execution, or policy decisions.
@@ -40,6 +40,7 @@ cmd/bering CLI entrypoint
4040
internal/app command wiring
4141
internal/config serve-mode config parsing and validation
4242
internal/connectors/traces file/dir trace loading and normalization
43+
internal/connectors/topology non-trace topology_api file loading and normalization
4344
internal/connectors/otlp OTLP request decoding into normalized spans
4445
internal/discovery source-agnostic discovery engine and overlay application
4546
internal/model stable core model structs, semantic checks, canonical IO
@@ -57,7 +58,7 @@ scripts/ci CI helper scripts
5758
## Commands
5859

5960
```bash
60-
bering discover --input <trace-file|dir> [--out bering-model.json] [--snapshot-out bering-snapshot.json] [--replicas replicas.yaml|json] [--overlay overlay.yaml] [--discovered-at RFC3339]
61+
bering discover --input <trace-file|topology-file|dir> [--out bering-model.json] [--snapshot-out bering-snapshot.json] [--replicas replicas.yaml|json] [--overlay overlay.yaml] [--discovered-at RFC3339]
6162
bering validate --input <bering-model.json|bering-snapshot.json>
6263
bering serve --config configs/serve.sample.yaml [--listen :4318] [--grpc-listen :4317] [--window-size 30s] [--flush-interval 5s]
6364
```
@@ -121,7 +122,8 @@ go run ./cmd/sheaft run \
121122
## Examples
122123

123124
- Batch inputs: [examples/traces/normalized.sample.json](examples/traces/normalized.sample.json), [examples/traces/otel.sample.json](examples/traces/otel.sample.json)
124-
- Batch outputs: [examples/outputs/bering-model.normalized.sample.json](examples/outputs/bering-model.normalized.sample.json), [examples/outputs/bering-snapshot.normalized.sample.json](examples/outputs/bering-snapshot.normalized.sample.json)
125+
- Topology input: [examples/topology/topology-api.sample.yaml](examples/topology/topology-api.sample.yaml)
126+
- Batch outputs: [examples/outputs/bering-model.normalized.sample.json](examples/outputs/bering-model.normalized.sample.json), [examples/outputs/bering-snapshot.normalized.sample.json](examples/outputs/bering-snapshot.normalized.sample.json), [examples/outputs/bering-model.topology-api.sample.json](examples/outputs/bering-model.topology-api.sample.json), [examples/outputs/bering-snapshot.topology-api.sample.json](examples/outputs/bering-snapshot.topology-api.sample.json)
125127
- Runtime config: [configs/serve.sample.yaml](configs/serve.sample.yaml)
126128
- Discovery overlay: [configs/discovery.overlay.sample.yaml](configs/discovery.overlay.sample.yaml)
127129
- Collector sidecar: [examples/collector/otelcol.sidecar.yaml](examples/collector/otelcol.sidecar.yaml)
@@ -184,6 +186,7 @@ The runtime service exports Prometheus/OpenMetrics metrics including:
184186
- [docs/architecture.md](docs/architecture.md)
185187
- [docs/runtime-config.md](docs/runtime-config.md)
186188
- [docs/trace-input-format.md](docs/trace-input-format.md)
189+
- [docs/topology-input-format.md](docs/topology-input-format.md)
187190
- [docs/schema-publishing.md](docs/schema-publishing.md)
188191
- [docs/migration-notes.md](docs/migration-notes.md)
189192
- [docs/mvp-scope-and-limits.md](docs/mvp-scope-and-limits.md)

‎docs/architecture.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@ Bering now has two user-facing flows built on one normalized discovery core.
77
### Batch flow
88

99
1. `bering discover` loads trace JSON files or directories.
10-
2. File inputs are normalized into the internal `traces.Span` shape.
11-
3. The discovery engine infers services, edges, endpoints, confidence, and overlay-driven metadata.
10+
2. Trace inputs are normalized into the internal `traces.Span` shape, while explicit `topology_api` inputs are normalized into the same downstream model and snapshot discovery contracts.
11+
3. The discovery engine or adapter-specific builder produces services, edges, endpoints, confidence, and overlay-driven metadata.
1212
4. Bering writes the stable `io.mb3r.bering.model` artifact.
1313
5. Optional `--snapshot-out` also writes a `io.mb3r.bering.snapshot` envelope.
1414

‎docs/migration-notes.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ You can now opt into additional discovery-side features.
1616

1717
- `--overlay` for generic discovery metadata overlays
1818
- `--snapshot-out` for a snapshot envelope in batch mode
19+
- `bering discover` can now ingest explicit `topology_api` YAML/JSON files in addition to trace inputs
1920
- `bering serve` for OTLP/HTTP runtime discovery
2021
- optional OTLP/gRPC ingest on a separate runtime listener
2122

‎docs/mvp-scope-and-limits.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
## In scope
44

55
- deterministic batch discovery from trace files/directories
6+
- deterministic batch discovery from explicit `topology_api` files
67
- long-running runtime service that accepts OTLP/HTTP spans, with optional OTLP/gRPC ingest
78
- stable core model artifacts (`io.mb3r.bering.model`)
89
- snapshot envelopes for observability/runtime consumers (`io.mb3r.bering.snapshot`)

‎docs/topology-input-format.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# Topology Input Format
2+
3+
Bering also supports an explicit non-trace batch source: `topology_api`.
4+
5+
This is intended for cases where topology is already available from a topology API, service registry, mesh export, or another inventory source and you want Bering to publish the same model and snapshot artifacts without going through trace inference.
6+
7+
## Example
8+
9+
```yaml
10+
source:
11+
type: topology_api
12+
ref: https://topology.internal.example/api/v1/topology
13+
services:
14+
- id: frontend
15+
replicas: 2
16+
edges:
17+
- from: frontend
18+
to: checkout
19+
kind: sync
20+
endpoints:
21+
- entry_service: frontend
22+
method: GET
23+
path: /checkout
24+
```
25+
26+
## Fields
27+
28+
### `source`
29+
30+
- `type`: defaults to `topology_api`
31+
- `ref`: optional upstream source reference that will appear in snapshot `sources[]` and per-record provenance
32+
33+
### `services[]`
34+
35+
- `id`: required
36+
- `name`: optional, defaults to `id`
37+
- `replicas`: optional, defaults to `1`
38+
- `support.observations`: optional, defaults to `1`
39+
- `support.evidence`: optional, defaults to `[topology_api]`
40+
- `first_seen`, `last_seen`: optional RFC3339 timestamps
41+
- metadata fields: `labels`, `tags`, `slo_refs`, `attributes`, `failure_eligible`
42+
43+
### `edges[]`
44+
45+
- `from`, `to`, `kind`: required
46+
- `kind`: `sync` or `async`
47+
- `blocking`: optional, defaults to `true` for `sync` and `false` for `async`
48+
- `id`: optional, derived from `from|to|kind|blocking`
49+
- `support.*`, `first_seen`, `last_seen`, `labels`, `tags`, `slo_refs`, `attributes`, `weight`
50+
51+
### `endpoints[]`
52+
53+
- `entry_service`: required
54+
- `id`: optional if `method` and `path` are provided
55+
- `method`: normalized to uppercase
56+
- `path`: normalized to start with `/`
57+
- `predicate_ref`: optional, defaults to endpoint `id`
58+
- `support.*`, `first_seen`, `last_seen`, `labels`, `tags`, `slo_refs`, `attributes`, `weight`
59+
60+
## Notes
61+
62+
- `bering discover` still emits `metadata.source_type = "bering"` in the stable model and snapshot envelope metadata.
63+
- The upstream non-trace source is represented in snapshot `sources[]` and `discovery.*[].provenance`.
64+
- For explicit `topology_api` input, `metadata.confidence` is set to `1.0` because the topology is supplied directly rather than inferred from traces.
65+
- Discovery overlays continue to work on top of `topology_api` input.

‎docs/trace-input-format.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
Bering supports two batch JSON formats and two runtime network ingest paths.
44

5+
For explicit non-trace batch topology input, see [topology-input-format.md](topology-input-format.md).
6+
57
## 1) Normalized spans JSON
68

79
Top-level shape:

‎examples/outputs/bering-model.topology-api.sample.json‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
{
2+
"edges": [
3+
{
4+
"blocking": false,
5+
"from": "checkout",
6+
"kind": "async",
7+
"to": "payment"
8+
},
9+
{
10+
"blocking": true,
11+
"from": "frontend",
12+
"kind": "sync",
13+
"to": "checkout"
14+
}
15+
],
16+
"endpoints": [
17+
{
18+
"entry_service": "checkout",
19+
"id": "checkout:POST /process",
20+
"success_predicate_ref": "checkout:POST /process"
21+
},
22+
{
23+
"entry_service": "frontend",
24+
"id": "frontend:GET /checkout",
25+
"success_predicate_ref": "catalog.frontend.checkout.success"
26+
}
27+
],
28+
"metadata": {
29+
"confidence": 1,
30+
"discovered_at": "2026-03-11T00:00:00Z",
31+
"schema": {
32+
"digest": "sha256:272277c093f37580adcd2dded225bd37c86539d642d7910baad7e4228227d1a7",
33+
"name": "io.mb3r.bering.model",
34+
"uri": "https://mb3r-lab.github.io/Bering/schema/model/v1.0.0/model.schema.json",
35+
"version": "1.0.0"
36+
},
37+
"source_ref": "bering://discover?input=examples%2Ftopology%2Ftopology-api.sample.yaml",
38+
"source_type": "bering"
39+
},
40+
"services": [
41+
{
42+
"id": "checkout",
43+
"name": "checkout",
44+
"replicas": 3
45+
},
46+
{
47+
"id": "frontend",
48+
"name": "frontend",
49+
"replicas": 2
50+
},
51+
{
52+
"id": "payment",
53+
"name": "payment",
54+
"replicas": 1
55+
}
56+
]
57+
}

0 commit comments

Comments
 (0)