Skip to content

Commit 320a777

Browse files
Enrich Hybrid Mesh Platform docs with fuller narrative.
Align depth with the platform-hub-spoke-config site: journey intro, end-to-end flows, Skupper listeners, gateway circuit breaking, and expanded observability and scaffolding guidance. Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 1bfb278 commit 320a777

8 files changed

Lines changed: 536 additions & 239 deletions

File tree

content/patterns/hybrid-mesh-platform/_index.md

Lines changed: 21 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -35,31 +35,33 @@ contributor:
3535

3636
**Maintainer:** Maximiliano Pizarro, Specialist Solution Architect at Red Hat
3737

38-
This platform deploys in one `helm upgrade`, connects three OpenShift clusters (hub + east + west), and shows IoT sensor data across Grafana and Developer Hub within about 30 minutes. The pages below follow one continuous story — concept, install, operate, scaffold — so you can read straight through or jump to any chapter.
38+
> **Your journey:** This platform deploys in one `helm upgrade`, connects three OpenShift clusters (hub + east + west), and shows IoT sensor data across Grafana and Developer Hub within about 30 minutes. The pages below follow one continuous story — concept, install, operate, scaffold — so you can read straight through or jump to any chapter.
3939
40-
Multi-cluster GitOps platform using Red Hat products a hub-spoke topology that centralizes governance with Red Hat Advanced Cluster Management (ACM), delivers [Industrial Edge](/patterns/industrial-edge/) workloads on regional spokes, uses OpenShift Service Mesh in ambient mode for east-west connectivity, layers Connectivity Link (Kuadrant) for API-aware ingress policy, exposes Grafana dashboards for cross-cluster visibility, and integrates Advanced Cluster Security (ACS) for vulnerability and runtime protection.
40+
**Hybrid Mesh Platform** is a multi-cluster GitOps platform using Red Hat products. It implements a hub-spoke topology that centralizes governance with Red Hat Advanced Cluster Management (ACM), delivers [Industrial Edge](/patterns/industrial-edge/) workloads on regional spokes, uses OpenShift Service Mesh in ambient mode for east-west connectivity, layers Connectivity Link (Kuadrant) for API-aware ingress policy, exposes Grafana dashboards for cross-cluster visibility, and integrates Advanced Cluster Security (ACS) for vulnerability and runtime protection.
4141

42-
**Supported on:** Red Hat OpenShift Container Platform 4.20 (and 4.14 or newer per cluster).
42+
**Supported on:** Red Hat OpenShift Container Platform 4.20 (4.14 or newer per cluster).
4343

44-
Read **concept → mechanics → operations**: start with [Architecture](architecture), install via [Getting Started](getting-started), scaffold workloads via [Scaffolding](scaffolding), then use platform chapters (Hub Gateway, Observability, Industrial Edge) before drilling into the pattern repository.
44+
Read **concept → mechanics → operations**: start with [Architecture](architecture), install via [Getting Started](getting-started), scaffold workloads via [Scaffolding](scaffolding), then use platform chapters (**Hub Gateway**, **Observability**, **Industrial Edge**) before drilling into the [pattern repository](https://github.com/maximilianopizarro/platform-hub-spoke-config).
4545

4646
## Overview
4747

4848
This repository models a **GitOps-first platform** where:
4949

50-
- **Hub cluster** runs ACM, OpenShift GitOps (Argo CD), observability aggregation, Developer Hub, ACS Central, Mailpit for notifications, and gateway-style HTTP routing with circuit breaking for shared services.
51-
- **Spoke clusters** (east/west regions) host **Industrial Edge** workloads: sensor and MQTT-style ingestion, Kafka pipelines, optional ML scoring, and dashboards fed by Prometheus-compatible metrics.
50+
- **Hub cluster** runs ACM, OpenShift GitOps (Argo CD), observability aggregation, Developer Hub, ACS Central, Mailpit for notifications, and gateway-style HTTP routing with **circuit breaking** for shared services.
51+
- **Spoke clusters** (east/west regions) host **Industrial Edge** patterns: sensor and MQTT-style ingestion, Kafka pipelines, optional ML scoring, and dashboards fed by Prometheus-compatible metrics.
5252
- **Service Mesh 3 ambient** reduces sidecar overhead while retaining ztunnel-based L4 and waypoint-based L7 policy where needed.
53-
- **Hub Gateway** splits traffic into front and API services per spoke, with per-service **circuit breaking** via `DestinationRule`.
54-
- **Service Interconnect (Skupper)** bridges spoke services and metrics to the hub via a Virtual Application Network, without VPN or firewall changes.
53+
- **Hub Gateway** splits traffic into **front** and **API** services per spoke, with per-service **circuit breaking** via `DestinationRule`.
54+
- **Service Interconnect (Skupper)** bridges spoke services and metrics to the hub via a Virtual Application Network (VAN), without VPN or firewall changes.
5555
- **Spoke Gateways** aggregate Industrial Edge services per spoke for simplified cross-cluster exposure.
5656
- **Kiali + OSSM Console** provides service mesh topology visualization on every cluster via the OpenShift Console plugin.
5757
- **Grafana dashboards** roll up cluster and application signals from all clusters.
5858
- **ACS** provides centralized policy, CVE visibility, and SecuredCluster agents on spokes.
5959

6060
[![Platform hub-spoke overview](/images/hybrid-mesh-platform/arch-overview.png)](/images/hybrid-mesh-platform/arch-overview.png)
6161

62-
_Hub cluster aggregates observability and Developer Hub; east and west spokes run Industrial Edge workloads connected via Service Interconnect (Skupper)._
62+
_Hub cluster aggregates observability and Developer Hub; east and west spokes run Industrial Edge workloads connected via Service Interconnect (Skupper). Click the image to open the full diagram._
63+
64+
Architecture diagrams in this documentation illustrate Git, ACM, Skupper VAN, and Industrial Edge placement on east/west — use them as the visual companion to the install chapters.
6365

6466
## Quick links
6567

@@ -71,16 +73,20 @@ _Hub cluster aggregates observability and Developer Hub; east and west spokes ru
7173
| Observability | [Observability](observability) |
7274
| Industrial Edge (multi-cluster) | [Industrial Edge](industrial-edge) |
7375
| Scaffolding | [Scaffolding](scaffolding) |
74-
| Customization | [Ideas for customization](ideas-for-customization) |
76+
| Branch strategy and customization | [Ideas for customization](ideas-for-customization) |
7577

7678
## Recommended reading order
7779

78-
1. [Architecture](architecture) — mental model of hub, spokes, GitOps, and observability
79-
2. [Getting Started](getting-started) — bring clusters under GitOps
80+
1. [Architecture](architecture) — mental model of hub, spokes, GitOps, Skupper, and observability
81+
2. [Getting Started](getting-started) — bring clusters under GitOps (includes ACM + ApplicationSet detail)
8082
3. [Scaffolding](scaffolding) — deploy Industrial Edge instances on east/west from Developer Hub
8183
4. [Hub Gateway](hub-gateway) — weighted ingress and circuit breaking across spokes
8284
5. [Observability](observability) — Grafana, Kiali, Kafka Console
83-
6. [Industrial Edge](industrial-edge) — factory data pipeline on multiple spokes
85+
6. [Industrial Edge](industrial-edge) — factory data pipeline: sensors, Kafka, Camel, ML on multiple spokes
86+
87+
Screenshots and architecture diagrams in the pattern repository support full-screen review — handy after deploying dashboards and verifying cross-cluster traffic.
88+
89+
**Next →** [Architecture](architecture) — understand how Git, ACM, and Skupper wire the three clusters together.
8490

8591
## Red Hat products used
8692

@@ -96,3 +102,5 @@ _Hub cluster aggregates observability and Developer Hub; east and west spokes ru
96102
- Red Hat OpenShift Pipelines (Tekton)
97103
- Red Hat Developer Hub (Backstage)
98104
- Red Hat Service Interconnect (Skupper)
105+
- Mailpit (SMTP testing for notifications)
106+
- Observability stack (Prometheus-compatible metrics, Grafana, OpenTelemetry, Kiali)

content/patterns/hybrid-mesh-platform/architecture.md

Lines changed: 95 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,11 @@ aliases: /hybrid-mesh-platform/architecture/
66

77
# Architecture
88

9+
Understand how Git, ACM, Service Interconnect, and Industrial Edge wire the hub and spoke clusters together before you install or scaffold workloads.
10+
911
## Hub-spoke theory in multi-cluster Kubernetes
1012

11-
In multi-cluster Kubernetes, a hub-spoke model designates one administrative cluster (the hub) and one or more workload clusters (spokes). The hub owns fleet APIs: cluster inventory, policy placement, credentials for spoke registration, and often centralized GitOps controllers that fan out desired state.
13+
In multi-cluster Kubernetes, a hub-spoke model designates one administrative cluster (the **hub**) and one or more workload clusters (**spokes**). The hub owns fleet APIs: cluster inventory, policy placement, credentials for spoke registration, and often centralized GitOps controllers that fan out desired state.
1214

1315
Spokes remain the execution venues for application namespaces, data-plane components (Kafka, MQTT bridges, mesh dataplane), and regional isolation for latency, data residency, or blast-radius control.
1416

@@ -18,29 +20,45 @@ Spokes remain the execution venues for application namespaces, data-plane compon
1820
| --- | --- |
1921
| Centralized management | One control plane for membership, RBAC patterns, and bulk upgrades. |
2022
| Policy enforcement | Kubernetes policies, compliance checks, and security baselines propagate from the hub. |
21-
| Observability | Aggregated metrics, logging, and tracing strategies start at the hub and uniform dashboards span spokes. |
23+
| Observability | Aggregated metrics, logging, and tracing strategies start at the hub; uniform dashboards span spokes. |
2224
| GitOps consistency | A single Git revision strategy (with branch or overlay variants) drives spoke drift correction. |
2325

2426
## Platform architecture overview
2527

26-
Single `main` branch: hub chart at `.`, spoke charts at `east/` and `west/`, shared `components/` referenced by all clusters.
28+
The pattern repository uses a **single `main` branch** with cluster-specific directories:
29+
30+
- Hub chart at `.` (repository root)
31+
- Spoke charts at `east/` and `west/`
32+
- Shared Helm charts under `components/` referenced by hub and spoke Application CRs
2733

2834
[![Platform architecture overview](/images/hybrid-mesh-platform/arch-overview.png)](/images/hybrid-mesh-platform/arch-overview.png)
2935

3036
[![Hub-spoke GitOps flow](/images/hybrid-mesh-platform/arch-hub-spoke-flow.png)](/images/hybrid-mesh-platform/arch-hub-spoke-flow.png)
3137

3238
## Follow the request — one temperature reading end to end
3339

34-
When a machine sensor on the east spoke publishes a temperature sample, the path is: MQTT (`messaging` broker) → Camel K (`mqtt-to-kafka` integration) → Kafka (`dev-cluster` topic) → optional ML scoring (KServe) → line-dashboard WebSocket consumer. In parallel, Thanos Querier on east scrapes Istio and Kafka metrics; a Skupper Connector (`prometheus-east`) tunnels HTTP to the hub, where Grafana datasource `prometheus-east` plots the series. The Hub Gateway can route browser traffic to the east line-dashboard via spoke-gateway and Skupper listener `ie-gateway-east`. Developer Hub Topology shows the same pods when the catalog entity carries `backstage.io/kubernetes-cluster: east` and spoke API tokens are synced.
40+
When a machine sensor on the **east** spoke publishes a temperature sample, the path is:
41+
42+
1. MQTT (`messaging` broker) → Camel K (`mqtt-to-kafka` integration)
43+
2. Kafka (`dev-cluster` topic) → optional ML scoring (KServe)
44+
3. `line-dashboard` WebSocket consumer
45+
46+
In parallel:
47+
48+
- Thanos Querier on east scrapes Istio and Kafka metrics
49+
- Skupper Connector `prometheus-east` tunnels HTTP to the hub
50+
- Hub Grafana datasource `prometheus-east` plots the series
51+
- Hub Gateway routes browser traffic to the east line-dashboard via spoke-gateway and Skupper listener `ie-gateway-east`
52+
- Developer Hub Topology shows the same pods when the catalog entity carries `backstage.io/kubernetes-cluster: east` and spoke API tokens are synced
3553

3654
[![End-to-end data flow](/images/hybrid-mesh-platform/arch-data-flow.png)](/images/hybrid-mesh-platform/arch-data-flow.png)
3755

3856
## Components on the hub vs spokes
3957

4058
| Area | Hub | Spokes | Config path |
41-
| --- | --- | --- | --- |
59+
| --- | :---: | :---: | --- |
4260
| ACM hub operator & APIs | yes | | `values.yaml` |
43-
| ArgoCD / App-of-Apps root | yes | yes | `.` / `east/` / `west/` |
61+
| Argo CD / App-of-Apps root | yes | yes | `.` / `east/` / `west/` |
4462
| ApplicationSet (spoke apps) | yes | | `values.yaml` |
4563
| ACS Central | yes | | `values.yaml` |
4664
| ACS Secured Cluster | | yes | `east/` `west/` |
@@ -61,48 +79,106 @@ When a machine sensor on the east spoke publishes a temperature sample, the path
6179
| Kubecost (agent) | | yes | `east/` `west/` |
6280
| Kafka Console (all clusters) | yes | | `values.yaml` |
6381

82+
Industrial Edge components exist **only** in spoke charts. The hub chart never includes factory workloads.
83+
6484
## GitOps application delivery flow
6585

66-
Hub syncs first; ApplicationSet pushes per-spoke charts; each spoke Argo CD reconciles child Applications locally.
86+
1. Hub Argo CD syncs the root Application (operators, ACM, gateway, observability).
87+
2. ACM ApplicationSet reads PlacementDecision and pushes `east/` and `west/` to remote clusters.
88+
3. Each spoke Argo CD reconciles child Applications locally (namespaces → operators → workloads).
6789

6890
[![GitOps sync sequence](/images/hybrid-mesh-platform/arch-gitops-sync-sequence.png)](/images/hybrid-mesh-platform/arch-gitops-sync-sequence.png)
6991

92+
Remote deployment model:
93+
94+
```
95+
Hub Argo CD → ApplicationSet
96+
→ east-spoke-components (source: east/, destination: east cluster)
97+
→ east/templates/ generates child Application CRs
98+
→ East Argo CD syncs child apps locally
99+
→ west-spoke-components (source: west/, destination: west cluster)
100+
→ west/templates/ generates child Application CRs
101+
→ West Argo CD syncs child apps locally
102+
```
103+
70104
## Sync wave ordering
71105

72-
Components deploy in strict order via ArgoCD sync waves:
106+
Components deploy in strict order via Argo CD sync waves. Lower waves complete before higher waves start.
73107

74108
[![Sync wave ordering](/images/hybrid-mesh-platform/arch-sync-waves.png)](/images/hybrid-mesh-platform/arch-sync-waves.png)
75109

76-
Sync waves prevent operators from racing workloads — mesh and namespaces land before Industrial Edge and gateways.
110+
Sync waves prevent operators from racing workloads — mesh and namespaces land before Industrial Edge and gateways. Typical progression:
77111

78-
## Service Interconnect (Skupper) topology
112+
1. Namespaces and RBAC
113+
2. Operator subscriptions (ACM, GitOps, Service Mesh, Skupper, and others)
114+
3. Platform services (Developer Hub, ACS, interconnect sites)
115+
4. Observability stack
116+
5. Industrial Edge and gateway workloads
79117

80-
Red Hat Service Interconnect creates a Virtual Application Network (VAN) that bridges services across clusters without VPN or direct network connectivity.
118+
## Service Interconnect (Skupper) topology
81119

82-
Connectors expose spoke-gateway and prometheus-auth-proxy; Listeners materialize ClusterIP services on the hub.
120+
Red Hat Service Interconnect creates a **Virtual Application Network (VAN)** that bridges services across clusters without VPN tunnels, direct routes, or firewall changes. Skupper exposes spoke Industrial Edge services and Prometheus metrics to the hub for centralized observability and ingress.
83121

84122
[![Skupper topology](/images/hybrid-mesh-platform/arch-skupper-topology.png)](/images/hybrid-mesh-platform/arch-skupper-topology.png)
85123

86124
[![Service Interconnect console topology](/images/hybrid-mesh-platform/service-interconnect-console-topology.png)](/images/hybrid-mesh-platform/service-interconnect-console-topology.png)
87125

88-
## Spoke gateway aggregation
126+
### Hub listeners (namespace `service-interconnect`)
89127

90-
Each spoke runs a Gateway API gateway that fronts all Industrial Edge services, providing a single entry point for Skupper to expose to the hub.
128+
| Listener | Port | Purpose |
129+
| --- | --- | --- |
130+
| `ie-gateway-east` | 8080 | Spoke-gateway traffic from east |
131+
| `ie-gateway-west` | 8080 | Spoke-gateway traffic from west |
132+
| `prometheus-east` | 9091 | Prometheus metrics from east |
133+
| `prometheus-west` | 9091 | Prometheus metrics from west |
134+
| `kafka-east-tst` | 9092 | Kafka bootstrap (dev-cluster) from east |
135+
| `kafka-west-tst` | 9092 | Kafka bootstrap (dev-cluster) from west |
91136

92-
One Connector per spoke exposes the gateway instead of every microservice individually.
137+
### Spoke connectors
138+
139+
| Connector | Exposes |
140+
| --- | --- |
141+
| `ie-gateway-<spoke>` | Local spoke-gateway |
142+
| `prometheus-<spoke>` | Nginx auth proxy → Thanos Querier |
143+
| `kafka-<spoke>-tst` | `dev-cluster-kafka-bootstrap` |
144+
| `kafka-<spoke>-stormshift` | `factory-cluster-kafka-bootstrap` |
145+
146+
**Link establishment:** Hub `AccessGrant` generates URL, code, and CA. Spoke `AccessToken` redeems the grant over HTTPS, then establishes the inter-router link. Use the Skupper grant-server CA from `skupper-grant-server-ca` — not the OpenShift Ingress CA — or spokes fail with `x509: certificate signed by unknown authority`.
147+
148+
```bash
149+
oc get secret skupper-grant-server-ca -n openshift-operators \
150+
-o jsonpath='{.data.ca\.crt}' | base64 -d
151+
```
152+
153+
Charts: `components/service-interconnect` (hub), `components/spoke-interconnect` (spokes).
154+
155+
## Spoke gateway aggregation
156+
157+
Each spoke runs a Gateway API gateway (`components/spoke-gateway`) that fronts all Industrial Edge services. Skupper exposes **one** gateway per spoke instead of every microservice individually.
93158

94159
## Multi-cluster observability pipeline
95160

96-
Spoke Thanos Querier is reached through nginx auth-proxy Connectors; hub Grafana uses HTTP datasources without bearer tokens.
161+
Spoke Thanos Querier is reached through nginx auth-proxy Connectors. Hub Listeners `prometheus-east` and `prometheus-west` become Grafana HTTP datasources (no bearer token from hub).
97162

98163
[![Observability pipeline](/images/hybrid-mesh-platform/arch-observability-pipeline.png)](/images/hybrid-mesh-platform/arch-observability-pipeline.png)
99164

100165
## Data flow (sensors to dashboard)
101166

102-
Telemetry path on each spoke; MirrorMaker replicates to the hub-centralized MinIO data lake.
167+
On each spoke, telemetry flows from sensors through MQTT, Camel, and Kafka to dashboards. Strimzi MirrorMaker replicates factory topics toward the hub-centralized data lake (MinIO/S3 archiver on hub and spokes).
103168

104169
## Comparison with Red Hat Validated Patterns
105170

106-
The [Multicloud GitOps](/patterns/multicloud-gitops) validated pattern demonstrates fleet GitOps with OpenShift GitOps and ACM patterns that resemble this repository's hub-push model: a declarative root Application, cluster grouping, and progressive rollout.
171+
The [Multicloud GitOps](/patterns/multicloud-gitops) validated pattern demonstrates fleet GitOps with OpenShift GitOps and ACM: declarative root Application, cluster grouping, and progressive rollout.
172+
173+
**Hybrid Mesh Platform** extends that foundation with:
174+
175+
- [Industrial Edge](/patterns/industrial-edge/) on **east** and **west** spokes (not duplicated here — see maintained pattern for OT/factory depth)
176+
- OpenShift Service Mesh 3 **ambient** mode (ztunnel L4, waypoints L7)
177+
- Connectivity Link for Gateway API ingress policy
178+
- Optional OpenShift AI (KServe) on spokes
179+
- ACS Central + SecuredCluster agents
180+
- Service Interconnect for cross-cluster service and metrics exposure
181+
182+
The result is tuned for factory-style telemetry and east-west observability, not only infrastructure provisioning.
107183

108-
This platform extends that idea with [Industrial Edge](/patterns/industrial-edge/) workloads deployed on multiple spokes, Service Mesh ambient, Connectivity Link, optional OpenShift AI, ACS depth, and Service Interconnect for cross-cluster service exposure — tuned for factory-style telemetry and east-west observability rather than only infrastructure provisioning.
184+
**Next →** [Getting Started](getting-started) to translate these diagrams into a live deployment, or [Hub Gateway](hub-gateway) for cross-cluster HTTP routing detail.

0 commit comments

Comments
 (0)