Skip to content

docs: add fleet observability phase 1 guide#2371

Merged
djwfyi merged 32 commits into
mainfrom
docs/fleet-monitoring-phase-1
Jul 20, 2026
Merged

docs: add fleet observability phase 1 guide#2371
djwfyi merged 32 commits into
mainfrom
docs/fleet-monitoring-phase-1

Conversation

@djwfyi

@djwfyi djwfyi commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Content Description

Adds a new Fleet monitoring docs section for the Phase 1 monitoring release. The new pages document the observability gateway, Write Gateway, Query Proxy, metrics access keys, edge collector configuration, PromQL querying, platform labels, and troubleshooting.

The monitoring overview now points to this release path first, while the older OpenTelemetry pages remain available as simpler or advanced collector references.

Preview Link

https://deploy-preview-2371--vcluster-docs-site.netlify.app/docs/platform/next/maintenance/observability/overview

PR adds that file and the following to the same doc folder:

  • /platform/maintenance/monitoring/fleet/install-observability-gateway
  • /platform/maintenance/monitoring/fleet/deploy-with-argocd
  • /platform/maintenance/monitoring/fleet/configure-metrics-access
  • /platform/maintenance/monitoring/fleet/configure-edge-collectors
  • /platform/maintenance/monitoring/fleet/query-fleet-metrics
  • /platform/maintenance/monitoring/fleet/labels-reference
  • /platform/maintenance/monitoring/fleet/troubleshooting

Internal Reference

Closes ENGPLAT-471

Validation:

  • git diff --check
  • npm run validate-glossary
  • npm run validate-mdx-links (LIVE-BROKEN: 0; existing fragment/partial warnings remain outside this change)
  • npm run build

AI review: mention @claude in a comment to request a review or changes. See CONTRIBUTING.md for available commands.

FORK LIMITATION: @claude does not work on pull requests opened from forks. GitHub Actions cannot access the required secrets for fork-originated PRs. To use AI review, push your branch directly to this repository.

@netlify /docs

@netlify

netlify Bot commented Jul 9, 2026

Copy link
Copy Markdown

Deploy Preview for vcluster-docs-site ready!

Name Link
🔨 Latest commit 00f8079
🔍 Latest deploy log https://app.netlify.com/projects/vcluster-docs-site/deploys/6a5e6fc604718e00081e24d1
😎 Deploy Preview https://deploy-preview-2371--vcluster-docs-site.netlify.app/docs
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Comment thread platform/maintenance/monitoring/fleet/configure-edge-collectors.mdx Outdated
Comment thread platform/maintenance/monitoring/fleet/configure-metrics-access.mdx Outdated
Comment thread platform/maintenance/observability/install-observability-gateway.mdx Outdated
Comment thread platform/maintenance/monitoring/fleet/labels-reference.mdx Outdated
Comment thread platform/maintenance/observability/query-fleet-metrics.mdx Outdated
Comment thread platform/maintenance/monitoring/fleet/query-fleet-metrics.mdx Outdated
Comment thread platform/maintenance/monitoring/fleet/troubleshooting.mdx Outdated
Comment thread platform/maintenance/monitoring/fleet/troubleshooting.mdx Outdated
Comment thread platform/maintenance/observability/troubleshooting.mdx
@djwfyi
djwfyi force-pushed the docs/fleet-monitoring-phase-1 branch from cadd8da to 817cc78 Compare July 9, 2026 21:31
@djwfyi

djwfyi commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Noticed a naming mismatch while doing a style pass on this PR's docs: fleet-monitoring-otel.mdx's panel-reference table for the vCluster projects dashboard now says "Total tenant clusters" / "Tenant clusters by Project" (renamed from "vClusters" to comply with the trademark policy against the plural "vClusters" in prose).

The downloadable dashboard JSON itself (vcluster-projects.json, linked from that same section) still has its panels literally titled "Total vClusters" and "vClusters by Project". So a reader who downloads the dashboard will see panel names that no longer match the docs table.

Fixing that means editing the panel titles in the dashboard JSON asset, which is outside a docs style pass. Flagging so it can get tracked and fixed at the source (or the docs table reverted to match, if renaming the JSON isn't planned).

@djwfyi

djwfyi commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

I added three candidate dashboard screenshots as static assets so engineering can decide whether any are worth using in the fleet monitoring docs:

  • static/img/fleet-monitoring/fleet-observability-overview-dashboard.png — strongest candidate for the fleet monitoring overview if we want to show the actual Fleet Observability dashboard with multi-project / multi-cluster metrics.
  • static/img/fleet-monitoring/fleet-observability-gpu-detail-dashboard.png — possible GPU-focused example, but only if GPU observability is intended to be a headline use case for this release.
  • static/img/fleet-monitoring/tenant-cluster-observability-dashboard.png — possible tenant-cluster scoped example for a later tenant observability page, less useful for the fleet overview.

If engineering decides these screenshots should not be used, we should delete them before merging so the PR does not carry unused binary assets.

@github-actions

Copy link
Copy Markdown
Contributor

djwfyi and others added 10 commits July 16, 2026 17:05
Fix Google.Headings sentence-case, Loft.no-latinisms, and
Google.WordList (ingest, regex) warnings across the fleet monitoring
guides. Add TLS, mTLS, and PromQL to the accepted vocab since Google.Headings
was misflagging them as improper title-case capitalization in headings.
Fix missing .mdx suffixes on relative links, remaining virtual/host
cluster terminology, contractions, Flow step verbs, run-on sentences,
and the vClusters trademark plural in a dashboard panel table.
…et monitoring

Add a dedicated page describing the default ArgoCDApplicationTemplate
resources Platform ships (metrics backend, Grafana, edge collectors) and
how to deploy them from ArgoCDApplication resources, with YAML and
Platform UI tabs.

Wire the new page into the overview teaser and page guide, the install
note, and the edge collectors tip, and renumber the sidebar to insert it
after the install page. Note the intended gap that vCluster Standalone
tenant clusters can't use the Argo CD integration, so their collectors
must be deployed manually.
"Phase 1" is internal roadmap jargon that leaked into the docs and
doesn't mean anything to readers. Frame the scope note around the paths
that actually exist today instead.
…ply snippets

Static YAML with a placeholder key value left readers to generate and
insert the secret themselves, which is easy to get wrong. Show a single
copy-paste flow that generates the key and applies the AccessKey, driven
by editable PROJECT/TENANT_CLUSTER/CLUSTER values.
…fana

The bundled Grafana is the primary way most operators will read fleet
metrics, but the page only covered raw PromQL. Explain sign-in, the
shipped dashboards, and how the scoped datasources authorize as the
signed-in user so datasource choice can't widen access.
@johannesfrey
johannesfrey force-pushed the docs/fleet-monitoring-phase-1 branch from 0e26a8b to 12c3830 Compare July 16, 2026 15:18
Comment thread platform/maintenance/observability/deploy-with-argocd.mdx Outdated
Comment thread platform/maintenance/observability/deploy-with-argocd.mdx Outdated
@johannesfrey

Copy link
Copy Markdown
Contributor

I added three candidate dashboard screenshots as static assets so engineering can decide whether any are worth using in the fleet monitoring docs:

  • static/img/fleet-monitoring/fleet-observability-overview-dashboard.png — strongest candidate for the fleet monitoring overview if we want to show the actual Fleet Observability dashboard with multi-project / multi-cluster metrics.
  • static/img/fleet-monitoring/fleet-observability-gpu-detail-dashboard.png — possible GPU-focused example, but only if GPU observability is intended to be a headline use case for this release.
  • static/img/fleet-monitoring/tenant-cluster-observability-dashboard.png — possible tenant-cluster scoped example for a later tenant observability page, less useful for the fleet overview.

If engineering decides these screenshots should not be used, we should delete them before merging so the PR does not carry unused binary assets.

Thx. Included them in the new Grafana section.

@johannesfrey

johannesfrey commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Noticed a naming mismatch while doing a style pass on this PR's docs: fleet-monitoring-otel.mdx's panel-reference table for the vCluster projects dashboard now says "Total tenant clusters" / "Tenant clusters by Project" (renamed from "vClusters" to comply with the trademark policy against the plural "vClusters" in prose).

The downloadable dashboard JSON itself (vcluster-projects.json, linked from that same section) still has its panels literally titled "Total vClusters" and "vClusters by Project". So a reader who downloads the dashboard will see panel names that no longer match the docs table.

Fixing that means editing the panel titles in the dashboard JSON asset, which is outside a docs style pass. Flagging so it can get tracked and fixed at the source (or the docs table reverted to match, if renaming the JSON isn't planned).

IMHO it's a bit confusing that we now have "Fleet monitoring" and "Fleet monitoring with OpenTelemetry" next to each other. Are there plans to consolidate them or at least move them further apart? cc @cbron @Stillstellung

Screenshot 2026-07-16 at 18 10 03

@johannesfrey
johannesfrey force-pushed the docs/fleet-monitoring-phase-1 branch from 75caa03 to 4d64993 Compare July 16, 2026 16:36
@johannesfrey
johannesfrey force-pushed the docs/fleet-monitoring-phase-1 branch from 4d64993 to 951cc77 Compare July 16, 2026 16:39
@Stillstellung

Copy link
Copy Markdown

Noticed a naming mismatch while doing a style pass on this PR's docs: fleet-monitoring-otel.mdx's panel-reference table for the vCluster projects dashboard now says "Total tenant clusters" / "Tenant clusters by Project" (renamed from "vClusters" to comply with the trademark policy against the plural "vClusters" in prose).
The downloadable dashboard JSON itself (vcluster-projects.json, linked from that same section) still has its panels literally titled "Total vClusters" and "vClusters by Project". So a reader who downloads the dashboard will see panel names that no longer match the docs table.
Fixing that means editing the panel titles in the dashboard JSON asset, which is outside a docs style pass. Flagging so it can get tracked and fixed at the source (or the docs table reverted to match, if renaming the JSON isn't planned).

IMHO it's a bit confusing that we now have "Fleet monitoring" and "Fleet monitoring with OpenTelemetry" next to each other. Are there plans to consolidate them or at least move them further apart? cc @cbron @Stillstellung

Screenshot 2026-07-16 at 18 10 03

I spoke w/ @djwfyi and he's going to think about which path to take, but you're right, it's a little clunky as is

@johannesfrey

johannesfrey commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Noticed a naming mismatch while doing a style pass on this PR's docs: fleet-monitoring-otel.mdx's panel-reference table for the vCluster projects dashboard now says "Total tenant clusters" / "Tenant clusters by Project" (renamed from "vClusters" to comply with the trademark policy against the plural "vClusters" in prose).
The downloadable dashboard JSON itself (vcluster-projects.json, linked from that same section) still has its panels literally titled "Total vClusters" and "vClusters by Project". So a reader who downloads the dashboard will see panel names that no longer match the docs table.
Fixing that means editing the panel titles in the dashboard JSON asset, which is outside a docs style pass. Flagging so it can get tracked and fixed at the source (or the docs table reverted to match, if renaming the JSON isn't planned).

IMHO it's a bit confusing that we now have "Fleet monitoring" and "Fleet monitoring with OpenTelemetry" next to each other. Are there plans to consolidate them or at least move them further apart? cc @cbron @Stillstellung
Screenshot 2026-07-16 at 18 10 03

I spoke w/ @djwfyi and he's going to think about which path to take, but you're right, it's a little clunky as is

Thank you. Also, we recently switched terms from monitoring/metrics to observability. So probably a chance to introduce "Observability" under "Maintenance" and start consolidating/removing older stuff there?!

djwfyi added 3 commits July 16, 2026 17:26
Screenshots still go stale as the UI changes, but blocking them
outright was too strict for cases like dashboard views that are
hard to describe in prose.
Promotes the new gateway-based feature out of monitoring/fleet into
its own Observability category alongside Monitoring and Logging under
Maintenance, fixing the nav proximity confusion with the older
"Fleet monitoring with OpenTelemetry" page. Also clarifies that this
release covers metrics only, softens an overclaiming "full-stack
observability" line, and cross-links vCluster's own logging guide for
readers who land here looking for log collection.
Only Fleet Observability routes through the Write Gateway and Query
Proxy; the older OpenTelemetry guides push directly to the backend.
Also clarifies that bundled Grafana dashboards ship today (only their
lifecycle automation is future work), and swaps "Fleet Observability
data" for "fleet metrics" to keep the product name off the metrics-only
payload.
@djwfyi

djwfyi commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed three follow-up commits addressing feedback on the fleet monitoring section's naming and info architecture:

59d7a6170 — docs: allow platform screenshots when warranted
Relaxes the "no UI screenshots" rule in CLAUDE.md to allow them where warranted (e.g. dashboard views that are hard to describe in prose), rather than banning them outright.

e67920f44 — docs(observability): rename fleet monitoring to fleet observability

  • Renamed the feature from "Fleet monitoring" to "Fleet Observability" throughout the new section (titles, descriptions, prose, link text).
  • Moved platform/maintenance/monitoring/fleet/ to platform/maintenance/observability/, promoting it to a sibling category of Monitoring and Logging under Maintenance instead of nesting it inside Monitoring — this was to resolve nav proximity confusion with the older "Fleet monitoring with OpenTelemetry" page.
  • Renamed the diagram SVG and dashboard screenshot asset paths to match.
  • Added a scope note clarifying this release covers metrics only (no logs/traces), softened an overclaiming "full-stack observability" line, and cross-linked vCluster's own logging guide for readers who land here looking for log collection.

79daf4141 — docs(observability): distinguish gateway path from direct otel pipelines

  • Monitoring overview no longer implies all fleet-wide approaches route through the Platform observability gateway — only Fleet Observability does; the two OpenTelemetry guides push directly to the backend via remote_write.
  • Clarified that bundled Grafana dashboards ship today; only their per-tenant lifecycle automation is future work.
  • Reworded "Fleet Observability data" to "fleet metrics" to keep the product name off the metrics-only payload description.

All three verified clean against vale, validate-glossary, and validate-mdx-links before pushing.

djwfyi added 2 commits July 16, 2026 18:14
…racies

Corrects several implementation-accuracy gaps found by comparing the docs
against loft-enterprise:

- The fleet connector label is platform.vcluster.com/fleet-observability-connector,
  not loft.sh/fleet-observability-connector.
- The gateway always serves OTLP and query traffic over TLS; docs treated it
  as optional and used http:// in examples.
- metricsBackendCaData and observabilityCaSecretName aren't read by the
  controller; removed them and documented metricsBackendCertSecretName's
  actual ca.crt/tls.crt/tls.key requirement.
- Documents serviceType, serviceAnnotations, and the Grafana connector keys
  (grafanaUrl, grafanaCertSecretName, fleetObservabilityDashboardUid) that
  the controller and UI actually use.
- The connector ID isn't required to be "fleet-observability"; gateway
  resource names derive from the connector Secret's own name.
- Reframes datasource lifecycle management as already implemented today
  (per-VCI and per-cluster reconcilers exist); only per-tenant connector
  selection and access-key generation remain future work.
- Adds the vCluster Platform license as an explicit prerequisite, since the
  controllers aren't registered at all without it.
- Uses the bundled Prometheus template's exact endpoints, including the
  required /api/v1/otlp path.
- Fixes control plane cluster casing to lowercase per house style.

Left the cluster-collector template's current kube-state-metrics/node-exporter
description and the otlpEndpoint default intact: loft-enterprise PRs #7526 and
#7532 will change that behavior, but both are still open, so documenting the
pre-merge state would be premature.
The query examples referenced $GATEWAY_CA_FILE without showing how to
extract it from the gateway's serving certificate Secret.
@djwfyi

djwfyi commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed two more commits fixing implementation-accuracy gaps found by comparing the docs against loft-enterprise:

fdc1b098f — docs(observability): fix connector label, gateway TLS, and key inaccuracies

  • The fleet connector label is platform.vcluster.com/fleet-observability-connector, not loft.sh/fleet-observability-connector (verified against pkg/constants/observability.go) — the docs' own example would have silently produced a gateway Platform doesn't recognize as the fleet connector.
  • The gateway always serves OTLP and query traffic over TLS; it's never optional. Removed the http:// examples and documented the self-signed default plus how to supply a custom certificate (verified against gateway_controller.go and datasource_shared.go).
  • Removed metricsBackendCaData and observabilityCaSecretName, which the controller never reads, and documented that metricsBackendCertSecretName actually requires ca.crt, tls.crt, and tls.key together (verified against manifests.go).
  • Documented serviceType, serviceAnnotations, and the Grafana connector keys (grafanaUrl, grafanaCertSecretName, fleetObservabilityDashboardUid) that the controller and UI already use but weren't in the connector reference.
  • The connector ID doesn't have to be fleet-observability — gateway resource names derive from the connector Secret's own name; the fleet designation comes from the label instead.
  • Reframed datasource lifecycle management as already implemented (per-VCI and per-cluster reconcilers exist today); only per-tenant connector selection and access-key generation are genuinely future work.
  • Added the vCluster Platform license as an explicit prerequisite, since the observability controllers aren't registered at all without the FleetObservability license feature.
  • Bundled Prometheus template instructions now use its exact endpoints, including the required /api/v1/otlp path.
  • Fixed Control Plane Cluster casing to lowercase in prose per house style (left literal UI label references like <Label>Control Plane Clusters</Label> alone).

One thing deliberately not changed: the cluster-collector template's current kube-state-metrics/node-exporter description and the otlpEndpoint default. Two loft-enterprise PRs (#7526, #7532) will change that behavior, but both are still open as of this push, so documenting the pre-merge state would make the docs wrong today. Worth a follow-up once those land.

e7abbafe4 — docs(observability): show how to populate GATEWAY_CA_FILE
The query examples referenced $GATEWAY_CA_FILE without showing how to extract it from the gateway's serving certificate Secret. Added the kubectl get secret ... | base64 -d one-liner.

All changes verified against the actual loft-enterprise controller and UI source (not just asserted), and checked clean against vale, validate-glossary, and validate-mdx-links before pushing.

@seanschneeweiss

Copy link
Copy Markdown
Contributor

Added a LoadBalancer exposure section for the gateway (serviceType/annotations, address retrieval, per-port endpoint table for OTLP + query), plus an OTLP wording pass. Clean against validate-glossary and the build.

@seanschneeweiss

Copy link
Copy Markdown
Contributor

Added an out-of-the-box "Get started" section, switched the tabbed examples to UI-first, and reordered the observability pages for the happy path (Grafana-first querying, Argo CD grouped by deployment target, flow-ordered page guide).

@djwfyi
djwfyi marked this pull request as ready for review July 20, 2026 18:46
@djwfyi
djwfyi requested a review from a team as a code owner July 20, 2026 18:46
…tel guide

.vale.ini and CLAUDE.md conflicts were additive on both sides; kept both
blocks. In the otel guide, prefer main's UI-drift accuracy fixes (button
label, nav path) and its more precise "collection time" wording, but keep
this branch's "and so on" (matches the Loft.no-latinisms substitution
exactly) and its "tenant cluster" terminology fix (vCluster is not the name
of the cluster it creates).
@djwfyi djwfyi changed the title docs: add fleet monitoring phase 1 guide docs: add fleet observability phase 1 guide Jul 20, 2026
@djwfyi
djwfyi merged commit c2dc818 into main Jul 20, 2026
13 checks passed
@djwfyi
djwfyi deleted the docs/fleet-monitoring-phase-1 branch July 20, 2026 19:03
djwfyi pushed a commit that referenced this pull request Jul 20, 2026
* Backport: Copy platform/maintenance/monitoring/central-hostpath-mapper.mdx to platform_versioned_docs/version-4.11.0/maintenance/monitoring/central-hostpath-mapper.mdx

* Backport: Copy platform/maintenance/monitoring/fleet-monitoring-otel.mdx to platform_versioned_docs/version-4.11.0/maintenance/monitoring/fleet-monitoring-otel.mdx

* Backport: Copy platform/maintenance/monitoring/metrics.mdx to platform_versioned_docs/version-4.11.0/maintenance/monitoring/metrics.mdx

* Backport: Copy platform/maintenance/monitoring/overview.mdx to platform_versioned_docs/version-4.11.0/maintenance/monitoring/overview.mdx

* Backport: Copy platform/maintenance/monitoring/prometheus-shared-nodes.mdx to platform_versioned_docs/version-4.11.0/maintenance/monitoring/prometheus-shared-nodes.mdx

* Backport: Copy platform/maintenance/observability/_category_.json to platform_versioned_docs/version-4.11.0/maintenance/observability/_category_.json

* Backport: Copy platform/maintenance/observability/configure-edge-collectors.mdx to platform_versioned_docs/version-4.11.0/maintenance/observability/configure-edge-collectors.mdx

* Backport: Copy platform/maintenance/observability/configure-metrics-access.mdx to platform_versioned_docs/version-4.11.0/maintenance/observability/configure-metrics-access.mdx

* Backport: Copy platform/maintenance/observability/deploy-with-argocd.mdx to platform_versioned_docs/version-4.11.0/maintenance/observability/deploy-with-argocd.mdx

* Backport: Copy platform/maintenance/observability/install-observability-gateway.mdx to platform_versioned_docs/version-4.11.0/maintenance/observability/install-observability-gateway.mdx

* Backport: Copy platform/maintenance/observability/labels-reference.mdx to platform_versioned_docs/version-4.11.0/maintenance/observability/labels-reference.mdx

* Backport: Copy platform/maintenance/observability/overview.mdx to platform_versioned_docs/version-4.11.0/maintenance/observability/overview.mdx

* Backport: Copy platform/maintenance/observability/query-fleet-metrics.mdx to platform_versioned_docs/version-4.11.0/maintenance/observability/query-fleet-metrics.mdx

* Backport: Copy platform/maintenance/observability/troubleshooting.mdx to platform_versioned_docs/version-4.11.0/maintenance/observability/troubleshooting.mdx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants