docs: add fleet observability phase 1 guide (#2371)

docs(observability): add fleet observability guides

Add the new gateway-based fleet observability doc set: install and
Argo CD deployment, metrics access keys, edge collector configuration,
querying via PromQL and the bundled Grafana, platform labels, and
troubleshooting. Point the monitoring overview at these guides first,
keeping the older OpenTelemetry remote_write pages as a simpler or
more advanced alternative.

Promote the feature out of monitoring/fleet into its own Observability
category to fix nav confusion with the older OpenTelemetry page, and
correct several implementation-accuracy gaps found by comparing the
docs against loft-enterprise: the connector label key, gateway TLS
being mandatory rather than optional, the access-key secret's actual
fields, and which datasource lifecycle pieces are already implemented
versus future work.

Also loosens the docs style rule against platform UI screenshots for
cases like dashboard views that are hard to describe in prose.

Closes ENGPLAT-471

Co-authored-by: Johannes Frey <johannes.frey@loft.sh>
Co-authored-by: Sean Schneeweiss <sean.schneeweiss@vcluster.com>

Author: 3 people

.github/styles/config/vocabularies/Loft/accept.txt

@@ -81,6 +81,9 @@ SSH
 SSHKey[s]?
 StatefulSet
 TSNet
+TLS
+mTLS
+PromQL
 URL
 vCluster
 vcluster.yaml
@@ -89,4 +92,4 @@ rpm.vcluster.com
 [vVirtual] [cC]luster
 Velero
 VolumeAttachment
-(?i)yaml
+(?i)yaml

.vale.ini

@@ -76,6 +76,17 @@ Vale.Terms = NO
 BasedOnStyles = Loft, Google, Vale
 Vale.Terms = NO
 
+# `metrics-backend-mtls` is a literal Kubernetes Secret name (must stay
+# lowercase per RFC 1123), and "Deploy Grafana" is correct sentence case with
+# a proper noun that Google.Headings doesn't recognize as one. Neither needs
+# to change, so scope both rules off for this page instead of the vocab (which
+# would create new false positives on the "grafana" anchors and reverse-proxy
+# path used throughout the fleet monitoring pages).
+[**/observability/deploy-with-argocd.mdx]
+BasedOnStyles = Loft, Google, Vale
+Vale.Terms = NO
+Google.Headings = NO
+
 # Auto-generated by scripts/generate-platform-cli-index.js from
 # vcluster/cli/vcluster_platform*.md. The regenerate-instructions comment
 # references that file path in plain text, and command descriptions are

CLAUDE.md

@@ -178,7 +178,10 @@ When writing docs for new features:
 ### Examples format
 
 - vcluster.yaml: description + code block
-- Platform: description + step-by-step (no UI screenshots - they go stale)
+- Platform: description + step-by-step. Minimize UI screenshots since the UI
+  changes often and screenshots go stale, but they're okay where warranted
+  (for example, illustrating a dashboard or layout that's hard to describe in
+  prose).
 
 ### Checklist
 

platform/maintenance/monitoring/central-hostpath-mapper.mdx

@@ -1,7 +1,7 @@
 ---
 title: Central HostPath Mapper
 sidebar_label: Central HostPath Mapper
-sidebar_position: 6
+sidebar_position: 7
 sidebar_class_name: free
 ---
 
@@ -17,9 +17,9 @@ import FeatureTable from '@site/src/components/FeatureTable';
 
 You can install a common logging solution such as Fluentd or the ELK stack on the Control Plane Cluster and use it within the tenant cluster. However, in some scenarios, installing these tools directly on the tenant cluster is preferred.
 
-These logging solutions access pod and container logs by mounting node host paths and using specific filename formats to associate logs with the correct pods. This approach doesn’t work in a vCluster because the vCluster rewrites pod names to a vCluster-specific format when syncing them to the Control Plane Cluster.
+These logging solutions access pod and container logs by mounting node host paths and using specific filename formats to associate logs with the correct pods. This approach doesn't work in a tenant cluster because vCluster rewrites pod names to a vCluster-specific format when syncing them to the Control Plane Cluster.
 
-The HostPath Mapper collects logs from tenant clusters. It works by matching virtual pod and container names to their physical names on the host and does this through symbolic links.
+The HostPath Mapper collects logs from tenant clusters. It works by matching virtual pod and container names to their physical names on the host through symbolic links.
 
 To enable HostPath Mapper for a tenant cluster, install a DaemonSet in the same namespace. This setup works well for small environments with few nodes and tenant clusters.
 
@@ -32,7 +32,7 @@ To use this feature, install the Central HostPath Mapper on the Control Plane Cl
 :::
 
 :::warning Templates
-This feature is only configurable from the UI for tenant clusters created without a template. When using a custom template that has the feature enabled, the UI does not display the configuration in **Advanced Options** as described below. In this case, configure the feature using the CLI or by modifying the manifests.
+This feature is only configurable from the UI for tenant clusters created without a template. When using a custom template that has the feature enabled, the UI doesn't display the configuration in **Advanced Options** as described below. In this case, configure the feature using the CLI or by modifying the manifests.
 :::
 
 <Tabs
@@ -45,28 +45,28 @@ This feature is only configurable from the UI for tenant clusters created withou
   <TabItem value="creation">
     <Flow id="enable-central-hpm-virtualcluster-creation">
       <Step>
-        From the project drop-down menu, select the project you'd like to create
-        the tenant cluster in.
+        From the project drop-down menu, click the project where you want to
+        create the tenant cluster.
       </Step>
       <Step>
-        Click the <Button>New Tenant Cluster</Button> button above the list of tenant clusters
+        Click the <Button>New Tenant Cluster</Button> button above the list of tenant clusters.
       </Step>
       <Step>
-        <b>[Optional]</b> select the cluster in which to create the tenant
+        <b>[Optional]</b> Click the cluster where you want to create the tenant
         cluster.
       </Step>
       <Step>
         Leave the <Label>Select a template</Label> section blank and click the <Button>Configure</Button> to continue.
       </Step>
       <Step>
         Click the <Label>Advanced Options</Label> configuration tab and expand
-        the 'Host Path Mapper' section.
+        the **Host Path Mapper** section.
       </Step>
       <Step>
-        Toggle the slider <Label>Enable</Label>. This will add the
-        'loft.sh/hpm-enabled' annotation to the template metadata which will be
-        automatically synced to the tenant cluster on creation. Secondly, it
-        will add `enabled: true` & `central: true` under the '[controlPlane.hostPathMapper](/docs/vcluster/configure/vcluster-yaml/control-plane/components/host-path-mapper)' field in the vcluster.yaml.
+        Toggle the slider <Label>Enable</Label>. This adds the
+        `loft.sh/hpm-enabled` annotation to the template metadata, which syncs
+        automatically to the tenant cluster on creation. It also sets
+        `enabled: true` and `central: true` under the [controlPlane.hostPathMapper](/docs/vcluster/configure/vcluster-yaml/control-plane/components/host-path-mapper) field in the vcluster.yaml.
       </Step>
       <Step>
         Finish configuring anything else on your tenant cluster,
@@ -78,21 +78,22 @@ This feature is only configurable from the UI for tenant clusters created withou
   <TabItem value="existing">
     <Flow id="enable-central-hpm-virtualcluster-existing">
       <Step>
-        Select the project that contains the tenant cluster you want to enable Central HostPath Mapper on from the project drop-down menu.
+        From the project drop-down menu, click the project that contains the
+        tenant cluster where you want to enable Central HostPath Mapper.
       </Step>
       <Step>
-        Hover over the tenant cluster you would like to enable Central HostPath Mapper on to see an <Label>Edit</Label> button displayed.
+        Hover over the tenant cluster where you want to enable Central HostPath Mapper to reveal the <Label>Edit</Label> button.
         Click <Label>Edit</Label>.
       </Step>
       <Step>
         Click the <Label>Advanced Options</Label> configuration tab and expand
         the **Host Path Mapper** section.
       </Step>
       <Step>
-        Toggle the slider <Label>Enable</Label>. This will add the
-        'loft.sh/hpm-enabled' annotation to the template metadata which will be
-        automatically synced to the tenant cluster on creation. Secondly, it
-        will add `enabled: true` & `central: true` under the '[controlPlane.hostPathMapper](/docs/vcluster/configure/vcluster-yaml/control-plane/components/host-path-mapper)' field in the vcluster.yaml.
+        Toggle the slider <Label>Enable</Label>. This adds the
+        `loft.sh/hpm-enabled` annotation to the template metadata, which syncs
+        automatically to the tenant cluster on creation. It also sets
+        `enabled: true` and `central: true` under the [controlPlane.hostPathMapper](/docs/vcluster/configure/vcluster-yaml/control-plane/components/host-path-mapper) field in the vcluster.yaml.
       </Step>
       <Step>
         Finish configuring anything else on your tenant cluster,

platform/maintenance/monitoring/fleet-monitoring-otel.mdx

@@ -1,7 +1,7 @@
 ---
 title: "Fleet monitoring with OpenTelemetry"
 sidebar_label: "Fleet monitoring with OpenTelemetry"
-sidebar_position: 3
+sidebar_position: 4
 description: "Monitor tenant clusters across shared and private nodes using OpenTelemetry Collectors with automatic label enrichment and Prometheus remote write."
 ---
 
@@ -160,7 +160,7 @@ node. The `groupbyattrs` processor splits the batch into per-pod resource scopes
 (by `namespace`, `pod`, `node`) so each pod is matched correctly.
 
 The `k8sattributes` processor resolves vCluster identity from Platform-managed
-namespace labels (`loft.sh/project`, `loft.sh/vcluster-instance-name`, and others) and
+namespace labels (`loft.sh/project`, `loft.sh/vcluster-instance-name`, and so on) and
 pod labels/annotations set by the vCluster syncer
 (`vcluster.loft.sh/namespace`, `vcluster.loft.sh/name`).
 
@@ -170,7 +170,7 @@ which also prevents duplicate series with any existing Prometheus scrapes.
 
 ### Private nodes
 
-The private-nodes collector runs inside each vCluster as a DaemonSet with one
+The private-nodes collector runs inside each tenant cluster as a DaemonSet with one
 pod per node. Each pod scrapes only its local node's kubelet `/metrics`,
 cAdvisor `/metrics/cadvisor`, and API server `/metrics` endpoints.
 
@@ -188,14 +188,14 @@ prometheus receiver (kubelet, cAdvisor, API server)
                             pod → vcluster_virtual_pod)
 ```
 
-Since the collector runs inside the vCluster, it can't access host-cluster
-namespace labels. Instead, the Platform injects `{{ .Values.loft.* }}` template
+Since the collector runs inside the tenant cluster, it can't access Control
+Plane Cluster namespace labels. Instead, the Platform injects `{{ .Values.loft.* }}` template
 variables at deploy time, which are set as `external_labels` on the
-`prometheusremotewrite` exporter. These are static per-vCluster values applied
+`prometheusremotewrite` exporter. These are static per-tenant-cluster values applied
 to all exported metrics.
 
 The `metric_relabel_configs` copy `namespace` to `vcluster_virtual_namespace`
-and `pod` to `vcluster_virtual_pod`. Inside a private-nodes vCluster, the
+and `pod` to `vcluster_virtual_pod`. Inside a private-nodes tenant cluster, the
 `namespace` and `pod` labels already represent virtual names, so this copy
 ensures dashboard compatibility with the shared-nodes collector.
 
@@ -219,7 +219,7 @@ All metrics from both apps carry a consistent set of identity labels:
 `vcluster_virtual_namespace` and `vcluster_virtual_pod` are missing on some
 metrics. These are vCluster system pods (syncer, CoreDNS) that don't have the
 syncer labels and annotations because they aren't user workloads synced from
-inside the vCluster.
+inside the tenant cluster.
 :::
 
 ## Prerequisites
@@ -248,7 +248,7 @@ server:
 
 - kubelet scraping disabled in any existing kube-prometheus-stack to avoid
   duplicate cAdvisor series (`kubelet.enabled: false`).
-- Platform namespace labels present (added automatically by the vCluster Platform).
+- Platform namespace labels present (added automatically by vCluster Platform).
 
 ### Private nodes prerequisites
 
@@ -593,7 +593,7 @@ spec:
 - **`serviceMonitorSelector: app: vcluster`**: Without filtering, the Target
   Allocator discovers all ServiceMonitors in the cluster, overwhelming
   collectors with memory pressure.
-- **`operator.targetallocator.mtls: true`**: Each vCluster exposes its API
+- **`operator.targetallocator.mtls: true`**: Each tenant cluster exposes its API
   server metrics over mTLS. Without this feature gate, the Target Allocator
   redacts TLS private keys when passing scrape configs to collectors.
 - **`otel/opentelemetry-collector-contrib` image**: The default image doesn't
@@ -622,7 +622,7 @@ kubectl apply -f otel-collector-shared-nodes-app.yaml
     Navigate to the <NavStep>Apps</NavStep> tab.
   </Step>
   <Step>
-    Click <Button>Install App</Button> and select the
+    Click <Button>Install App</Button> and click the
     **OTEL Collector - Shared Nodes** app.
   </Step>
   <Step>
@@ -645,7 +645,7 @@ Repeat these steps for each Control Plane Cluster.
 ## Deploy the private nodes collector
 <!-- vale on -->
 
-Deploy one private-nodes collector into each private-nodes vCluster. All
+Deploy one private-nodes collector into each private-nodes tenant cluster. All
 vCluster identity labels are injected automatically by the Platform using
 `{{ .Values.loft.* }}`.
 
@@ -889,7 +889,7 @@ spec:
   ensures one collector per node, scraping only the local kubelet and cAdvisor
   using `${env:K8S_NODE_NAME}` filtering. No Target Allocator is needed.
 - **`external_labels` instead of `k8sattributes` for identity**: The collector
-  runs inside the vCluster, so it can't access Control Plane Cluster namespace labels.
+  runs inside the tenant cluster, so it can't access Control Plane Cluster namespace labels.
   The Platform injects `{{ .Values.loft.* }}` template variables at deploy time.
 - **`resource_to_telemetry_conversion: false`**: Setting this to `true` causes
   duplicate labels that break Grafana dashboards.
@@ -916,7 +916,7 @@ kubectl apply -f otel-collector-private-nodes-app.yaml
     Go to the <NavStep>Projects</NavStep> section using the menu on the left.
   </Step>
   <Step>
-    Select the project containing your private-nodes tenant cluster.
+    Click the project containing your private-nodes tenant cluster.
   </Step>
   <Step>
     Click on the tenant cluster, then navigate to the <NavStep>Config</NavStep>
@@ -1215,7 +1215,8 @@ with the event rate.
 
 :::note
 This section assumes Grafana is already deployed. For setup instructions, see
-[Aggregating Metrics — Deploy Grafana](./aggregating-metrics.mdx#deploy-grafana).
+the [Deploy Grafana](./aggregating-metrics.mdx#deploy-grafana) section in
+Aggregating Metrics.
 :::
 
 Two Grafana dashboards are provided for visualizing metrics collected by the
@@ -1234,14 +1235,12 @@ time, so all panels use straightforward PromQL queries without joins.
 ### vCluster projects dashboard
 <!-- vale on -->
 
-A platform admin overview of vCluster projects across shared and private node
-tenant clusters. Use this dashboard to monitor resource consumption and API health at
-the project level.
+A platform admin overview of vCluster projects across Shared Nodes and Private
+Nodes tenant clusters. Use this dashboard to monitor resource consumption and
+API health at the project level.
 
 **Dashboard JSON file:** [Download dashboard JSON](/media/dashboards/vcluster-projects.json)
 
-
-
 **Template variables:**
 
 | Variable | Description |
@@ -1255,8 +1254,8 @@ the project level.
 <!-- vale off -->
 | Section | Panels |
 |---------|--------|
-| Project summary | Total Projects, Total vClusters, Total Pods, Total CPU (cores), Total Memory, Max Error Rate (stat panels) |
-| Resource usage by project | CPU by Project (cores), Memory by Project, Pods by Project, vClusters by Project (bar charts) |
+| Project summary | Total Projects, Total tenant clusters, Total Pods, Total CPU (cores), Total Memory, Max Error Rate (stat panels) |
+| Resource usage by project | CPU by Project (cores), Memory by Project, Pods by Project, Tenant clusters by Project (bar charts) |
 | API health by project | API Request Rate, API Error Rate (5xx), API P95 Latency (time series) |
 | Resource trends | CPU Usage Trend (stacked), Memory Usage Trend (stacked) (time series) |
 <!-- vale on -->