[v4.10.0] docs(doc-1573): clarify platform database, move to air-gapped section, split troubleshoot (#2332)

platform_versioned_docs/version-4.10.0/administer/clusters/advanced/external-database/external-database.mdx

@@ -9,6 +9,10 @@ import VersionBadge from '@site/src/components/VersionBadge';
 
 <VersionBadge platformVersion="v4.8.0"/>
 
+:::note Unrelated to the Platform Database feature
+This page covers the Kine-backed external database used for high-availability multi-replica deployments. If you are looking for the local PostgreSQL instance used for usage reporting in air-gapped environments, see [Platform Database](../../../../install/air-gapped/platform-database.mdx). That is a separate, lighter-weight feature gated to air-gapped installs with offline license keys.
+:::
+
 ## What is an external database deployment?
 
 An external database deployment runs multiple vCluster Platform replicas in a

platform_versioned_docs/version-4.10.0/install/air-gapped/platform-database.mdx

@@ -0,0 +1,65 @@
+---
+title: Platform Database
+sidebar_label: Platform Database
+sidebar_position: 3
+description: The platform database is a local PostgreSQL instance that stores usage snapshots for license reporting in air-gapped installations.
+---
+
+import VersionBadge from '@site/src/components/VersionBadge';
+
+<VersionBadge platformVersion="v4.4.0"/>
+
+The platform database is a local PostgreSQL instance provisioned only in air-gapped environments running an [offline license key](./with-offline-license-key.mdx). It stores usage snapshots, which are periodic records of CPU and GPU resource consumption used for license billing. The platform attempts to transmit these snapshots to the vCluster license server every 15 minutes. The local database acts as a staging area. Once a snapshot is successfully transmitted, vCluster's license server holds the authoritative record. Data that has not yet been transmitted exists only in this database.
+
+The platform database does not store core platform state. Tenant cluster definitions, project settings, users, and all other platform objects are stored as Kubernetes custom resources on the control plane cluster. This database exists solely for usage reporting.
+
+:::note Unrelated to the External Database feature
+This page covers the local PostgreSQL instance used for usage reporting in air-gapped environments. If you need high availability with multiple platform replicas, see [Platform External Database](../../administer/clusters/advanced/external-database/external-database.mdx). That is a separate feature using a Kine-backed PostgreSQL database as a shared backing store for multi-replica HA deployments, and it requires a fresh installation configured before the platform first starts.
+:::
+
+## When the platform database is provisioned
+
+The platform database is only created when all of the following conditions are true at startup:
+
+- The platform is configured with an [offline license key](./with-offline-license-key.mdx) (air-gapped mode).
+- The platform is not running as a hosted instance (managed and operated by vCluster Labs on your behalf).
+- The active license plan does not disable the platform database.
+
+Standard online-licensed installations do not provision this database. If you do not see a `platform-db` PVC or StatefulSet in your platform namespace, the database is not active in your environment and this page does not apply.
+
+## How the platform database surfaces to administrators
+
+Administrators encounter the platform database in several situations:
+
+**Namespace inspection.** The platform database runs as a StatefulSet in the platform namespace. A routine `kubectl get all` or `kubectl get pvc` will show the `platform-db` StatefulSet, Service, PVC, and Secret alongside platform resources.
+
+**Cluster conditions.** The controller sets `EmbeddedPostgresDeployed` and `EmbeddedPostgresAvailable` conditions on the `Cluster` resource. If the database fails to start, `kubectl describe cluster.storage.loft.sh <cluster-name>` shows a condition message directing you to investigate.
+
+**Platform logs.** If the database is unreachable, the platform logs `error sending usage snapshot` every 15 minutes. Administrators tailing logs during an incident will see this repeating.
+
+**Platform upgrades.** The controller restarts the postgres pods during platform upgrades. Administrators watching pod activity during an upgrade will see `platform-db` pods cycling, which is expected behavior.
+
+See [Troubleshoot Platform Database](../../troubleshoot/platform-database.mdx) for steps to resolve conditions and log errors.
+
+## What usage snapshots record
+
+Each snapshot captures a point-in-time count of resource capacity across the platform, not live workload utilization. The data is used to calculate consumption against the committed amounts in your license.
+
+| Metric | Description |
+|---|---|
+| Tenant cluster count | Total number of tenant clusters across the platform |
+| Node count | Total number of nodes across all connected clusters |
+| CPU count | Total CPU capacity across all nodes |
+| GPU count | Total GPU capacity across all nodes |
+| Per-node detail | Machine ID, creation timestamp, and full resource capacity for each node |
+| Per-tenant-cluster detail | Name, namespace, UID, creation timestamp, and node associations |
+
+## Configure
+
+By default, the platform database uses the default StorageClass. To specify a different StorageClass, navigate to `Admin > Config` and edit the `platformDB.storageClass` field.
+The value can also be configured in the vCluster Platform Helm values during install or upgrade:
+
+```yaml
+platformDB:
+  storageClass: <storage-class>
+```

platform_versioned_docs/version-4.10.0/troubleshoot/platform-database.mdx

@@ -0,0 +1,66 @@
+---
+title: Troubleshoot Platform Database
+sidebar_label: Platform Database
+description: Diagnose and resolve platform database issues in air-gapped installations.
+---
+
+The platform database is a local PostgreSQL instance provisioned only in air-gapped environments. For background on what it is and how it is configured, see [Platform Database](../install/air-gapped/platform-database.mdx).
+
+If the platform UI shows an **Action Required: Usage History Database Down** banner, the postgres pod is unhealthy. Start with the diagnostics below.
+
+### Diagnose a postgres pod that fails to start
+
+If `kubectl describe cluster.storage.loft.sh <cluster-name>` shows `EmbeddedPostgresAvailable=False`, the postgres pod is not healthy. Check the pod status and events in the platform namespace:
+
+```bash
+kubectl get pods -n <platform-namespace> -l loft.sh/system-app=platform-db
+kubectl describe pod -n <platform-namespace> -l loft.sh/system-app=platform-db
+```
+
+Common causes are storage provisioning failures (the PVC cannot bind to a PersistentVolume) and image pull errors. Resolve the underlying pod issue and the condition should clear on the next reconcile.
+
+### Repeated "error sending usage snapshot" in logs
+
+This error appears every 15 minutes when the platform cannot reach the database. It does not affect platform operation, but usage data is not recorded during the outage. Investigate the postgres pod health as described above.
+
+### Recover from a deleted PVC
+
+The PersistentVolumeClaim named `platform-db` backs the platform database and should never be deleted. Deleting it permanently destroys any usage snapshots that have not yet been transmitted to the vCluster license server. Those snapshots cannot be reconstructed, which creates a gap in the billing record for that period. Snapshots already transmitted to the license server are not affected. In the case that the PVC is deleted, follow these steps to return the platform database to a working state:
+
+1. Set $PLATFORM_NS, $PLATFORM_NAME, and $CLUSTER_NAME based on the name and namespace of the vCluster Platform Deployment.
+
+```bash
+export PLATFORM_NS=<platform-namespace>
+export PLATFORM_NAME=<platform-deployment-name>
+export CLUSTER_NAME=<cluster-name>
+```
+
+2. Get current vCluster Platform replicas.
+
+```bash
+export DESIRED_REPLICAS=$(kubectl get deployment -n $PLATFORM_NS $PLATFORM_NAME -o json | jq -r '.spec.replicas')
+```
+
+3. Scale down vCluster Platform Deployment to 0.
+
+```bash
+kubectl scale deployment -n $PLATFORM_NS $PLATFORM_NAME --replicas 0
+```
+
+4. Edit the cluster that vCluster Platform is installed in and change the label `loft.sh/platform-db-applied-hash` to a different value, for example add a random letter to the string. Deleting the label will also work.
+
+```bash
+kubectl edit cluster.storage.loft.sh $CLUSTER_NAME
+```
+
+5. Scale up the platform Deployment to its original replica count.
+
+```bash
+kubectl scale deployment -n $PLATFORM_NS $PLATFORM_NAME --replicas $DESIRED_REPLICAS
+```
+
+6. Confirm that vCluster Platform database becomes available.
+
+```bash
+kubectl rollout status deployment/$PLATFORM_NAME -n $PLATFORM_NS
+```

vcluster/production-guide/overview.mdx

@@ -57,7 +57,7 @@ Common operations that apply across all paths.
 | --- | --- |
 | Monitor Platform and tenant workloads | [Monitoring overview](/platform/maintenance/monitoring/overview), [fleet monitoring](/platform/maintenance/monitoring/fleet-monitoring-otel) |
 | Back up and restore tenant clusters | [Snapshots](../manage/backup-restore/backup.mdx), [restore](../manage/backup-restore/restore.mdx), [Velero](../manage/backup-restore/velero.mdx) |
-| Back up and restore Platform | [Backup and restore Platform](/platform/maintenance/backup-restore/backup-restore-platform), [Platform database](/platform/maintenance/platform-database/database) |
+| Back up and restore Platform | [Backup and restore Platform](/platform/maintenance/backup-restore/backup-restore-platform), [Platform database](/platform/install/air-gapped/platform-database) |
 | Upgrade Platform and tenant clusters | [Upgrade vCluster](../manage/upgrade/upgrade-version.mdx), [upgrade Platform](/platform/maintenance/upgrade-migrate/upgrade), [lifecycle policy](/platform/maintenance/upgrade-migrate/lifecycle-policy) |
 | Rotate certificates | [Certificate rotation](../manage/certificates.mdx) |
 | Manage private worker nodes | [Manage private nodes](../deploy/worker-nodes/private-nodes/manage.mdx), [Auto Nodes](/platform/administer/node-providers/overview) |

vcluster_versioned_docs/version-0.34.0/production-guide/overview.mdx

@@ -57,7 +57,7 @@ Common operations that apply across all paths.
 | --- | --- |
 | Monitor Platform and tenant workloads | [Monitoring overview](/platform/maintenance/monitoring/overview), [fleet monitoring](/platform/maintenance/monitoring/fleet-monitoring-otel) |
 | Back up and restore tenant clusters | [Snapshots](../manage/backup-restore/backup.mdx), [restore](../manage/backup-restore/restore.mdx), [Velero](../manage/backup-restore/velero.mdx) |
-| Back up and restore Platform | [Backup and restore Platform](/platform/maintenance/backup-restore/backup-restore-platform), [Platform database](/platform/maintenance/platform-database/database) |
+| Back up and restore Platform | [Backup and restore Platform](/platform/maintenance/backup-restore/backup-restore-platform), [Platform database](/platform/install/air-gapped/platform-database) |
 | Upgrade Platform and tenant clusters | [Upgrade vCluster](../manage/upgrade/upgrade-version.mdx), [upgrade Platform](/platform/maintenance/upgrade-migrate/upgrade), [lifecycle policy](/platform/maintenance/upgrade-migrate/lifecycle-policy) |
 | Rotate certificates | [Certificate rotation](../manage/certificates.mdx) |
 | Manage private worker nodes | [Manage private nodes](../deploy/worker-nodes/private-nodes/manage.mdx), [Auto Nodes](/platform/administer/node-providers/overview) |

vcluster_versioned_docs/version-0.35.0/production-guide/overview.mdx

@@ -57,7 +57,7 @@ Common operations that apply across all paths.
 | --- | --- |
 | Monitor Platform and tenant workloads | [Monitoring overview](/platform/maintenance/monitoring/overview), [fleet monitoring](/platform/maintenance/monitoring/fleet-monitoring-otel) |
 | Back up and restore tenant clusters | [Snapshots](../manage/backup-restore/backup.mdx), [restore](../manage/backup-restore/restore.mdx), [Velero](../manage/backup-restore/velero.mdx) |
-| Back up and restore Platform | [Backup and restore Platform](/platform/maintenance/backup-restore/backup-restore-platform), [Platform database](/platform/maintenance/platform-database/database) |
+| Back up and restore Platform | [Backup and restore Platform](/platform/maintenance/backup-restore/backup-restore-platform), [Platform database](/platform/install/air-gapped/platform-database) |
 | Upgrade Platform and tenant clusters | [Upgrade vCluster](../manage/upgrade/upgrade-version.mdx), [upgrade Platform](/platform/maintenance/upgrade-migrate/upgrade), [lifecycle policy](/platform/maintenance/upgrade-migrate/lifecycle-policy) |
 | Rotate certificates | [Certificate rotation](../manage/certificates.mdx) |
 | Manage private worker nodes | [Manage private nodes](../deploy/worker-nodes/private-nodes/manage.mdx), [Auto Nodes](/platform/administer/node-providers/overview) |