[v4.9.0] docs(doc-1415): add maintenance/logging section (#2133)

platform_versioned_docs/version-4.9.0/configure/platform-configs/audit.mdx

@@ -27,13 +27,16 @@ Auditing in the platform is similar to [auditing Kubernetes clusters](https://ku
 
 ## Enable auditing
 
-:::warning Two audit fields
-There is a field named as `audit` in `values.yaml` of `vcluster-platform` chart and a same field under `config` section, ie, `config.audit`. The former
-is to configure the audit log settings for the platform pod container. Enable this to record logs. The `config.audit` enables Audit Log UI feature in vCluster Platform UI.
-Thus, make sure the audit logs is being recorded by setting `audit: true` in `values.yaml` first.
+:::info Two separate audit fields
+`values.yaml` has two distinct `audit` fields with different purposes:
+
+- **`audit`** (top-level Helm value) — configures audit log recording and persistence for the platform pod. This is a Helm-only setting. Changes require a Helm upgrade and a platform restart.
+- **`config.audit`** (under the `config` section) — controls audit policy and enables the Audit Log UI. This can be set in **Admin > Config** in the UI or under `config` in `values.yaml`.
+
+`audit.enabled: true` must be set at the top level first. Without it, no audit events are recorded regardless of `config.audit` settings.
 :::
 
-Configure auditing through the platform config in the platform UI (`Admin -> Config`). The content added in the UI will automatically be under the `config` section of the `values.yaml`.
+Configure auditing through the platform config in the platform UI (**Admin > Config**). Content added in the UI goes automatically under the `config` section of `values.yaml`.
 
 An example configuration could look like:
 
@@ -43,7 +46,9 @@ audit:
   level: 1
 ```
 
-Changing the platform auditing configuration requires a restart to take effect. Restart the platform either through the platform UI or via kubectl: `kubectl rollout restart deploy/loft -n loft`.
+:::warning
+Changing `config.audit` settings requires a platform restart to take effect. Restart through **Admin > Config** in the UI or with `kubectl rollout restart deploy/loft -n loft`.
+:::
 
 Each request on each stage of its execution generates an audit event, which is then pre-processed according to a certain policy and written to a backend (currently only log backends are supported). The policy determines what's recorded and the backends persist the records.
 
@@ -136,18 +141,18 @@ audit:
     # size: 30Gi
 ```
 
-:::warning
-The `audit` here is the `audit` in `values.yaml`, not the `audit` under `config` section of `values.yaml`. It controls whether the 
-audit log is recorded and saved.
+:::note
+This is the top-level `audit` field in `values.yaml`, not `config.audit`. See [Enable auditing](#enable-auditing) for the distinction.
 :::
 
 Apply the values using Helm:
 
 ```bash title="Apply Helm values"
-Helm upgrade vcluster-platform vcluster-platform -n vcluster-platform --version 4.0.0-alpha.12 \
---repo https://charts.loft.sh \
---reuse-values \
--f values.yaml
+helm upgrade vcluster-platform vcluster-platform \
+  --namespace vcluster-platform \
+  --repo https://charts.loft.sh \
+  --reuse-values \
+  -f values.yaml
 ```
 
 ### Use a persistent database as platform audit backend
@@ -191,21 +196,18 @@ audit:
   enableSideCar: true
 ```
 
-:::warning
-The `audit` here is the `audit` in `values.yaml`, not the `audit` under `config` section of `values.yaml`. It controls whether the 
-audit log is recorded and saved. 
-
-Thus, you cannot configure this under `Admin > Config`, since this requires a change in the platform deployment itself, which is why this is a Helm option only
+:::note
+This is the top-level `audit` field in `values.yaml`, not `config.audit`. It cannot be set in **Admin > Config**. See [Enable auditing](#enable-auditing) for the distinction.
 :::
 
 Update the Helm release:
 
-```bash title="Update helm release"
-helm upgrade vcluster-platform vcluster-platform --namespace vcluster-platform \
-                                         --repo https://charts.loft.sh \
-                                         --version 4.0.0-alpha.12 \
-                                         --reuse-values \
-                                         --values values.yaml
+```bash title="Update Helm release"
+helm upgrade vcluster-platform vcluster-platform \
+  --namespace vcluster-platform \
+  --repo https://charts.loft.sh \
+  --reuse-values \
+  --values values.yaml
 ```
 
 Wait until the platform has restarted, then view the audit logs:

platform_versioned_docs/version-4.9.0/maintenance/logging/_category_.json

@@ -0,0 +1,6 @@
+{
+  "label": "Logging",
+  "position": 3.5,
+  "collapsible": true,
+  "collapsed": true
+}

platform_versioned_docs/version-4.9.0/maintenance/logging/overview.mdx

@@ -0,0 +1,94 @@
+---
+title: Logging
+sidebar_label: Overview
+sidebar_position: 1
+description: Understand platform audit logs, platform process logs, and workload log collection in vCluster Platform.
+---
+
+import Icon from '@site/src/components/Icon';
+
+vCluster Platform surfaces three distinct types of logs, each serving a different purpose and requiring different configuration.
+
+| Type | What it captures | Configure |
+|------|-----------------|-----------|
+| [Platform audit logs](#platform-audit-logs) | API requests that pass through the platform gateway, including who acted and when | [Audit Logging](../../configure/platform-configs/audit.mdx) |
+| [Platform process logs](#platform-process-logs) | Operational output from the platform pod, including startup, controller activity, and errors | [Platform Process Logging](./platform-logging.mdx) |
+| [Workload logs](#workload-logs) | Output from applications running inside tenant clusters | [Central HostPath Mapper](../monitoring/central-hostpath-mapper.mdx) |
+
+These three types are independent. Enabling audit logging does not affect process log verbosity. A separate infrastructure component handles workload log collection.
+
+## Platform audit logs
+
+The platform's API gateway records every request that passes through it. This covers requests from the UI, the CLI, and any API call routed through the platform proxy. Each log entry captures:
+
+- The requesting user (or service account), their groups, and source IP
+- The resource being accessed (type, name, namespace, cluster)
+- The HTTP verb (`get`, `list`, `create`, `update`, `delete`, `patch`)
+- Whether impersonation was in use
+- Timestamps and a unique audit ID
+
+### What gets logged
+
+Four audit levels control the volume and detail of what the platform captures.
+
+| Level | Writes (create/update/delete/patch) | Reads (get/list/watch) | Request body | Response body |
+|-------|-------------------------------------|------------------------|:------------:|:-------------:|
+| 1 | Metadata only | Not logged | <Icon type="cross" /> | <Icon type="cross" /> |
+| 2 | Metadata only | Metadata only | <Icon type="cross" /> | <Icon type="cross" /> |
+| 3 | Logged with request body | Logged with request body | <Icon type="check" /> | <Icon type="cross" /> |
+| 4 | Logged with request and response | Logged with request and response | <Icon type="check" /> | <Icon type="check" /> |
+
+At all levels, the following are always excluded from audit logs:
+
+- Internal access-review requests (`selfsubjectaccessreviews`, `subjectaccessreviews`)
+- Agent coordination resources (`agentauditevents`, `directclusterendpointtokens`, `ingressauthtokens`)
+- Platform token resources (`licensetokens`)
+
+The platform records long-running streaming operations (`pods/log`, `pods/exec`, `pods/portforward`) at metadata level only, regardless of the configured level. This prevents unbounded log growth.
+
+The platform limits sensitive resources (`sharedsecrets`, `ownedaccesskeys`, `resetaccesskeys`, `users/profile`) to metadata level. This prevents credential material from appearing in logs.
+
+### Configure audit logging
+
+See [Audit Logging](../../configure/platform-configs/audit.mdx) for how to enable audit logging, choose a level, define a custom policy, and persist logs to a PVC or external database.
+
+## Platform process logs
+
+The platform pod writes operational logs to stdout. These cover initialization, controller reconciliation, API request handling, and runtime errors.
+
+The platform supports three log levels.
+
+| Level | What it captures | When to use |
+|-------|-----------------|-------------|
+| `info` | Startup sequences, controller events, handled requests | Default for all environments |
+| `debug` | Full operation context for every request and reconciliation loop | Temporarily, when diagnosing an issue |
+| `error` | Failures and warnings only | Stable production environments with low noise requirements |
+
+By default, logs use a human-readable console format. Switch to JSON format when feeding logs into an aggregation pipeline such as Elasticsearch or Grafana Loki.
+
+Access platform logs with:
+
+```bash
+kubectl logs -n vcluster-platform -l app=loft -f
+```
+
+See [Platform Process Logging](./platform-logging.mdx) for how to change log level and output format.
+
+## Workload logs
+
+Standard log collectors such as Fluentd, the ELK stack, and Grafana Loki read container logs from the host filesystem at paths like:
+
+```
+/var/log/pods/<namespace>_<pod-name>_<uid>/<container>/
+```
+
+Tenant clusters introduce a naming problem. When vCluster syncs pods to the control plane cluster, it rewrites their names. A pod named `web-6d4f9b` inside a tenant cluster appears as `web-6d4f9b-x-default-x-my-vcluster` on the control plane cluster. Log collectors see only the physical name and cannot correlate entries back to the originating tenant cluster workload.
+
+The HostPath Mapper solves this by creating symlinks on each node that map physical pod paths to their virtual names. Log collectors follow the symlinks and see the names they expect.
+
+vCluster Platform offers two deployment options:
+
+- **Central HostPath Mapper**: A single DaemonSet on the control plane cluster handles path remapping for all tenant clusters. Recommended for fleet deployments.
+- **Per-cluster HostPath Mapper**: A DaemonSet installed in each tenant cluster's namespace. Better suited to small environments or where per-cluster isolation is needed.
+
+See [Central HostPath Mapper](../monitoring/central-hostpath-mapper.mdx) for setup instructions for both options.

platform_versioned_docs/version-4.9.0/maintenance/logging/platform-logging.mdx

@@ -0,0 +1,101 @@
+---
+title: Platform Process Logging
+sidebar_label: Platform Process Logging
+sidebar_position: 2
+description: Configure the log level and output format for the vCluster Platform pod.
+---
+
+import Tabs from "@theme/Tabs";
+import TabItem from "@theme/TabItem";
+
+vCluster Platform writes operational logs to `stdout` from the platform pod. These logs cover initialization, controller reconciliation, API request handling, and runtime errors. Check these logs first when diagnosing unexpected platform behavior.
+
+By default, logs use a human-readable console format at `info` level.
+
+## Log levels
+
+Platform logging supports three log levels:
+- `info`
+- `debug`
+- `error`
+
+For a description of each level, see [Platform process logs](./overview.mdx#platform-process-logs).
+
+:::warning
+Debug logging significantly increases log volume. Enable it only while reproducing an issue, then return to `info`.
+:::
+
+To change the log level, upgrade the Helm release with the `logging.level` value:
+
+```bash
+helm upgrade vcluster-platform vcluster-platform \
+  --repo https://charts.loft.sh \
+  --namespace vcluster-platform \
+  --reuse-values \
+  --set logging.level=debug
+```
+
+## Log format
+
+vCluster Platform supports two output formats.
+
+<Tabs
+  defaultValue="console"
+  values={[
+    { label: "Console", value: "console" },
+    { label: "JSON", value: "json" },
+  ]}
+>
+<TabItem value="console">
+
+The default console format produces human-readable output suited to manual log review:
+
+```text
+2023-07-11 09:20:56     INFO    controller-runtime.metrics      metrics/listener.go:44  Metrics server is starting to listen    {"component": "loft", "addr": "127.0.0.1:12000"}
+2023-07-11 09:20:56     INFO    initialize/context.go:68        Initialize...   {"component": "loft"}
+2023-07-11 09:20:56     INFO    initialize/context.go:72        Ensure certificates...  {"component": "loft"}
+```
+
+</TabItem>
+<TabItem value="json">
+
+The JSON format produces structured output suited to log aggregation systems such as Elasticsearch, Splunk, or Grafana Loki:
+
+```json
+{"level":"info","ts":1689067388.209614,"logger":"controller-runtime.metrics","caller":"metrics/listener.go:44","msg":"Metrics server is starting to listen","component":"loft","addr":"127.0.0.1:12000"}
+{"level":"info","ts":1689067388.2099726,"caller":"initialize/context.go:68","msg":"Initialize...","component":"loft"}
+{"level":"info","ts":1689067388.2100549,"caller":"initialize/context.go:72","msg":"Ensure certificates...","component":"loft"}
+```
+
+Each entry contains at minimum:
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `level` | string | Log level (`info`, `warn`, `error`, `debug`) |
+| `ts` | number | Unix timestamp (seconds since epoch) |
+| `caller` | string | Source file and line number |
+| `msg` | string | Log message |
+| `component` | string | Platform component that emitted the log |
+
+Additional fields vary by subsystem and operation type. For example, controller events may include `namespace` and `name` while API handler logs may include `method` and `status`.
+
+To switch to JSON format:
+
+```bash
+helm upgrade vcluster-platform vcluster-platform \
+  --repo https://charts.loft.sh \
+  --namespace vcluster-platform \
+  --reuse-values \
+  --set logging.encoding=json
+```
+
+</TabItem>
+</Tabs>
+
+## View logs
+
+```bash
+kubectl logs -n vcluster-platform -l app=loft -f
+```
+
+In high-availability deployments, multiple platform pods run concurrently. Pipe through a log aggregator, or target a specific pod, to avoid interleaved output from multiple replicas.

platform_versioned_docs/version-4.9.0/maintenance/monitoring/metrics.mdx

@@ -1,62 +1,13 @@
 ---
-title: Metrics And Monitoring
-sidebar_label: Metrics And Monitoring
+title: Metrics
+sidebar_label: Metrics
 sidebar_position: 4
+description: Prometheus-conformant metrics exposed by vCluster Platform, including how to configure scraping with a ServiceMonitor.
 ---
 
-## Configure logging
-
-By default, vCluster Platform prints out structured logs to the console with a log level of `info`.
-
-These logs include constant log messages, a human-readable timestamp, log levels, the file and line number of the source caller, and with variables in the body:
-
-```text
-2023-07-11 09:20:56     INFO    controller-runtime.metrics      metrics/listener.go:44  Metrics server is starting to listen    {"component": "loft", "addr": "127.0.0.1:12000"}
-2023-07-11 09:20:56     INFO    initialize/context.go:68        Initialize...   {"component": "loft"}
-2023-07-11 09:20:56     INFO    initialize/context.go:72        Ensure certificates...  {"component": "loft"}
-2023-07-11 09:21:02     INFO    initialize/context.go:80        Ensure crds...  {"component": "loft"}
-```
-
-You can change the log level by setting the Helm value `logging.level` to one of the following values: `debug`, `info`, `error`, and upgrading your vCluster Platform release with the following command:
-
-```bash
-helm upgrade loft vcluster-platform --repo https://charts.loft.sh/ \
-  --namespace vcluster-platform \
-  --reuse-values \
-  --set logging.level=error # or debug or info
-```
-
-### Use JSON output encoding
-
-In addition to the console output, vCluster Platform also supports printing logs encoded in JSON format.
-
-This output format is helpful for structured logging, which allows you to search and filter logs easily.
-
-You can change the log output by setting the Helm value `logging.encoding` to one of the following values: `json`, and upgrading your vCluster Platform release with the following command:
-
-```bash
-helm upgrade loft vcluster-platform --repo https://charts.loft.sh/ \
-  --namespace vcluster-platform \
-  --reuse-values \
-  --set logging.encoding=json
-```
-
-These JSON-encoded logs include constant log messages with different variables in the body, like the default console output:
-
-```json
-{"level":"info","ts":1689067388.209614,"logger":"controller-runtime.metrics","caller":"metrics/listener.go:44","msg":"Metrics server is starting to listen","component":"loft","addr":"127.0.0.1:12000"}
-{"level":"info","ts":1689067388.2099726,"caller":"initialize/context.go:68","msg":"Initialize...","component":"loft"}
-{"level":"info","ts":1689067388.2100549,"caller":"initialize/context.go:72","msg":"Ensure certificates...","component":"loft"}
-{"level":"info","ts":1689067389.678155,"caller":"initialize/context.go:80","msg":"Ensure crds...","component":"loft"}
-```
-
-The JSON encoded logs contain at least the following fields:
-
-- `level`: The log level of the message
-- `ts`: The timestamp of the message (floating-point number of seconds since the Unix epoch)
-- `caller`: The file and line number of the message
-- `msg`: The message itself
-- `component`: The vCluster Platform component that emitted the message
+:::note Platform process logging
+To configure log level or output format for the platform pod, see [Platform Process Logging](../logging/platform-logging.mdx).
+:::
 
 ## Metrics
 

platform_versioned_docs/version-4.9.0/maintenance/monitoring/overview.mdx

@@ -44,10 +44,12 @@ For aggregating workload metrics across multiple tenant clusters, use the OpenTe
 
 vCluster Platform exposes [Prometheus-conformant metrics](./metrics.mdx) from its internal components, including the API gateway, integrated Kubernetes API server, controller manager, and Go runtime. These metrics cover request counts, latency, and error rates for all platform operations.
 
-Use a [Prometheus `ServiceMonitor`](./metrics.mdx#create-the-prometheus-servicemonitor) to scrape these metrics automatically, or [access the `/metrics` endpoint directly](./metrics.mdx#access-the-metrics-endpoint-directly-without-a-servicemonitor). You can also [configure Platform log levels and JSON encoding](./metrics.mdx#configure-logging) from the same page.
+Use a [Prometheus `ServiceMonitor`](./metrics.mdx#create-the-prometheus-servicemonitor) to scrape these metrics automatically, or [access the `/metrics` endpoint directly](./metrics.mdx#access-the-metrics-endpoint-directly-without-a-servicemonitor). For platform pod log level and output format, see [Platform Process Logging](../logging/platform-logging.mdx).
 
 ## Log collection
 
 To collect logs from workloads running inside tenant clusters, use the Central HostPath Mapper. It installs a single DaemonSet on the Control Plane Cluster. The DaemonSet handles log path remapping for all tenant clusters and removes the need for per-cluster logging agents.
 
 - [Central HostPath Mapper](./central-hostpath-mapper)
+
+For platform audit logging and platform process log configuration, see [Logging](../logging/overview.mdx).