Skip to content

docs(observability): document GPU monitoring templates and NVSentinel#2472

Open
flomedja wants to merge 8 commits into
loft-sh:mainfrom
flomedja:ENGPLAT-834-docs-gpu-template-and-nv-apps-document-by-design-behaviors-teardown
Open

docs(observability): document GPU monitoring templates and NVSentinel#2472
flomedja wants to merge 8 commits into
loft-sh:mainfrom
flomedja:ENGPLAT-834-docs-gpu-template-and-nv-apps-document-by-design-behaviors-teardown

Conversation

@flomedja

@flomedja flomedja commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Content Description

Preview Link

Note: Every page in the observability nav section has some updates. The above pages are the main ones.

Internal Reference

Closes ENGPLAT-834

AI review: mention @claude in a comment to request a review or changes. See CONTRIBUTING.md for available commands.

FORK LIMITATION: @claude does not work on pull requests opened from forks. GitHub Actions cannot access the required secrets for fork-originated PRs. To use AI review, push your branch directly to this repository.

@netlify /docs

@flomedja
flomedja requested a review from a team as a code owner July 20, 2026 19:37
@netlify

netlify Bot commented Jul 20, 2026

Copy link
Copy Markdown

Deploy Preview for vcluster-docs-site ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit c9bce77
🔍 Latest deploy log https://app.netlify.com/projects/vcluster-docs-site/deploys/6a5eee7eee854b00089a0a37
😎 Deploy Preview https://deploy-preview-2472--vcluster-docs-site.netlify.app/docs
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

  Add two pages to the fleet observability section for ENGPLAT-834:

  - GPU monitoring templates: summary of the bundled Argo CD templates
    (cert-manager, GPU Operator, Fleet Intelligence Agent, NVSentinel),
    deployment order, the two DCGM metric families with the double-count
    warning, per-pod GPU attribution, and stack teardown
  - NVSentinel GPU monitoring: deploy guide with UI and YAML flows, what
    dry-run does and does not do, full parameter reference including
    Janitor providers, and cleanup of the retained MongoDB PVCs

  Link both pages from the section overview and the Argo CD templates
  page, and shift later pages down in the sidebar.
@flomedja
flomedja force-pushed the ENGPLAT-834-docs-gpu-template-and-nv-apps-document-by-design-behaviors-teardown branch from af7219c to 08d5373 Compare July 20, 2026 19:37
@github-actions

Copy link
Copy Markdown
Contributor

djwfyi added 2 commits July 20, 2026 16:46
…pendency

Add GPU/DCGM/NVSentinel to the Google.Headings vocab and scope off
Google.WordList for the NVSentinel page, where "terminate" and "service
account name" are accurate cloud/API terminology rather than false
positives. Also clarify that enableNodeDrainer's Fault Quarantine
dependency, unlike the MongoDB store and Janitor dependencies, is not
auto-resolved by the ArgoCD template, per upstream NVSentinel docs and
the loft-enterprise template source.
Switch do not/does not/cannot to don't/doesn't/can't across both new
GPU/NVSentinel pages, keeping the "Does/Doesn't" dry-run bullet list
parallel. Fixes the internal anchor link broken by the "does not do" ->
"doesn't do" heading rename.
Comment thread platform/maintenance/observability/gpu-monitoring-templates.mdx Outdated

@seanschneeweiss seanschneeweiss left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to add a small section that the Cluster Collector is preconfigured to scrape metrics from gpu-operator, nvsentinel (only if destionationNamespace kept at default value).

Comment thread platform/maintenance/observability/gpu-monitoring-templates.mdx Outdated
Comment thread platform/maintenance/observability/deploy-with-argocd.mdx Outdated
djwfyi and others added 3 commits July 20, 2026 21:39
Mention cert-manager in the GPU stack overview so the bundled-templates
table doesn't introduce a fourth item the intro never named. In the
NVSentinel prerequisites, defer the cert-manager conditional-requirement
detail to a new section placed after the component table, so the
prerequisites checklist stops naming components (Fault Quarantine, Node
Drainer, Janitor, etc.) before they're defined. Also cross-link to the
deployment order guidance so readers landing directly on the NVSentinel
page know to deploy GPU Operator first.
…wording

Reorder the sidebar so Argo CD templates precedes Install gateway,
matching the explicit "Get started" step order on the overview page
(deploy backend/Grafana, then create the connector) instead of
contradicting it.

Replace "control plane cluster" with "local cluster" everywhere the
metrics backend, Grafana, and gateway are described, per engineering
manager feedback questioning whether that wording was deliberately
vague. Confirmed against loft-enterprise source that these components
are hardcoded to Platform's own local cluster (loft-cluster), not an
arbitrary registered control plane cluster, so the prior "replace with
your control plane cluster name" framing was inaccurate.

Also clarify "Deploy collectors to each cluster": the opening sentence
implied tenant-cluster-only, while a control-plane-cluster option was
only revealed at the end of the section. State both cluster types up
front instead.
deploy-with-argocd.mdx bundled two non-adjacent workflow steps (backend/Grafana
before the gateway, collectors after it), so the sidebar order didn't match the
order a reader actually needed to follow it. Dissolve that page and move each
Argo CD template's instructions onto the step's own page, alongside the manual
path. Add a quickstart page with a single linear walkthrough of the default
path for readers who want to deploy end to end without cross-referencing every
option on the reference pages.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
djwfyi and others added 2 commits July 20, 2026 23:53
…template-and-nv-apps-document-by-design-behaviors-teardown

# Conflicts:
#	platform/maintenance/observability/deploy-with-argocd.mdx
…ollectors

Add --context to the GPU stack teardown commands so they can't run against
the wrong cluster's cert-manager or namespaces, per Sean's review comment.
Document that the cluster-collector template already scrapes the GPU stack
(Fleet Intelligence Agent, dcgm-exporter, GPU Health Monitor) as long as
those components stay at their default namespaces, verified against the
template source in loft-enterprise.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@djwfyi

djwfyi commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Addressed the review feedback:

  • Added --context to every GPU stack teardown command, so they can't accidentally target the wrong cluster's cert-manager or namespaces.
  • Added a new section, The cluster-collector template scrapes GPU metrics automatically, covering Sean's review comment about the Cluster Collector being preconfigured to scrape the Fleet Intelligence Agent, dcgm-exporter, and NVSentinel's GPU Health Monitor when those stay at their default namespaces. Verified against the cluster-collector template source in loft-enterprise.
  • Replied to and resolved the three open review threads (metric-family emphasis, teardown context, and the GPU-templates-table suggestion, which no longer applies since deploy-with-argocd.mdx was split up in a later restructuring pass).

All commits pushed to this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants