Skip to content

Commit 81d02d7

Browse files
docs(explanations): simplify NKP architecture page
1 parent 1960d7a commit 81d02d7

7 files changed

Lines changed: 292 additions & 248 deletions

File tree

docs/docs/explanations/index.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
11
# Explanations
22

3-
Coming soon.
3+
Conceptual guides that build a mental model of how Nebari works. Read these
4+
when you want to understand the *why* behind a design choice, not just the
5+
*how* of a workflow.
6+
7+
- [NKP architecture](/docs/explanations/nkp-architecture): the layered stack
8+
that makes up the Nebari Kubernetes Platform, what the `nic` CLI does and
9+
doesn't do, and how applications reach the cluster via GitOps.

docs/docs/explanations/nkp-architecture.mdx

Lines changed: 47 additions & 116 deletions
Original file line numberDiff line numberDiff line change
@@ -6,17 +6,11 @@ description: A conceptual tour of how Nebari Kubernetes Platform is layered, wha
66

77
# NKP architecture
88

9-
Nebari Kubernetes Platform (NKP) is the first **Deploy Target** in the Nebari
10-
ecosystem: the infrastructure that hosts a Nebari platform and runs the
11-
[Software Packs](/docs/introduction#software-packs) installed on it. This page
12-
explains how NKP is put together and why, so you can reason about deployments,
13-
upgrades, and extension points without reading the source.
9+
Nebari Kubernetes Platform (NKP) is the first **Deploy Target** in the Nebari ecosystem: the infrastructure that hosts a Nebari platform and runs the [Software Packs](/docs/introduction#software-packs) installed on it. This page explains how NKP is put together and why, so you can reason about deployments, upgrades, and extension points without reading the source.
1410

1511
## The layers, top to bottom
1612

17-
NKP is a stack of layers, each doing one job. Because the layers connect
18-
through stable, well-defined interfaces, you can change one layer without
19-
rewriting the others.
13+
NKP is a stack of layers, each doing one job. Because the layers connect through stable, well-defined interfaces, you can change one layer without rewriting the others.
2014

2115
{/* Diagram source: docs/static/img/explanations/_sources/nkp-architecture.html
2216
(open in a browser to view, or import the JSX into Paper / any React renderer to edit) */}
@@ -25,95 +19,56 @@ rewriting the others.
2519

2620
**What users and developers see and control**
2721

28-
- **Dynamic landing page**: the home page where users browse and open
29-
installed Capabilities.
30-
- **Software Pack**: an installable Capability (chat assistant, document
31-
analyzer, code review tool, and so on) built by a developer.
22+
- **Dynamic landing page**: the home page where users browse and open installed Capabilities.
23+
- **Software Pack**: an installable Capability (chat assistant, document analyzer, code review tool, and so on) built by a developer.
3224

3325
**Platform-managed, hidden from users**
3426

35-
- **Nebari Operator**: the automation that deploys each Software Pack and
36-
connects it to the Foundational software.
37-
- **Foundational software**: shared services (secure connections, login,
38-
traffic routing, monitoring, continuous delivery from Git) that power the
39-
operator and every running pack.
27+
- **Nebari Operator**: the automation that deploys each Software Pack and connects it to the Foundational software.
28+
- **Foundational software**: shared services (secure connections, login, traffic routing, monitoring, continuous delivery from Git) that power the operator and every running pack.
4029

4130
**Where it all runs**
4231

43-
- **Managed Kubernetes cluster**: hosts every container above. EKS, GKE, AKS,
44-
or K3s for local development.
45-
- **Cloud or bare-metal provider**: the physical infrastructure underneath.
46-
AWS, GCP, Azure, dedicated servers (Hetzner, Equinix Metal), or your own
47-
machine for development.
48-
49-
### What's in the foundational layer
50-
51-
- **cert-manager**: keeps secure connections working. Automatically requests,
52-
renews, and rotates HTTPS certificates for every domain in the cluster.
53-
- **Envoy Gateway**: routing on the Kubernetes Gateway API. Header- and
54-
weight-based routing, rate limiting, JWT validation.
55-
- **Keycloak**: OIDC provider with user federation, MFA, and SSO across
56-
ArgoCD, Grafana, and Software Packs.
57-
- **ArgoCD**: GitOps reconciliation (more on this below).
58-
- **LGTM telemetry stack**: OpenTelemetry Collector, Loki for logs, Grafana
59-
for visualization, Tempo for traces, Mimir for metrics. Backed by cloud
60-
object storage for long-term retention.
61-
62-
### How the Operator reconciles a pack
63-
64-
On every reconcile of a `NebariApplication` (`nebari.dev/v1alpha1`), the
65-
controller-runtime operator walks four steps in order:
66-
67-
1. **Routing**: a cert-manager `Certificate` (if TLS is enabled) and an Envoy
68-
`HTTPRoute` for the application's domain and paths.
69-
2. **Authentication**: a Keycloak OAuth2 client scoped to the application's
70-
allowed groups, users, and public paths; the client secret lands in a
71-
Kubernetes `Secret`.
72-
3. **Observability**: a `ServiceMonitor` for metrics and any Grafana
73-
dashboards declared by the pack (sourced from a URL or a ConfigMap).
74-
4. **Status**: phase moves through `Pending``Provisioning``Ready`, and
75-
the public URL plus per-step conditions are written back to the CRD.
32+
- **Managed Kubernetes cluster**: hosts every container above. EKS, GKE, AKS, or K3s for local development.
33+
- **Cloud or bare-metal provider**: the physical infrastructure underneath. AWS, GCP, Azure, dedicated servers (Hetzner, Equinix Metal), or your own machine for development.
34+
35+
<details>
36+
<summary>**What's in the foundational layer**</summary>
37+
38+
- **cert-manager**: keeps secure connections working. Automatically requests, renews, and rotates HTTPS certificates for every domain in the cluster.
39+
- **Envoy Gateway**: handles traffic routing. Inspects incoming requests and forwards each one to the right service, with built-in rate limiting and authentication checks.
40+
- **Keycloak**: handles login. One sign-on covers every app on the platform (ArgoCD, Grafana, Software Packs), with support for multi-factor authentication and connection to an existing identity provider like Active Directory.
41+
- **ArgoCD**: keeps the cluster in sync with a Git repository. Whenever the platform team commits a change to Git (a new pack, an updated config), ArgoCD applies it to the cluster automatically.
42+
- **LGTM telemetry stack**: collects logs, metrics, and traces from every running pack and presents them in shared dashboards.
43+
44+
</details>
7645

7746
## The `nic` CLI
7847

79-
NKP is provisioned by **NIC** (Nebari Infrastructure Core) and its `nic` CLI.
80-
Understanding the boundary of what `nic` owns is the single most important
81-
mental model on this page.
48+
The `nic` CLI (short for **Nebari Infrastructure Core**) is the command-line tool for managing NKP: installing, updating, tearing down, plus inspecting the cloud resources underneath.
8249

8350
### What `nic` does
8451

85-
- **Provisions the cloud substrate.** `nic` invokes OpenTofu (Terraform)
86-
modules to create the VPC, the managed Kubernetes cluster, node pools, IAM,
87-
and durable storage on the chosen provider.
88-
- **Bootstraps the cluster.** It applies namespaces, RBAC, storage classes, and
89-
network policies needed before higher layers can run.
90-
- **Installs ArgoCD.** This is the only Helm chart `nic` installs directly.
91-
Everything above ArgoCD is reconciled by ArgoCD itself, not by `nic`.
92-
93-
That is the boundary. After `nic deploy` finishes, the cluster has Kubernetes,
94-
the bootstrap resources, and ArgoCD pointed at a Git repository.
52+
- **Creates the cloud infrastructure.** This includes the network, the managed Kubernetes cluster and its worker machines, the identity and access controls, and the persistent storage. Under the hood, `nic` uses Terraform to do this.
53+
- **Prepares the cluster.** Sets up the basic Kubernetes structures the platform relies on: organizational groupings (namespaces), permission rules (RBAC), storage templates (storage classes), and network rules (network policies).
54+
- **Installs ArgoCD.** ArgoCD is the GitOps engine that delivers everything above the cluster (the foundational software, the operator, and the packs). `nic` installs it directly so the rest of the platform can flow in from a Git repository.
9555

9656
### What `nic` does not do
9757

98-
- **`nic` is not an application manager.** It does not install or upgrade
99-
Software Packs, the Nebari Operator, or any of the foundational software
100-
beyond ArgoCD. Those are GitOps concerns (see below).
101-
- **`nic` does not push to the cluster imperatively.** Once ArgoCD is running,
102-
changes flow from Git into the cluster, not from a developer's laptop.
58+
- **`nic` is not an application manager.** It does not install or upgrade Software Packs, the Nebari Operator, or any of the foundational software beyond ArgoCD. Those are all delivered from Git through ArgoCD.
59+
- **`nic` does not change the cluster directly after install.** Anything that needs to happen inside a running cluster (a new pack, an updated config) goes through Git first, and ArgoCD applies it.
60+
61+
This split keeps cluster work (the bottom of the stack) and pack work (the top) from blocking each other. Updating the cluster's underlying machines doesn't affect the running packs, and rolling out a new pack version doesn't require a `nic` deploy.
10362

104-
This separation lets infrastructure and applications evolve on independent
105-
lifecycles: re-rolling a node pool does not touch application state, and
106-
shipping a new pack version does not require a `nic` run.
63+
{/* Diagram source: docs/static/img/explanations/_sources/nkp-architecture-split.html
64+
(open in a browser to view, or import the HTML into Paper via the write_html MCP tool) */}
65+
![NKP architecture annotated to show the split between pack work (Landing page and Software Pack, shipped via Git) and cluster work (Nebari Operator and Foundational software, provisioned by the nic CLI). The two halves can change independently.](/img/explanations/nkp-architecture-split.png)
10766

10867
## GitOps via ArgoCD
10968

110-
Above the cluster, NKP is **declarative and pull-based**. ArgoCD watches a
111-
Git repository (`nebari-foundational-software`) and continuously reconciles
112-
the live cluster state to match it.
69+
Above the cluster, NKP is **declarative and pull-based**. ArgoCD watches a Git repository (`nebari-foundational-software`) and continuously reconciles the live cluster state to match it.
11370

114-
ArgoCD uses the **app-of-apps** pattern: a top-level Application resource
115-
points to a folder of child Applications, each one a piece of foundational
116-
software with explicit dependencies.
71+
ArgoCD uses the **app-of-apps** pattern: a top-level Application resource points to a folder of child Applications, each one a piece of foundational software with explicit dependencies.
11772

11873
```mermaid
11974
flowchart LR
@@ -127,23 +82,15 @@ flowchart LR
12782

12883
Three properties follow from this:
12984

130-
- **Single source of truth.** What is in Git is what is in the cluster. There
131-
is no "live but undocumented" configuration drift on the application side.
132-
- **Self-healing.** If an operator deletes a resource by hand, ArgoCD puts it
133-
back on the next reconciliation tick.
134-
- **Auditable change.** Every platform change is a Git commit; rollbacks are a
135-
Git revert.
85+
- **Single source of truth.** What is in Git is what is in the cluster. There is no "live but undocumented" configuration drift on the application side.
86+
- **Self-healing.** If an operator deletes a resource by hand, ArgoCD puts it back on the next reconciliation tick.
87+
- **Auditable change.** Every platform change is a Git commit; rollbacks are a Git revert.
13688

137-
Software Packs plug into this model from above. A pack ships its workload
138-
manifests plus a `NebariApplication` custom resource that declares what the
139-
pack needs (domain, paths, auth scope, dashboards). Once that custom resource
140-
lands in the cluster, the Nebari Operator picks it up and runs the four
141-
reconciliation steps described in the layers section.
89+
Software Packs plug into this model from above. A pack ships its workload manifests plus a `NebariApplication` custom resource that declares what the pack needs (domain, paths, auth scope, dashboards). Once that custom resource lands in the cluster, the Nebari Operator picks it up and runs the four reconciliation steps described in the layers section.
14290

14391
## Terraform state and drift detection
14492

145-
`nic` uses a standard **Terraform state file** to track everything it
146-
provisioned. State is persisted in a remote backend matched to the provider:
93+
`nic` uses a standard **Terraform state file** to track everything it provisioned. State is persisted in a remote backend matched to the provider:
14794

14895
| Provider | Backend | Locking |
14996
| --- | --- | --- |
@@ -154,13 +101,8 @@ provisioned. State is persisted in a remote backend matched to the provider:
154101

155102
Two operational properties matter here:
156103

157-
- **Locking prevents concurrent writes.** Two engineers cannot accidentally
158-
apply conflicting changes at the same time; the second `nic deploy` blocks
159-
until the first releases the lock.
160-
- **Drift detection is just `terraform plan`.** Running `nic deploy` against
161-
an unchanged config produces a diff between the state file and the live
162-
cloud, surfacing any out-of-band changes (a node group resized in the
163-
console, an IAM policy edited by hand) as proposed corrections.
104+
- **Locking prevents concurrent writes.** Two engineers cannot accidentally apply conflicting changes at the same time; the second `nic deploy` blocks until the first releases the lock.
105+
- **Drift detection is just `terraform plan`.** Running `nic deploy` against an unchanged config produces a diff between the state file and the live cloud, surfacing any out-of-band changes (a node group resized in the console, an IAM policy edited by hand) as proposed corrections.
164106

165107
State operations are exposed as first-class CLI verbs:
166108

@@ -173,16 +115,11 @@ nic state mv <from> <to> # rename a resource address
173115

174116
### How this differs from Nebari Classic
175117

176-
Classic also used Terraform, but the platform layer above the cluster was
177-
installed imperatively at deploy time and configured per-environment. NKP
178-
splits that responsibility: `nic`'s state covers infrastructure only, while
179-
the platform layer is reconciled by ArgoCD from a Git repository. As a
180-
result:
118+
Classic also used Terraform, but the platform layer above the cluster was installed imperatively at deploy time and configured per-environment. NKP splits that responsibility: `nic`'s state covers infrastructure only, while the platform layer is reconciled by ArgoCD from a Git repository. As a result:
181119

182120
- **Infrastructure drift** is detectable via `nic deploy` (Terraform plan).
183121
- **Application drift** is detectable via the ArgoCD UI / CLI (sync status).
184-
- The two surfaces never overlap, so a Terraform run cannot break an
185-
application and a pack upgrade cannot rotate a VPC.
122+
- The two surfaces never overlap, so a Terraform run cannot break an application and a pack upgrade cannot rotate a VPC.
186123

187124
## Separation of concerns
188125

@@ -196,19 +133,13 @@ NKP is built around three roles with non-overlapping lifecycles:
196133

197134
The boundary is enforced by the architecture, not by convention:
198135

199-
- A pack developer never needs cluster admin or Terraform credentials. They
200-
publish a pack to the registry; the operator and ArgoCD do the rest.
201-
- A platform team upgrade (rotating cert-manager, resizing nodes, swapping
202-
the ingress controller) does not require coordination with pack developers.
203-
- An end user never sees Kubernetes. They see a Capability on the landing
204-
page and click into it.
136+
- A pack developer never needs cluster admin or Terraform credentials. They publish a pack to the registry; the operator and ArgoCD do the rest.
137+
- A platform team upgrade (rotating cert-manager, resizing nodes, swapping the ingress controller) does not require coordination with pack developers.
138+
- An end user never sees Kubernetes. They see a Capability on the landing page and click into it.
205139

206-
This is the "why" behind the layering: each role can move at its own cadence
207-
without breaking the others.
140+
This is the "why" behind the layering: each role can move at its own cadence without breaking the others.
208141

209142
## Where to go next
210143

211-
- [Get started with NKP](/docs/nkp/get-started): install the platform and
212-
run your first cluster.
213-
- [Browse Software Packs](/docs/software-packs/): see what Capabilities are
214-
available out of the box.
144+
- [Get started with NKP](/docs/nkp/get-started): install the platform and run your first cluster.
145+
- [Browse Software Packs](/docs/software-packs/): see what Capabilities are available out of the box.

docs/sidebars.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ module.exports = {
3232
label: "Explanations",
3333
link: { type: "doc", id: "explanations/index" },
3434
items: [
35-
// Add explanation docs here
35+
"explanations/nkp-architecture",
3636
],
3737
},
3838
{

0 commit comments

Comments
 (0)