Skip to content

Commit bac1589

Browse files
butler54butler54
committed
fix: add agents.md
Signed-off-by: Chris Butler <chris.butler@redhat.com>
1 parent c2889e7 commit bac1589

3 files changed

Lines changed: 303 additions & 4 deletions

File tree

‎AGENTS.md‎

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# CoCo Pattern — AI Coding Assistant Guidance
2+
3+
This is a [Validated Pattern](https://validatedpatterns.io/) for deploying confidential containers (CoCo) on OpenShift.
4+
This file provides rules and context for any AI coding assistant working in this repository.
5+
6+
## Critical Rules
7+
8+
- **DO NOT** edit anything under `/common/`. It is a read-only git subtree from the upstream validated patterns framework.
9+
- **DO NOT** commit secrets, credentials, or private keys. `values-secret.yaml.template` is a template only.
10+
- **DO NOT** use Kustomize. This project uses Helm exclusively.
11+
- **DO NOT** create charts with `apiVersion: v1`. Use `apiVersion: v2` (Helm 3+).
12+
- **DO NOT** place cloud-provider-specific logic in chart templates. Use `/overrides/` via `sharedValueFiles` instead.
13+
- **DO NOT** hardcode secrets in templates. Use External Secrets Operator with vault paths (see `charts/hub/trustee/templates/dynamic-eso.yaml` for reference).
14+
15+
## Feature Development Precedence Order
16+
17+
Use the **first** approach that fits your requirement:
18+
19+
1. **Helm charts** — Declarative Kubernetes resources in `/charts/`, deployed by ArgoCD. Preferred for installing operators, configuring CRDs, and creating Kubernetes resources.
20+
2. **ACM policies** — Red Hat Advanced Cluster Management policies for propagating configuration from hub to spoke clusters and enforcing multi-cluster governance. Reference: `charts/hub/sandbox-policies/templates/`.
21+
3. **Imperative framework (Ansible)** — Playbooks in `/ansible/`, executed as Kubernetes Jobs on a 10-minute schedule. **Must be idempotent.** Use for API calls, runtime data lookups, and multi-step orchestration that cannot be expressed declaratively. Register playbooks in `clusterGroup.imperative.jobs` as an ordered list.
22+
4. **Out-of-band scripts** — `/scripts/` or `/rhdp/`. Last resort for one-time setup or local development tooling. These are not managed by GitOps.
23+
24+
## Project Structure
25+
26+
```
27+
├── ansible/ # Ansible playbooks (imperative jobs)
28+
├── charts/
29+
│ ├── all/
30+
│ │ └── letsencrypt/ # Shared across cluster groups
31+
│ ├── coco-supported/
32+
│ │ ├── baremetal/ # Bare-metal TDX configuration
33+
│ │ ├── hello-openshift/ # Sample workloads
34+
│ │ ├── kbs-access/ # KBS access verification workload
35+
│ │ └── sandbox/ # Sandboxed containers runtime
36+
│ └── hub/
37+
│ ├── lvm-storage/ # LVM storage for bare-metal
38+
│ ├── sandbox-policies/ # ACM policies (hub → spoke)
39+
│ └── trustee/ # Trustee / KBS
40+
├── common/ # READ-ONLY — upstream framework subtree
41+
├── overrides/ # Cloud-provider value overrides
42+
│ ├── values-AWS.yaml
43+
│ ├── values-Azure.yaml
44+
│ └── values-IBMCloud.yaml
45+
├── rhdp/ # Red Hat Demo Platform tooling
46+
├── scripts/ # Utility scripts
47+
├── values-global.yaml # Global configuration
48+
├── values-simple.yaml # Cluster group: simple
49+
├── values-baremetal.yaml # Cluster group: baremetal
50+
├── values-trusted-hub.yaml # Cluster group: trusted-hub
51+
├── values-untrusted-spoke.yaml # Cluster group: untrusted-spoke
52+
└── values-secret.yaml.template # Secrets template (never commit filled-in copy)
53+
```
54+
55+
## Companion Chart Repositories
56+
57+
Several charts in this repo have companion repositories for independent versioning and reuse. Develop and test in this repo first (charts deploy via `path:`), then sync changes to the companion repo.
58+
59+
| Local Path | Companion Repo | Purpose |
60+
|---|---|---|
61+
| `charts/hub/trustee/` | `trustee-chart` | Trustee / KBS on hub |
62+
| `charts/hub/sandbox-policies/` | `sandboxed-policies-chart` | ACM policies hub → spoke |
63+
| `charts/coco-supported/sandbox/` | `sandboxed-containers-chart` | Sandboxed runtime on spoke |
64+
65+
Large features may require coordinated changes across multiple companion repos. References are org-agnostic — contributors should fork all relevant repos as needed.
66+
67+
## Cluster Groups
68+
69+
Set via `main.clusterGroupName` in `values-global.yaml`.
70+
71+
| Cluster Group | Values File | Role | Description |
72+
|---|---|---|---|
73+
| `simple` | `values-simple.yaml` | Hub (single cluster) | All components on one cluster |
74+
| `baremetal` | `values-baremetal.yaml` | Hub (single cluster) | TDX + LVM storage on bare metal |
75+
| `trusted-hub` | `values-trusted-hub.yaml` | Multi-cluster hub | Trustee + ACM policies |
76+
| `untrusted-spoke` | `values-untrusted-spoke.yaml` | Multi-cluster spoke | Sandbox runtime + workloads |
77+
78+
## Values File Hierarchy
79+
80+
Merge order (last wins):
81+
82+
1. Chart defaults (`charts/<group>/<chart>/values.yaml`)
83+
2. `values-global.yaml`
84+
3. `values-<clustergroup>.yaml`
85+
4. `/overrides/values-{{ clusterPlatform }}.yaml` (via `sharedValueFiles`)
86+
5. `values-secret.yaml` (runtime only, never committed)
87+
88+
Key conventions:
89+
90+
- Global settings go under the `global:` key in `values-global.yaml`.
91+
- Subscriptions go under `clusterGroup.subscriptions:` in the cluster group values file.
92+
- Applications go under `clusterGroup.applications:` in the cluster group values file.
93+
- Local charts use `path:` (e.g., `path: charts/hub/trustee`). Shared framework charts use `chart:` + `chartVersion:`.
94+
- Imperative jobs go under `clusterGroup.imperative.jobs:` as an **ordered list** (not a hash — hashes lose ordering in Helm).
95+
96+
## Helm Chart Conventions
97+
98+
- Use `apiVersion: v2`. Place charts in `charts/<cluster-group>/<chart-name>/`.
99+
- Use ArgoCD sync-wave annotations to control deployment ordering.
100+
- Use `ExternalSecret` resources to pull secrets from vault. Reference: `charts/hub/trustee/templates/dynamic-eso.yaml`.
101+
- Use `.Values.global.clusterPlatform` for platform-conditional logic only when overrides files are insufficient.
102+
- Reference patterns:
103+
- ESO integration: `charts/hub/trustee/templates/dynamic-eso.yaml`
104+
- Template helpers: `charts/coco-supported/hello-openshift/templates/_helpers.tpl`
105+
106+
## Ansible Playbook Conventions
107+
108+
- Place playbooks in `/ansible/`. They **must be idempotent**.
109+
- Use `connection: local`, `hosts: localhost`, `become: false`.
110+
- Use `kubernetes.core.k8s` and `kubernetes.core.k8s_info` modules for cluster interaction.
111+
- Register playbooks in the cluster group values file under `clusterGroup.imperative.jobs` with `name`, `playbook`, `verbosity`, and `timeout` fields.
112+
113+
## Git Workflow
114+
115+
- **Fork-first**: ArgoCD reconciles against your fork. Clone and push to your own fork.
116+
- **Conventional commits**: Enforced by commitlint (`@commitlint/config-conventional`).
117+
- **Branch-based deployment**: The branch of your local checkout determines the ArgoCD deployment target.
118+
- **Changes require commit + push** to take effect — ArgoCD watches the remote.
119+
120+
## Commands Reference
121+
122+
All commands run via `./pattern.sh make <target>`:
123+
124+
| Command | Purpose |
125+
|---|---|
126+
| `install` | Install the pattern and load secrets |
127+
| `show` | Render the starting template without installing |
128+
| `preview-all` | Preview all applications across cluster groups |
129+
| `validate-schema` | Validate values files against JSON schema |
130+
| `validate-cluster` | Validate cluster prerequisites |
131+
| `super-linter` | Run super-linter locally |
132+
| `load-secrets` | Load secrets into the configured backend |
133+
| `uninstall` | Uninstall the pattern |
134+
135+
See the README for secrets backend configuration, RHDP environment variables, and additional maintenance commands.
136+
137+
## Validation and CI
138+
139+
CI runs the following checks on pull requests:
140+
141+
- **JSON schema validation** — values files validated against `common/clustergroup` schema
142+
- **Super Linter** — multi-language linting
143+
- **Conventional PR title lint** — PR titles must follow conventional commit format
144+
145+
Run locally before pushing:
146+
147+
```bash
148+
./pattern.sh make preview-all
149+
./pattern.sh make validate-schema
150+
```

‎docs/dell-tdx-configuration.md‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# Enable Intel TDX on Dell PowerEdge via iDRAC
2+
3+
This guide provides step-by-step instructions for enabling Intel Trust Domain Extensions (TDX) on Dell PowerEdge servers using the iDRAC console.
4+
5+
## Prerequisites
6+
7+
- Dell 16th Generation PowerEdge server:
8+
- PowerEdge R660, R660xs
9+
- PowerEdge R760, R760xs, R760xd2, R760XA
10+
- PowerEdge R860, R960
11+
- PowerEdge XE8640, XE9640, XE9680
12+
- PowerEdge C6620, MX760c
13+
- PowerEdge XR5610, XR7620, XR8610t, XR8620t
14+
- PowerEdge T360, T560
15+
- 5th Gen Intel Xeon Scalable processor with TDX support
16+
- **8 or 16 DIMMs per socket** (required memory configuration)
17+
- Latest BIOS firmware installed
18+
19+
## Step-by-Step Instructions (Order Matters)
20+
21+
> **IMPORTANT:** Settings must be configured in this exact order. Some options (like "Multiple Keys") will be greyed out until prerequisite settings are applied. You may need to **save and reboot between steps** for dependent options to become available.
22+
23+
### 1. Access BIOS Setup via iDRAC
24+
25+
1. Log into the iDRAC web console
26+
2. Navigate to **Configuration → BIOS Settings**
27+
3. Alternatively, launch **Virtual Console** and press **F2** during POST to enter System Setup
28+
29+
### 2. Configure Memory Settings (FIRST)
30+
31+
Navigate to: **System BIOS → Memory Settings**
32+
33+
| Setting | Value |
34+
| ---------------------- | ----------- |
35+
| **Node Interleaving** | Disabled |
36+
37+
**Save and reboot** before proceeding.
38+
39+
### 3. Configure Processor Prerequisites (SECOND)
40+
41+
Navigate to: **System BIOS → Processor Settings**
42+
43+
| Setting | Value |
44+
| -------------------------------- | -------- |
45+
| **Logical Processor (x2APIC)** | Enabled |
46+
| **CPU Physical Address Limit** | Disabled |
47+
48+
**Save and reboot** before proceeding.
49+
50+
### 4. Enable Memory Encryption - Multiple Keys (THIRD)
51+
52+
Navigate to: **System BIOS → System Security**
53+
54+
| Setting | Value |
55+
| --------------------- | -------------- |
56+
| **Memory Encryption** | Multiple Keys |
57+
58+
> If "Multiple Keys" is still greyed out, verify steps 2 and 3 were applied and the system was rebooted.
59+
60+
**Save and reboot** before proceeding.
61+
62+
### 5. Configure TDX Settings (FOURTH)
63+
64+
Navigate to: **System BIOS → System Security** (or **Processor Settings** depending on BIOS version)
65+
66+
| Setting | Value |
67+
| ------------------------------------------------- | ------- |
68+
| **Global Memory Integrity** | Disabled |
69+
| **Intel TDX (Trust Domain Extension)** | Enabled |
70+
| **TME-MT/TDX Key Split** | 1 |
71+
| **TDX Secure Arbitration Mode Loader (SEAM)** | Enabled |
72+
73+
### 6. Configure SGX Settings (FIFTH)
74+
75+
Navigate to: **System BIOS → Processor Settings → Software Guard Extensions (SGX)**
76+
77+
| Setting | Value |
78+
| -------------------- | ------------------------- |
79+
| **Intel SGX** | Enabled |
80+
| **SGX Factory Reset** | Off |
81+
| **SGX PRMRR Size** | As needed (e.g., 64GB) |
82+
83+
### 7. Final Save and Reboot
84+
85+
1. Press **Escape** to exit menus
86+
2. Select **Save Changes and Exit**
87+
3. System will reboot with TDX enabled
88+
89+
## Configuration Summary (Order of Operations)
90+
91+
```
92+
1. Disable Node Interleaving → Save & Reboot
93+
2. Enable x2APIC Mode → Save & Reboot
94+
3. Disable CPU Physical Address Limit → Save & Reboot
95+
4. Set Memory Encryption = Multiple Keys → Save & Reboot
96+
5. Disable Global Memory Integrity
97+
6. Enable Intel TDX
98+
7. Set TME-MT/TDX Key Split = 1
99+
8. Enable SEAM Loader
100+
9. Enable Intel SGX → Final Save & Reboot
101+
```
102+
103+
## Verification
104+
105+
After the OS boots, verify TDX is enabled:
106+
107+
```bash
108+
# Check kernel messages for TDX
109+
dmesg | grep -i tdx
110+
# Should show: "virt/tdx: BIOS enabled: private KeyID range: [X, Y)"
111+
112+
# Check for TDX module
113+
ls /sys/firmware/tdx_seam/
114+
```
115+
116+
## Troubleshooting
117+
118+
### "Multiple Keys" Option is Greyed Out
119+
120+
This is typically caused by:
121+
122+
1. **Node Interleaving is Enabled** - Must be disabled first
123+
2. **x2APIC Mode is Disabled** - Must be enabled first
124+
3. **CPU Physical Address Limit is Enabled** - Must be disabled first
125+
4. **System not rebooted** - Some changes require reboot before dependent options appear
126+
5. **Insufficient DIMMs** - Requires 8 or 16 DIMMs per socket
127+
128+
### Settings Not Available
129+
130+
If TDX-related settings are not visible:
131+
132+
1. Ensure BIOS firmware is updated to the latest version
133+
2. Verify your processor supports TDX (5th Gen Xeon Scalable required)
134+
3. Contact Dell support for BIOS with TDX support
135+
136+
### TDX Not Detected by OS
137+
138+
If the OS doesn't detect TDX after configuration:
139+
140+
1. Verify all settings are correctly applied in the order specified
141+
2. Ensure the OS/kernel supports TDX (Linux 6.2+ recommended)
142+
3. Check that Memory Encryption is set to "Multiple Keys" (not "Single Key")
143+
144+
## References
145+
146+
- [Dell: Enable Intel TDX on Dell 16G Intel Servers](https://www.dell.com/support/kbdoc/en-us/000226452/enableinteltdxondell16g)
147+
- [Intel TDX Enabling Guide - Hardware Setup](https://cc-enabling.trustedservices.intel.com/intel-tdx-enabling-guide/04/hardware_setup/)
148+
- [Dell Info Hub: Enable Intel TDX in BIOS](https://infohub.delltechnologies.com/en-us/l/securing-ai-workloads-on-dell-poweredge-with-intel-xeon-processors-using-intel-trust-domain-extensions/appendix-b-enable-intel-r-tdx-in-bios/)
149+
- [Linux Kernel TDX Documentation](https://docs.kernel.org/arch/x86/tdx.html)

‎values-simple.yaml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -110,8 +110,8 @@ clusterGroup:
110110
targetRevision: remove-ssh
111111
path: ./
112112
overrides:
113-
- name: global.coco.enableSSHDebug
114-
value: "true"
113+
# - name: global.coco.enableSSHDebug
114+
# value: "true"
115115
- name: global.secretStore.backend
116116
value: vault
117117
- name: secretStore.name
@@ -126,8 +126,8 @@ clusterGroup:
126126
targetRevision: rootvolume
127127
path: ./
128128
overrides:
129-
- name: global.coco.enableSSHDebug
130-
value: "true"
129+
# - name: global.coco.enableSSHDebug
130+
# value: "true"
131131
- name: global.coco.azure.defaultVMFlavour
132132
value: Standard_DC2as_v5
133133
- name: global.coco.azure.VMFlavours

0 commit comments

Comments
 (0)