Skip to content

Commit b7d877a

Browse files
jubradclaude
jubrad
and
claude
committed
Add Helm and CRD override variables for full customization
- Add `materialize_spec_override` to materialize-instance module for arbitrary CRD fields - Expose `helm_values_override` and `materialize_spec_override` in all cloud examples - Change default authenticator_kind to Password for all deployments - Add deepmerge provider for spec merging - Add Python script to generate Terraform types from upstream CRD/Helm schemas - Set up pyproject.toml with uv for dependency management - Add GitHub Action to verify schema sync with upstream Materialize - Update CONTRIBUTING.md with development setup and type generation docs 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 52947fd commit b7d877a

26 files changed

Lines changed: 2879 additions & 123 deletions

File tree

‎.github/workflows/check-schema-sync.yml‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
name: Check Schema Sync
2+
3+
on:
4+
merge_group:
5+
pull_request:
6+
branches: [main]
7+
schedule:
8+
# Run weekly to detect upstream schema changes
9+
- cron: "0 0 * * 0"
10+
workflow_dispatch:
11+
12+
jobs:
13+
check-schema-sync:
14+
name: Verify CRD and Helm Schema Sync
15+
runs-on: ubuntu-latest
16+
steps:
17+
- name: Checkout code
18+
uses: actions/checkout@v4
19+
20+
- name: Install uv
21+
uses: astral-sh/setup-uv@v4
22+
with:
23+
version: "latest"
24+
25+
- name: Set up Python
26+
run: uv python install 3.12
27+
28+
- name: Install dependencies
29+
run: uv sync
30+
31+
- name: Extract default Materialize version
32+
id: get-version
33+
run: |
34+
# Extract version from the materialize-instance module (marked with # META: mz version)
35+
VERSION=$(grep -E "default.*=.*\"v[0-9]+\.[0-9]+\.[0-9]+\".*# META: mz version" \
36+
kubernetes/modules/materialize-instance/variables.tf | \
37+
grep -oE "v[0-9]+\.[0-9]+\.[0-9]+" | head -1)
38+
echo "version=$VERSION" >> $GITHUB_OUTPUT
39+
echo "Detected Materialize version: $VERSION"
40+
41+
- name: Validate CRD schema is accessible
42+
run: |
43+
VERSION="${{ steps.get-version.outputs.version }}"
44+
CRD_URL="https://raw.githubusercontent.com/MaterializeInc/materialize/${VERSION}/doc/user/data/self_managed/materialize_crd_descriptions.json"
45+
echo "Checking CRD schema at: $CRD_URL"
46+
47+
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "$CRD_URL")
48+
if [ "$HTTP_STATUS" != "200" ]; then
49+
echo "::error::CRD schema not found at $CRD_URL (HTTP $HTTP_STATUS)"
50+
echo "This may indicate the Materialize version $VERSION does not exist or the schema path has changed."
51+
exit 1
52+
fi
53+
echo "CRD schema accessible"
54+
55+
- name: Validate Helm values schema is accessible
56+
run: |
57+
VERSION="${{ steps.get-version.outputs.version }}"
58+
HELM_URL="https://raw.githubusercontent.com/MaterializeInc/materialize/${VERSION}/misc/helm-charts/operator/values.yaml"
59+
echo "Checking Helm values at: $HELM_URL"
60+
61+
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "$HELM_URL")
62+
if [ "$HTTP_STATUS" != "200" ]; then
63+
echo "::error::Helm values not found at $HELM_URL (HTTP $HTTP_STATUS)"
64+
echo "This may indicate the Materialize version $VERSION does not exist or the values path has changed."
65+
exit 1
66+
fi
67+
echo "Helm values schema accessible"
68+
69+
- name: Run type generation script (dry run)
70+
run: |
71+
VERSION="${{ steps.get-version.outputs.version }}"
72+
echo "Running type generation script for version $VERSION..."
73+
uv run python scripts/generate_terraform_types.py --version "$VERSION" --output simplified > /dev/null
74+
echo "Type generation script completed successfully"
75+
76+
- name: Check for new CRD fields
77+
run: |
78+
VERSION="${{ steps.get-version.outputs.version }}"
79+
CRD_URL="https://raw.githubusercontent.com/MaterializeInc/materialize/${VERSION}/doc/user/data/self_managed/materialize_crd_descriptions.json"
80+
81+
# Fetch current CRD fields
82+
curl -s "$CRD_URL" | python3 -c "
83+
import json
84+
import sys
85+
86+
data = json.load(sys.stdin)
87+
88+
# Find MaterializeSpec
89+
for item in data:
90+
if item[0] == 'MaterializeSpec':
91+
fields = item[1]
92+
print('MaterializeSpec fields:')
93+
for field in fields:
94+
deprecated = ' (DEPRECATED)' if field.get('deprecated', False) else ''
95+
required = ' (required)' if field.get('required', False) else ''
96+
print(f\" - {field['name']}: {field['type']}{required}{deprecated}\")
97+
break
98+
"
99+
100+
- name: Summary
101+
run: |
102+
VERSION="${{ steps.get-version.outputs.version }}"
103+
echo "## Schema Sync Check Summary" >> $GITHUB_STEP_SUMMARY
104+
echo "" >> $GITHUB_STEP_SUMMARY
105+
echo "- **Materialize Version:** $VERSION" >> $GITHUB_STEP_SUMMARY
106+
echo "- **CRD Schema:** Accessible" >> $GITHUB_STEP_SUMMARY
107+
echo "- **Helm Values:** Accessible" >> $GITHUB_STEP_SUMMARY
108+
echo "- **Type Generation:** Successful" >> $GITHUB_STEP_SUMMARY
109+
echo "" >> $GITHUB_STEP_SUMMARY
110+
echo "To regenerate types after a version bump, run:" >> $GITHUB_STEP_SUMMARY
111+
echo "\`\`\`bash" >> $GITHUB_STEP_SUMMARY
112+
echo "uv run python scripts/generate_terraform_types.py --version $VERSION" >> $GITHUB_STEP_SUMMARY
113+
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY

‎.python-version‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.12

‎CONTRIBUTING.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,24 @@ Pull requests are the best way to propose changes to the codebase. We actively w
2222
6. Issue that pull request!
2323

2424

25+
## Development Setup
26+
27+
This project uses [uv](https://docs.astral.sh/uv/) for Python dependency management. Install uv first:
28+
29+
```bash
30+
# macOS/Linux
31+
curl -LsSf https://astral.sh/uv/install.sh | sh
32+
33+
# Or with Homebrew
34+
brew install uv
35+
```
36+
37+
Then install the development dependencies:
38+
39+
```bash
40+
uv sync
41+
```
42+
2543
## Generating Documentation
2644

2745
This module uses [terraform-docs](https://terraform-docs.io/user-guide/introduction/) to generate documentation. To generate the documentation, run the following command from the root of the repository:
@@ -30,6 +48,23 @@ This module uses [terraform-docs](https://terraform-docs.io/user-guide/introduct
3048
.github/scripts/generate-docs.sh
3149
```
3250

51+
## Generating Terraform Type Definitions
52+
53+
When the Materialize CRD or Helm chart schema changes, you can regenerate the Terraform type definitions:
54+
55+
```bash
56+
# Generate simplified types (recommended for most use cases)
57+
uv run python scripts/generate_terraform_types.py --output simplified
58+
59+
# Generate from a specific Materialize version
60+
uv run python scripts/generate_terraform_types.py --version v26.1.0
61+
62+
# Generate full auto-generated types from upstream schemas
63+
uv run python scripts/generate_terraform_types.py --output full
64+
```
65+
66+
The script fetches the CRD field descriptions and Helm values from the Materialize repository and generates corresponding Terraform variable type definitions.
67+
3368
## Development Process
3469

3570
1. Clone the repository

‎aws/examples/simple/main.tf‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -257,6 +257,9 @@ module "operator" {
257257
# node selector for operator and metrics-server workloads
258258
operator_node_selector = local.generic_node_labels
259259

260+
# Pass through any additional Helm values for full customization
261+
# See https://materialize.com/docs/installation/configuration/
262+
helm_values = var.helm_values_override
260263

261264
depends_on = [
262265
module.eks,
@@ -331,7 +334,7 @@ module "materialize_instance" {
331334
force_rollout = var.force_rollout
332335
request_rollout = var.request_rollout
333336

334-
# The password for the external login to the Materialize instance
337+
# Password authentication for the Materialize instance
335338
external_login_password_mz_system = random_password.external_login_password_mz_system.result
336339
authenticator_kind = "Password"
337340

@@ -347,6 +350,10 @@ module "materialize_instance" {
347350
kind = "ClusterIssuer"
348351
}
349352

353+
# Pass through any additional CRD spec fields for full customization
354+
# See https://materialize.com/docs/installation/appendix-materialize-crd-field-descriptions/
355+
materialize_spec_override = var.materialize_spec_override
356+
350357
depends_on = [
351358
module.eks,
352359
module.database,

0 commit comments

Comments
 (0)