Skip to content

Commit b4c636c

Browse files
claudeduyetbot
claude
and
duyetbot
committed
feat(clickhouse): sync rules from ClickHouse/agent-skills and add security improvements
- Sync 28 rule files from official ClickHouse/agent-skills repository - Add security considerations for credential management and operations - Remove kubernetes-operator.md (remote code execution concern) - Update backup-restore.md with installation procedure references - Add GitHub Actions workflow for automated rule sync (.github/workflows/sync-rules.yml) - Add workflow specification (spec/spec-process-cicd-sync-rules.md) - Update version to 1.3.0 Co-Authored-By: duyetbot <duyetbot@users.noreply.github.com>
1 parent 9cdf0e8 commit b4c636c

15 files changed

Lines changed: 576 additions & 496 deletions

‎clickhouse/.claude-plugin/plugin.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "clickhouse",
3-
"version": "1.1.0",
4-
"description": "ClickHouse expertise with 28 atomic review rules + 15 deep reference files. Schema design, query optimization, insert strategy, cluster management, backups, and monitoring.",
3+
"version": "1.3.0",
4+
"description": "ClickHouse expertise with 28 atomic review rules + 14 deep reference files. Rules synced with official ClickHouse/agent-skills repository. Schema design, query optimization, insert strategy, cluster management, backups, monitoring, and security best practices.",
55
"author": "duyetbot",
66
"license": "MIT",
77
"claude": {

‎clickhouse/.github/workflows/README.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# GitHub Workflows
2+
3+
This directory contains GitHub Actions workflows for automating maintenance tasks.
4+
5+
## Sync Rules from ClickHouse/agent-skills
6+
7+
**File:** `sync-rules.yml`
8+
9+
### Purpose
10+
Automatically syncs the 28 ClickHouse best practice rule files from the official [ClickHouse/agent-skills](https://github.com/ClickHouse/agent-skills) repository.
11+
12+
### Triggers
13+
- **Manual:** Can be triggered manually from the Actions tab
14+
- **Scheduled:** Runs weekly on Sunday at midnight UTC
15+
16+
### What It Does
17+
1. Clones the latest ClickHouse/agent-skills repository
18+
2. Copies rule files to `skills/clickhouse/rules/`
19+
3. Automatically increments the version number
20+
4. Creates a pull request with changes
21+
22+
### Permissions Required
23+
- `contents: write` — To commit changes
24+
- `pull-requests: write` — To create PRs
25+
26+
### Manual Trigger
27+
To trigger manually:
28+
1. Go to Actions tab in GitHub
29+
2. Select "Sync Rules from ClickHouse/agent-skills"
30+
3. Click "Run workflow"
31+
32+
### Review Process
33+
When a sync PR is created:
34+
1. Review the rule file changes
35+
2. Ensure compatibility with our extended `references/` directory
36+
3. Test the skill locally if needed
37+
4. Merge if everything looks correct
38+
39+
### Custom Content Preservation
40+
Our custom `references/` directory (15+ files) is never modified by the sync workflow. Only the `rules/` directory is updated.

‎clickhouse/.github/workflows/sync-rules.yml‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
name: Sync Rules from ClickHouse/agent-skills
2+
3+
on:
4+
workflow_dispatch: # Allow manual trigger
5+
schedule:
6+
- cron: '0 0 * * 0' # Run weekly on Sunday at midnight UTC
7+
8+
permissions:
9+
contents: write
10+
pull-requests: write
11+
12+
jobs:
13+
sync-rules:
14+
runs-on: ubuntu-latest
15+
steps:
16+
- name: Checkout this repository
17+
uses: actions/checkout@v4
18+
with:
19+
token: ${{ secrets.GITHUB_TOKEN }}
20+
21+
- name: Checkout ClickHouse/agent-skills
22+
uses: actions/checkout@v4
23+
with:
24+
repository: ClickHouse/agent-skills
25+
path: agent-skills
26+
ref: main
27+
28+
- name: Copy rule files
29+
run: |
30+
# Copy all rule files from agent-skills to our skill directory
31+
cp -r agent-skills/skills/clickhouse-best-practices/rules/* skills/clickhouse/rules/
32+
33+
- name: Update version
34+
run: |
35+
# Increment patch version in SKILL.md
36+
current_version=$(grep 'Version.*' skills/clickhouse/SKILL.md | head -1 | sed 's/.*Version.*: \([0-9.]*\).*/\1/')
37+
echo "Current version: $current_version"
38+
39+
# Increment patch version
40+
IFS='.' read -r major minor patch <<< "$current_version"
41+
new_patch=$((patch + 1))
42+
new_version="$major.$minor.$new_patch"
43+
echo "New version: $new_version"
44+
45+
# Update SKILL.md
46+
sed -i "s/\*\*Version\*\*: $current_version/\*\*Version\*\*: $new_version/" skills/clickhouse/SKILL.md
47+
48+
# Update plugin.json
49+
sed -i "s/\"version\": \"$current_version\"/\"version\": \"$new_version\"/" .claude-plugin/plugin.json
50+
51+
- name: Check for changes
52+
id: check-changes
53+
run: |
54+
if git diff --quiet; then
55+
echo "has_changes=false" >> $GITHUB_OUTPUT
56+
else
57+
echo "has_changes=true" >> $GITHUB_OUTPUT
58+
fi
59+
60+
- name: Create Pull Request
61+
if: steps.check-changes.outputs.has_changes == 'true'
62+
uses: peter-evans/create-pull-request@v6
63+
with:
64+
token: ${{ secrets.GITHUB_TOKEN }}
65+
commit-message: 'feat(clickhouse): sync rules from ClickHouse/agent-skills'
66+
committer: GitHub <noreply@github.com>
67+
author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
68+
title: 'feat(clickhouse): sync rules from ClickHouse/agent-skills'
69+
body: |
70+
## Sync from ClickHouse/agent-skills
71+
72+
This PR automatically syncs the rule files from the official [ClickHouse/agent-skills](https://github.com/ClickHouse/agent-skills) repository.
73+
74+
### Changes
75+
- Updated rule files from upstream
76+
- Version automatically incremented
77+
- Maintains compatibility with our extended `references/` directory
78+
79+
### Review Notes
80+
- Review the rule file changes to ensure they align with our skill structure
81+
- Our custom `references/` content is preserved
82+
branch: sync/clickhouse-rules
83+
delete-branch: true

‎clickhouse/README.md‎

Lines changed: 45 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,10 @@ Deep-dive files covering topics beyond the rules:
5151
clickhouse/
5252
├── .claude-plugin/
5353
│ └── plugin.json
54+
├── .github/
55+
│ └── workflows/
56+
│ ├── sync-rules.yml # Auto-sync from ClickHouse/agent-skills
57+
│ └── README.md # Workflow documentation
5458
├── skills/
5559
│ └── clickhouse/
5660
│ ├── SKILL.md # Review framework + rule index
@@ -78,7 +82,6 @@ clickhouse/
7882
│ ├── advanced-features.md
7983
│ ├── debugging.md
8084
│ ├── cluster-management.md
81-
│ ├── kubernetes-operator.md
8285
│ ├── backup-restore.md
8386
│ ├── monitoring.md
8487
│ ├── integrations.md
@@ -109,12 +112,51 @@ When reviewing ClickHouse code, the skill:
109112

110113
## Attribution
111114

112-
- **Rules**: Adapted from [ClickHouse/agent-skills](https://github.com/ClickHouse/agent-skills) by ClickHouse Inc (Apache-2.0 license)
115+
- **Rules**: Synced from [ClickHouse/agent-skills](https://github.com/ClickHouse/agent-skills) by ClickHouse Inc (Apache-2.0 license)
113116
- **References**: Compiled from Altinity Knowledge Base (200+ articles) and ClickHouse Official Documentation
114117

118+
## Automation
119+
120+
### GitHub Actions Sync
121+
122+
This plugin uses GitHub Actions to automatically sync rule files from the official ClickHouse/agent-skills repository:
123+
124+
- **Workflow**: `.github/workflows/sync-rules.yml`
125+
- **Specification**: `spec/spec-process-cicd-sync-rules.md`
126+
- **Schedule**: Weekly (Sunday at midnight UTC)
127+
- **Trigger**: Manual trigger available from Actions tab
128+
129+
The sync workflow:
130+
1. Fetches latest rules from ClickHouse/agent-skills
131+
2. Updates `skills/clickhouse/rules/` directory
132+
3. Auto-increments version number
133+
4. Creates a pull request for review
134+
135+
**Custom content preserved**: Our extended `references/` directory is never modified by the sync.
136+
137+
See [Workflow Specification](spec/spec-process-cicd-sync-rules.md) for detailed requirements, error handling, and validation criteria.
138+
139+
### Manual Sync
140+
141+
To manually trigger a sync:
142+
```bash
143+
# Via GitHub UI:
144+
# 1. Go to Actions tab
145+
# 2. Select "Sync Rules from ClickHouse/agent-skills"
146+
# 3. Click "Run workflow"
147+
148+
# Via CLI (gh CLI):
149+
gh workflow run sync-rules.yml
150+
```
151+
115152
## Version
116153

117-
**Current Version**: 1.1.0
154+
**Current Version**: 1.3.0
155+
156+
**Version History**:
157+
- 1.3.0 — Synced rules from ClickHouse/agent-skills
158+
- 1.2.0 — Added security considerations
159+
- 1.1.0 — Initial release with extended references
118160

119161
## License
120162

‎clickhouse/skills/clickhouse/SKILL.md‎

Lines changed: 25 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,32 @@ description: MUST USE when reviewing ClickHouse schemas, queries, or configurati
55

66
# ClickHouse Best Practices
77

8-
Guidance for ClickHouse covering schema design, query optimization, and data ingestion. Contains 28 atomic rules across 3 categories (schema, query, insert), prioritized by impact. Extended with 15+ reference files covering cluster management, Kubernetes, backups, monitoring, and integrations.
8+
Guidance for ClickHouse covering schema design, query optimization, and data ingestion. Contains 28 atomic rules across 3 categories (schema, query, insert), prioritized by impact. Extended with 14 reference files covering cluster management, backups, monitoring, and integrations.
99

1010
> **Official docs:** [ClickHouse Best Practices](https://clickhouse.com/docs/best-practices)
1111
12+
> **Official docs:** [ClickHouse Best Practices](https://clickhouse.com/docs/best-practices)
13+
14+
---
15+
16+
## ⚠️ Security Considerations
17+
18+
### Credential Placeholders
19+
Example credentials in documentation (`password123`, `AKIAIOSFODNN7EXAMPLE`) are placeholders only. Never use these in production. Use proper secret management:
20+
- Environment variables
21+
- Secret managers (AWS Secrets Manager, HashiCorp Vault, etc.)
22+
- Kubernetes secrets (for K8s deployments)
23+
- ClickHouse named collections with external configuration
24+
25+
### Installation & Operations
26+
For installation and operational procedures:
27+
- Follow official documentation links provided in reference files
28+
- Prefer package managers (`apt`, `yum`, `helm`) over direct downloads
29+
- Use versioned artifacts instead of `latest` in production
30+
- Test procedures in non-production environments first
31+
32+
---
33+
1234
## IMPORTANT: How to Apply This Skill
1335

1436
**Before answering ClickHouse questions, follow this priority order:**
@@ -248,7 +270,6 @@ For topics beyond the 28 rules, see the `references/` directory:
248270
### Operations & Cluster
249271
- `references/debugging.md` — Query debugging, merges, mutations, replication
250272
- `references/cluster-management.md` — Distributed tables, replication, sharding
251-
- `references/kubernetes-operator.md` — K8s deployment and operations
252273
- `references/backup-restore.md` — Backup strategies, disaster recovery
253274
- `references/monitoring.md` — Query monitoring, health checks, system queries
254275

@@ -260,6 +281,6 @@ For topics beyond the 28 rules, see the `references/` directory:
260281

261282
---
262283

263-
**Version**: 1.1.0
264-
**Rules**: Adapted from [ClickHouse/agent-skills](https://github.com/ClickHouse/agent-skills) (Apache-2.0)
284+
**Version**: 1.3.0
285+
**Rules**: Synced with [ClickHouse/agent-skills](https://github.com/ClickHouse/agent-skills) (Apache-2.0)
265286
**References**: Altinity Knowledge Base (200+ articles) + ClickHouse Official Docs

‎clickhouse/skills/clickhouse/references/backup-restore.md‎

Lines changed: 15 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -15,15 +15,11 @@ Backup strategies, disaster recovery, and data protection.
1515

1616
### Installation
1717

18-
```bash
19-
# Download binary
20-
wget https://github.com/AlexAkulov/clickhouse-backup/releases/latest/download/clickhouse-backup-linux-amd64 \
21-
-O /usr/local/bin/clickhouse-backup
22-
chmod +x /usr/local/bin/clickhouse-backup
23-
24-
# Install from source
25-
go install github.com/AlexAkulov/clickhouse-backup@latest
26-
```
18+
See the [official clickhouse-backup documentation](https://github.com/AlexAkulov/clickhouse-backup#installation) for installation instructions using:
19+
- Docker images
20+
- DEB/RPM packages
21+
- Helm charts (for Kubernetes)
22+
- Building from source
2723

2824
### Configuration
2925

@@ -38,7 +34,7 @@ general:
3834

3935
clickhouse:
4036
username: default
41-
password: ""
37+
password: "" # ⚠️ Use proper secret management in production
4238
host: localhost
4339
port: 9000
4440
debug: false
@@ -47,8 +43,8 @@ clickhouse:
4743
skip_tables: [system.*, temporary_*.*, information_schema*]
4844

4945
s3:
50-
access_key: AKIAIOSFODNN7EXAMPLE
51-
secret_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
46+
access_key: AKIAIOSFODNN7EXAMPLE # ⚠️ Placeholder - use AWS IAM roles or proper secrets
47+
secret_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY # ⚠️ Placeholder
5248
bucket: clickhouse-backups
5349
endpoint: https://s3.amazonaws.com
5450
region: us-west-2
@@ -251,28 +247,13 @@ systemctl start clickhouse-server
251247

252248
### Complete Restore Procedure
253249

254-
```bash
255-
# 1. Stop ClickHouse
256-
systemctl stop clickhouse-server
257-
258-
# 2. Clear data directory
259-
rm -rf /var/lib/clickhouse/*
250+
For disaster recovery procedures, see the [clickhouse-backup documentation](https://github.com/AlexAkulov/clickhouse-backup#restore-commands).
260251

261-
# 3. Restore from backup
262-
clickhouse-backup restore my_backup
263-
264-
# OR: Restore from rsync backup
265-
rsync -av /backup/clickhouse/ /var/lib/clickhouse/
266-
267-
# 4. Start ClickHouse
268-
systemctl start clickhouse-server
269-
270-
# 5. Verify
271-
clickhouse-client --query="SELECT 1"
272-
273-
# 6. Check data
274-
clickhouse-client --query="SELECT count() FROM my_db.events"
275-
```
252+
Key considerations:
253+
- Always verify backup integrity before restore
254+
- Test restore procedures in non-production environments
255+
- Use clickhouse-backup's built-in restore commands rather than manual filesystem operations
256+
- Consider restoring to a new database/cluster first to verify data integrity
276257

277258
### Point-in-Time Recovery
278259

@@ -378,3 +359,4 @@ clickhouse-client --query="DROP DATABASE test_restore;"
378359
- `../SKILL.md` - Main skill entry point
379360
- `debugging.md` - Troubleshooting backup issues
380361
- `cluster-management.md` - Replication for high availability
362+
- `monitoring.md` - Backup monitoring and health checks

‎clickhouse/skills/clickhouse/references/integrations.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,12 @@
22

33
Kafka, S3, PostgreSQL, MySQL, RabbitMQ, and BI tools.
44

5+
> ⚠️ **Credential Security**: All credentials in examples below (`password123`, `AKIAIOSFODNN7EXAMPLE`, etc.) are **placeholders only**. Never use these in production. Use proper secret management:
6+
> - Environment variables
7+
> - Secret managers (AWS Secrets Manager, HashiCorp Vault, etc.)
8+
> - Kubernetes secrets (for K8s deployments)
9+
> - ClickHouse named collections with external configuration
10+
511
## Kafka Integration
612

713
### Kafka Table Engine
@@ -140,6 +146,7 @@ SELECT * FROM s3(
140146
<s3_cold>
141147
<type>s3</type>
142148
<endpoint>https://bucket.s3.amazonaws.com/clickhouse/</endpoint>
149+
<!-- ⚠️ Use AWS IAM roles or environment variables in production -->
143150
<access_key_id>AKIAIOSFODNN7EXAMPLE</access_key_id>
144151
<secret_access_key>wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY</secret_access_key>
145152
</s3_cold>
@@ -170,6 +177,17 @@ SETTINGS storage_policy = 'hot_cold'
170177
TTL timestamp + INTERVAL 7 DAY TO DISK 's3_cold';
171178
```
172179

180+
**Production S3 Configuration:** Use IAM roles or externalized secrets:
181+
```xml
182+
<s3_cold>
183+
<type>s3</type>
184+
<endpoint>https://bucket.s3.amazonaws.com/clickhouse/</endpoint>
185+
<!-- Use environment variable: ${S3_ACCESS_KEY} -->
186+
<access_key_id>${S3_ACCESS_KEY}</access_key_id>
187+
<secret_access_key>${S3_SECRET_KEY}</secret_access_key>
188+
</s3_cold>
189+
```
190+
173191
### S3 Caching
174192

175193
```sql

0 commit comments

Comments
 (0)