Skip to content

adaptive export config#32

Merged
entlein merged 8 commits intomainfrom
entlein/adaptive-export-config
Apr 23, 2026
Merged

adaptive export config#32
entlein merged 8 commits intomainfrom
entlein/adaptive-export-config

Conversation

@entlein
Copy link
Copy Markdown

@entlein entlein commented Apr 21, 2026

Summary: Temporarily adding a dynamic method to toggle the plugin for clickhouse

Test Plan: manual

Type of change: /kind temporary

@entlein entlein temporarily deployed to pr-actions-approval April 21, 2026 12:38 — with GitHub Actions Inactive
Comment thread src/vizier/services/adaptive_export/cmd/main.go
@entlein entlein temporarily deployed to pr-actions-approval April 21, 2026 15:15 — with GitHub Actions Inactive
@entlein entlein temporarily deployed to pr-actions-approval April 21, 2026 16:20 — with GitHub Actions Inactive
@entlein
Copy link
Copy Markdown
Author

entlein commented Apr 21, 2026

merged in your e2e-clickhouse branch for local testing. A good time to setup that linter...

Copy link
Copy Markdown

@ddelnano ddelnano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the approach of this (having quiet ticks and different export modes).

I think this is the right harness, but we have to decide whether we will use the plugin system for its scheduling and script execution reporting or we will handle that entirely in this service. See my more detailed notes in this comment.

Comment thread k8s/vizier/bootstrap/adaptive_export_deployment.yaml
Comment thread src/vizier/services/adaptive_export/cmd/main.go
Comment thread src/vizier/services/adaptive_export/cmd/main.go
@entlein entlein force-pushed the entlein/adaptive-export-config branch from 418092c to 1b3b64a Compare April 22, 2026 10:07
@entlein entlein temporarily deployed to pr-actions-approval April 22, 2026 10:07 — with GitHub Actions Inactive
…e schema_creation option , probably needs complete rewrite, this is to make it work e2e for the lab now

Signed-off-by: entlein <einentlein@gmail.com>
@entlein entlein temporarily deployed to pr-actions-approval April 22, 2026 18:10 — with GitHub Actions Inactive
Signed-off-by: entlein <einentlein@gmail.com>
@entlein entlein temporarily deployed to pr-actions-approval April 23, 2026 10:28 — with GitHub Actions Inactive
Signed-off-by: entlein <einentlein@gmail.com>
@entlein entlein temporarily deployed to pr-actions-approval April 23, 2026 10:48 — with GitHub Actions Inactive
Signed-off-by: entlein <einentlein@gmail.com>
@entlein entlein temporarily deployed to pr-actions-approval April 23, 2026 11:18 — with GitHub Actions Inactive
Signed-off-by: entlein <einentlein@gmail.com>
@entlein entlein temporarily deployed to pr-actions-approval April 23, 2026 12:11 — with GitHub Actions Inactive
Signed-off-by: entlein <einentlein@gmail.com>
@entlein entlein temporarily deployed to pr-actions-approval April 23, 2026 12:29 — with GitHub Actions Inactive
Comment on lines +26 to +28
# v0.36.0 released 2026-04-22 (post-incident). Internally SHA-pins
# setup-trivy@3fb12ec = Aqua's safe v0.2.6 per GHSA-69fq-xp46-6x23.
- uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was fixed upstream (pixie-io#2335), but because we are ignoring certain file paths copybara is ignoring it.

Comment on lines +314 to +318
// Filter presets by an allow-list of case-insensitive substrings in the
// script name. Useful when the destination ClickHouse doesn't have every
// target table pre-created (Pixie's C++ ClickHouseExportSinkNode aborts
// kelvin on UNKNOWN_TABLE from CH — upstream bug), so we must not install
// retention scripts whose target table is missing.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry these presets were a footgun. I just added whatever we used for Atlanta. We can remove those if they don't make sense anymore.

I think having this ALLOWED_RETENTION_SCRIPTS is nice to keep regardless of what we do with the clickhouse plugin default scripts.

@entlein entlein marked this pull request as ready for review April 23, 2026 15:02
@entlein entlein merged commit 5fc4b57 into main Apr 23, 2026
18 of 21 checks passed
@entlein entlein deleted the entlein/adaptive-export-config branch April 23, 2026 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants