Skip to content

Release/25.12.1#68

Merged
xe-nvdk merged 11 commits into
mainfrom
release/25.12.1
Nov 28, 2025
Merged

Release/25.12.1#68
xe-nvdk merged 11 commits into
mainfrom
release/25.12.1

Conversation

@xe-nvdk
Copy link
Copy Markdown
Member

@xe-nvdk xe-nvdk commented Nov 27, 2025

No description provided.

Ignacio Van Droogenbroeck added 11 commits November 27, 2025 15:37
- Add cmd/arc/main.go (was ignored by git)
- Add helm/arc chart (was ignored by git)
- Remove -tags=duckdb_arrow from Dockerfile (not needed)
- Remove CGO_ENABLED=0 and comment from workflow
- Simplify build commands
DuckDB requires CGO, so we use Docker buildx for cross-compiling
arm64 binaries on the amd64 runner.
- Fix all arc-go URLs to arc (image name, homepage, docs)
- Add test-docker job: pulls image, tests /health endpoint
- Add test-binary job: runs binary, tests /health endpoint
- Update release notes with migration highlights:
  - 9.47M rec/s MessagePack (125% faster than Python)
  - Memory stable (no 372MB leak)
  - Full feature parity checklist
  - Breaking changes and upgrade guide
- Add tests to release job dependencies
- Add test status to job summary
- Add throughput badges (9.47M rec/s, 2.88M rows/s)
- Add Go version, license, docs, website, Discord badges
- Add :latest tag to Docker manifest (GHCR doesn't auto-tag)
- Update installation section with Docker, deb, rpm, Helm
- Update repo references from arc-go to arc
- Update Go version from 1.22 to 1.25
- Replace Docker/QEMU emulation with native GitHub runners
- Use ubuntu-24.04-arm for linux-arm64 builds (native ARM runner)
- Use ubuntu-latest for linux-amd64 builds
- Enable CGO (required for SQLite auth database)
- Build time reduced from 30+ minutes to ~2-3 minutes
BuildArch is for noarch packages only. For architecture-specific
binaries, use rpmbuild --target instead.
Existing databases created by Python don't have the expires_at column.
This adds a migration that runs ALTER TABLE to add the column if missing.
MessagePack sends integers (int8, int16, etc.) when values are whole
numbers like 0 or 10. This adds type coercion to float64 for all
integer types in float columns.

Fixes: "unexpected type in float column 'speed': int8"
- local.go: Retry CreateTemp on ENOENT (directory deleted externally),
  invalidate cache and recreate directory before retrying
- arrow_writer.go: Add float32/float64 cases to int column handler,
  truncating floats to int64 when schema inferred integer type
When Telegraf pushes columnar msgpack data with evolving schemas
(columns appear/disappear between batches), mergeBatches() would
fail because it only allocated space for columns where they exist,
causing: "panic: arrow/array: mismatch number of rows in column"

This fix adds schema-aware buffering that detects when columns change
between batches and flushes the buffer before adding new data. This
ensures all batches in a buffer have identical schemas.

Changes:
- Add bufferSchemas map to track column signature per buffer
- Add getColumnSignature() helper for schema comparison
- Detect schema evolution in writeColumnar() and flush on change
- Clean up bufferSchemas on buffer deletion
@xe-nvdk xe-nvdk merged commit ea703e3 into main Nov 28, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant