Release Post for v.2.0.0-rc.3 (#1546)

Author: wyattwenzel

website/docusaurus.config.ts

@@ -214,7 +214,7 @@ const config: Config = {
       { name: 'twitter:creator', content: '@spice_ai' }
     ],
     announcementBar: {
-      content: '<a href="/releases/v2.0-rc.2">Spice.ai OSS v2.0-rc.2</a> is now available! 🔥',
+      content: '<a href="/releases/v2.0-rc.3">Spice.ai OSS v2.0-rc.3</a> is now available! ⚡',
       backgroundColor: 'var(--announcement-bar-bg)',
       textColor: 'var(--announcement-bar-text)',
       isCloseable: true

website/releases/v2.0-rc.3.md

@@ -0,0 +1,235 @@
+---
+date: 2026-04-21
+title: 'Spice v2.0-rc.3 (Apr 21, 2026)'
+type: blog
+authors: [ewgenius]
+tags: [release, data-connector, databricks, snowflake, adbc, kafka, opentelemetry, observability]
+
+---
+Announcing the release of Spice v2.0-rc.3! ⚡
+
+v2.0.0-rc.3 is the third release candidate for advanced testing of v2.0, building on [v2.0.0-rc.2](https://github.com/spiceai/spiceai/releases/tag/v2.0.0-rc.2).
+
+Highlights in this release candidate include:
+
+- **HTTP Connector Enhancements** with OAuth2 refresh-token authentication, query-parameter pagination, and map-to-array conversion for broader API compatibility
+- **Databricks and Unity Catalog Reliability Improvements** with resilience controls, improved UC-awareness, permission checks, and structured error reporting
+- **Snowflake and ADBC Registration Performance Improvements** with better observability during dataset registration
+- **OpenTelemetry Exporter Improvements** with exporter fixes and support for authenticated metrics export
+- **Kafka, GitHub, and HTTP Connector Fixes** including Kafka reliability improvements, GitHub GraphQL resilience updates, and HTTP JSON union/reload fixes
+
+## What's New in v2.0.0-rc.3
+
+### HTTP Connector Enhancements
+
+The [HTTP connector](https://spiceai.org/docs/components/data-connectors/https) now supports more authentication and API response patterns, making it easier to integrate with modern REST APIs.
+
+**Key improvements**:
+
+- **OAuth2 Refresh-Token Authentication**: Added support for [OAuth2 refresh-token flows](https://spiceai.org/docs/next/components/data-connectors/https#using-oauth2-refresh-token-authentication) for APIs that issue short-lived access tokens.
+- **Query-Parameter Pagination**: Added pagination support using query parameters for APIs that expose page or cursor controls in the URL.
+- **Map-to-Array Conversion**: Added response transformation support for APIs that return map-shaped payloads that need to be normalized into arrays.
+- **Improved JSON Union Handling**: Better handling for heterogeneous JSON payloads during ingestion.
+- **More Reliable Reloads**: Fixed runtime behavior for HTTP-backed datasets during spicepod reloads.
+
+Example configuration of an HTTP connector using the [OAuth2 refresh token flow](https://spiceai.org/docs/next/components/data-connectors/https#using-oauth2-refresh-token-authentication):
+
+```yaml
+datasets:
+  - from: https://api.example.com
+    name: secure_data
+    params:
+      file_format: json
+      allowed_request_paths: '/v1/**'
+      auth_token_url: https://auth.example.com/oauth/token
+      http_auth_refresh_token: ${secrets:my_refresh_token}
+      http_auth_client_id: ${secrets:my_client_id}
+      http_auth_client_secret: ${secrets:my_client_secret}
+      auth_scopes: 'read:data offline_access'
+```
+
+### Databricks and Unity Catalog Reliability Improvements
+
+[Databricks](https://spiceai.org/docs/components/data-connectors/databricks) and [Unity Catalog](https://spiceai.org/docs/components/catalogs/unity-catalog) integrations are now more resilient and provide clearer behavior in permission-constrained environments.
+
+**Key improvements**:
+
+- **Resilience Controls**: Added controls to improve reliability when interacting with Databricks services.
+- **Unity Catalog Awareness**: Improved handling for Unity Catalog-specific behaviors and mixed deployment configurations.
+- **Permission Prechecks**: Databricks UC permission checks now distinguish explicit denials from ambiguous cases.
+- **Structured Error Reporting**: Advisory permission failures now surface with more actionable structured errors.
+- **Classic SQL Warehouse Compatibility**: Improved handling for foreign table scenarios with Classic SQL Warehouse combinations.
+- **Task History Instrumentation**: Added instrumentation to improve observability for Databricks-related operations.
+
+### Snowflake and ADBC Improvements
+
+[Snowflake](https://spiceai.org/docs/components/data-connectors/snowflake) and [ADBC](https://spiceai.org/docs/components/data-connectors/flightsql)-backed dataset registration is now faster and easier to observe.
+
+**Key improvements**:
+
+- **Faster Dataset Registration**: Improved registration performance for Snowflake and ADBC datasets.
+- **Better Observability**: Added better instrumentation and visibility into registration workflows.
+- **ADBC Alignment**: Updated ADBC dependencies and integration points for improved compatibility.
+- **Search Schema Fix**: Fixed a full-text search schema mismatch issue with the ADBC connector.
+
+### OpenTelemetry and Observability Improvements
+
+Spice improves telemetry export reliability and authenticated metrics delivery for the [OpenTelemetry](https://spiceai.org/docs/features/observability) integration.
+
+**Key improvements**:
+
+- **OTEL Exporter Fixes**: Fixed issues in the OpenTelemetry exporter.
+- **Authenticated Metrics Export**: Added support for authorization headers in the OTEL metrics exporter.
+- **Reduced Startup Noise**: Suppressed unnecessary AWS SDK noise and improved OTEL-related initialization behavior.
+- **Connector Initialization Reliability**: Fixed issues that could block connector initialization in telemetry-related code paths.
+
+### Dependency and Toolchain Updates
+
+| Dependency / Component  | Version / Update                      |
+| ----------------------- | ------------------------------------- |
+| **Rust toolchain**      | v1.94.1 (from v1.93.1)                |
+| **DataFusion**          | v52.5.0-rc1                           |
+| **mistral.rs / candle** | `mistral.rs` v0.8.x, `candle` v0.10.1 |
+| **ADBC Core**           | v0.23                                 |
+
+### Other Improvements
+
+- **Improved Query Pushdown**: Expanded sort and limit pushdown, including improved pushdown behavior for [Oracle](https://spiceai.org/docs/components/data-connectors/oracle) and [MSSQL](https://spiceai.org/docs/components/data-connectors/mssql) connectors.
+- **Partitioned Query Planning Improvements**: Improved `PartitionedTableScanRewrite` handling for `ORDER BY`, partition expressions, and fully qualified table references, while preventing incorrect bucketing partition pushdown to executors.
+- **MongoDB SRV Support**: Upgraded `datafusion-table-providers` with [MongoDB](https://spiceai.org/docs/components/data-connectors/mongodb) SRV support.
+- **Tantivy Logging**: [Search](https://spiceai.org/docs/features/search) logging now defaults to `warn` unless very verbose logging is enabled.
+- **Kafka Connector Fixes**: Improved reliability for the [Kafka](https://spiceai.org/docs/components/data-connectors/kafka) connector behavior.
+- **GitHub Connector Resilience**: Improved commit fetching for dynamic and slash refs, and reduced GraphQL page sizes on gateway errors for the [GitHub](https://spiceai.org/docs/components/data-connectors/github) connector.
+- **GitHub API Efficiency**: Lowered default comment fetch sizes to reduce pressure on [GitHub](https://spiceai.org/docs/components/data-connectors/github) GraphQL APIs.
+- **Embedding Validation**: Added validation for [embedding](https://spiceai.org/docs/components/embeddings) `row_id` columns during dataset initialization.
+- **View Cache Invalidation**: Cached plans are now cleared when [views](https://spiceai.org/docs/reference/spicepod/views) are updated.
+- **Refresh SQL Dedup Fix**: Fixed append [refresh](https://spiceai.org/docs/features/data-acceleration/data-refresh) deduplication when `refresh_sql` selects a subset of columns.
+
+## Contributors
+
+- [@ewgenius](https://github.com/ewgenius)
+- [@Jeadie](https://github.com/Jeadie)
+- [@krinart](https://github.com/krinart)
+- [@lukekim](https://github.com/lukekim)
+- [@peasee](https://github.com/peasee)
+- [@sgrebnov](https://github.com/sgrebnov)
+
+## Breaking Changes
+
+No breaking changes.
+
+## Cookbook Updates
+
+No new cookbook recipes.
+
+The [Spice Cookbook](https://spiceai.org/cookbook) includes 86 recipes to help you get started with Spice quickly and easily.
+
+## Upgrading
+
+To upgrade to v2.0.0-rc.3, use one of the following methods:
+
+**CLI**:
+
+```console
+spice upgrade v2.0.0-rc.3
+```
+
+**Homebrew**:
+
+```console
+brew upgrade spiceai/spiceai/spice
+```
+
+**Docker**:
+
+Pull the `spiceai/spiceai:2.0.0-rc.3` image:
+
+```console
+docker pull spiceai/spiceai:2.0.0-rc.3
+```
+
+For available tags, see [DockerHub](https://hub.docker.com/r/spiceai/spiceai/tags).
+
+**Helm**:
+
+```console
+helm repo update
+helm upgrade spiceai spiceai/spiceai --version 2.0.0-rc.3
+```
+
+**AWS Marketplace**:
+
+Spice is available in the [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-jmf6jskjvnq7i).
+
+## What's Changed
+
+### Changelog
+
+- fix: Full Text Search schema mismatch with ADBC connector by [@lukekim](https://github.com/lukekim) in [#10235](https://github.com/spiceai/spiceai/pull/10235)
+- docs: Update v2.0.0-rc.2 release notes with latest changes by [@lukekim](https://github.com/lukekim) in [#10238](https://github.com/spiceai/spiceai/pull/10238)
+- Fix append refresh dedup failure when refresh_sql selects column subset by [@sgrebnov](https://github.com/sgrebnov) in [#10225](https://github.com/spiceai/spiceai/pull/10225)
+- Revert "Properly mark dataset as Ready on Scheduler (#10215)" by [@sgrebnov](https://github.com/sgrebnov) in [#10242](https://github.com/spiceai/spiceai/pull/10242)
+- Fix failing merge conflicts for benchmarks by [@krinart](https://github.com/krinart) in [#10247](https://github.com/spiceai/spiceai/pull/10247)
+- fix(github): fetch commits for dynamic and slash refs by [@lukekim](https://github.com/lukekim) in [#10233](https://github.com/spiceai/spiceai/pull/10233)
+- Upgrade DataFusion to v52.5.0-rc1 by [@lukekim](https://github.com/lukekim) in [#10249](https://github.com/spiceai/spiceai/pull/10249)
+- Merge develop to trunk (2026-04-09) by [@claudespice](https://github.com/claudespice) in [#10248](https://github.com/spiceai/spiceai/pull/10248)
+- fix: Validate embedding row_id columns during dataset init (fixes #8226) by [@claudespice](https://github.com/claudespice) in [#10208](https://github.com/spiceai/spiceai/pull/10208)
+- fix: Update tpch benchmark snapshots for federated/glue[csv].yaml by [@app/github-actions](https://github.com/app/github-actions) in [#10244](https://github.com/spiceai/spiceai/pull/10244)
+- feat(databricks): add resilience controls, UC awareness, and task history instrumentation by [@lukekim](https://github.com/lukekim) in [#10246](https://github.com/spiceai/spiceai/pull/10246)
+- fix: Make PartitionManager resilient to bare vs fully qualified table references by [@sgrebnov](https://github.com/sgrebnov) in [#10257](https://github.com/spiceai/spiceai/pull/10257)
+- fix: Update tpch benchmark snapshots for accelerated/s3[parquet]-cayenne[file].yaml by [@app/github-actions](https://github.com/app/github-actions) in [#10256](https://github.com/spiceai/spiceai/pull/10256)
+- Merge develop to trunk (2026-04-10) by [@claudespice](https://github.com/claudespice) in [#10251](https://github.com/spiceai/spiceai/pull/10251)
+- Improve Snowflake/ADBC dataset registration performance and observability by [@lukekim](https://github.com/lukekim) in [#10266](https://github.com/spiceai/spiceai/pull/10266)
+- Fixes for kafka connector by [@krinart](https://github.com/krinart) in [#10263](https://github.com/spiceai/spiceai/pull/10263)
+- fix(runtime): gate otel code tags, suppress aws sdk noise, and unblock connector init by [@lukekim](https://github.com/lukekim) in [#10260](https://github.com/spiceai/spiceai/pull/10260)
+- fix(runtime): avoid regionless AWS SDK loads by [@lukekim](https://github.com/lukekim) in [#10271](https://github.com/spiceai/spiceai/pull/10271)
+- Add versioned release install workflow coverage by [@lukekim](https://github.com/lukekim) in [#10276](https://github.com/spiceai/spiceai/pull/10276)
+- fix(runtime): handle HTTP JSON unions and spicepod reloads by [@lukekim](https://github.com/lukekim) in [#10277](https://github.com/spiceai/spiceai/pull/10277)
+- Databricks UC permission prechecks: explicit denial as permanent error, ambiguous cases advisory by [@lukekim](https://github.com/lukekim) in [#10274](https://github.com/spiceai/spiceai/pull/10274)
+- Revert component status changes re-introduced by develop merge (#10248) by [@sgrebnov](https://github.com/sgrebnov) in [#10293](https://github.com/spiceai/spiceai/pull/10293)
+- Fix broken CI workflows by [@ewgenius](https://github.com/ewgenius) in [#10294](https://github.com/spiceai/spiceai/pull/10294)
+- Group dependabot updates by ecosystem by [@lukekim](https://github.com/lukekim) in [#10296](https://github.com/spiceai/spiceai/pull/10296)
+- fix(tests): Replace flaky S3 Vectors snapshot tests with structural validation by [@lukekim](https://github.com/lukekim) in [#10301](https://github.com/spiceai/spiceai/pull/10301)
+- Update test_github_workflows snapshot by [@lukekim](https://github.com/lukekim) in [#10304](https://github.com/spiceai/spiceai/pull/10304)
+- fix(ci): fix Bedrock runner mismatch and snapshot auto-merge failure by [@ewgenius](https://github.com/ewgenius) in [#10306](https://github.com/spiceai/spiceai/pull/10306)
+- feat(http): Add map-to-array conversion and query-parameter pagination by [@lukekim](https://github.com/lukekim) in [#10295](https://github.com/spiceai/spiceai/pull/10295)
+- New crate: `datafusion-ddl` by [@Jeadie](https://github.com/Jeadie) in [#10205](https://github.com/spiceai/spiceai/pull/10205)
+- Make Databricks UC permission checks advisory with structured error reporting by [@lukekim](https://github.com/lukekim) in [#10283](https://github.com/spiceai/spiceai/pull/10283)
+- build(deps): bump the github-actions-dependencies group with 4 updates by [@app/dependabot](https://github.com/app/dependabot) in [#10298](https://github.com/spiceai/spiceai/pull/10298)
+- fix: Clear cached plans on view updates by [@peasee](https://github.com/peasee) in [#10312](https://github.com/spiceai/spiceai/pull/10312)
+- build(deps): bump the aws-sdk group with 7 updates by [@app/dependabot](https://github.com/app/dependabot) in [#10299](https://github.com/spiceai/spiceai/pull/10299)
+- Code out of runtime. by [@Jeadie](https://github.com/Jeadie) in [#10178](https://github.com/spiceai/spiceai/pull/10178)
+- fix: Respect function registry denies for accelerated table filter pushdown by [@peasee](https://github.com/peasee) in [#10311](https://github.com/spiceai/spiceai/pull/10311)
+- fix: Don't block heartbeat when all slots acquired by [@peasee](https://github.com/peasee) in [#10322](https://github.com/spiceai/spiceai/pull/10322)
+- fix: strip only outer parens in `get_table_partition_expr_from_ctx` by [@Jeadie](https://github.com/Jeadie) in [#10323](https://github.com/spiceai/spiceai/pull/10323)
+- Upgrade datafusion-table-providers with MongoDB SRV support by [@lukekim](https://github.com/lukekim) in [#10317](https://github.com/spiceai/spiceai/pull/10317)
+- fix: Avoid pushing down bucketing partition expressions into executors by [@peasee](https://github.com/peasee) in [#10324](https://github.com/spiceai/spiceai/pull/10324)
+- Upgrade datafusion-table-providers to d1b911a5 and bump adbc to 0.23 by [@lukekim](https://github.com/lukekim) in [#10329](https://github.com/spiceai/spiceai/pull/10329)
+- fix: Update Search integration test snapshots by [@app/github-actions](https://github.com/app/github-actions) in [#10308](https://github.com/spiceai/spiceai/pull/10308)
+- Handle foreign table + Classic sql warehouse combination gracefully by [@krinart](https://github.com/krinart) in [#10318](https://github.com/spiceai/spiceai/pull/10318)
+- New crate `datafusion-flightsql` by [@Jeadie](https://github.com/Jeadie) in [#10201](https://github.com/spiceai/spiceai/pull/10201)
+- Set `tantivy=warn` unless very verbose logging by [@Jeadie](https://github.com/Jeadie) in [#10338](https://github.com/spiceai/spiceai/pull/10338)
+- Remove image registry and image name options from spidapter by [@ewgenius](https://github.com/ewgenius) in [#10241](https://github.com/spiceai/spiceai/pull/10241)
+- build(deps): bump sysinfo from 0.37.2 to 0.38.4 by [@app/dependabot](https://github.com/app/dependabot) in [#10291](https://github.com/spiceai/spiceai/pull/10291)
+- build(deps): bump futures from 0.3.31 to 0.3.32 by [@app/dependabot](https://github.com/app/dependabot) in [#10289](https://github.com/spiceai/spiceai/pull/10289)
+- New crate 'datafusion-dml' by [@Jeadie](https://github.com/Jeadie) in [#10334](https://github.com/spiceai/spiceai/pull/10334)
+- Jeadie/26 04 16/spice sql by [@Jeadie](https://github.com/Jeadie) in [#10343](https://github.com/spiceai/spiceai/pull/10343)
+- Add Teraswitch/Pittsburgh apt mirrors + retry config for CI runners by [@lukekim](https://github.com/lukekim) in [#10349](https://github.com/spiceai/spiceai/pull/10349)
+- Implement sort pushdown and fix pushdown gaps across providers by [@lukekim](https://github.com/lukekim) in [#10337](https://github.com/spiceai/spiceai/pull/10337)
+- Merge develop to trunk (2026-04-16) by [@claudespice](https://github.com/claudespice) in [#10345](https://github.com/spiceai/spiceai/pull/10345)
+- Update candle and mistral.rs lock-step pins by [@lukekim](https://github.com/lukekim) in [#10278](https://github.com/spiceai/spiceai/pull/10278)
+- docs: fix status badges in README by [@lukekim](https://github.com/lukekim) in [#10350](https://github.com/spiceai/spiceai/pull/10350)
+- Migrate secrets to vars by [@krinart](https://github.com/krinart) in [#10354](https://github.com/spiceai/spiceai/pull/10354)
+- Add limit pushdown and improve sort pushdown for Oracle and MSSQL by [@sgrebnov](https://github.com/sgrebnov) in [#10351](https://github.com/spiceai/spiceai/pull/10351)
+- Fix ubuntu mirror configuration by [@ewgenius](https://github.com/ewgenius) in [#10359](https://github.com/spiceai/spiceai/pull/10359)
+- fix: Increase throughput test default ready_wait from 30s to 300s (fixes #8207) by [@claudespice](https://github.com/claudespice) in [#10344](https://github.com/spiceai/spiceai/pull/10344)
+- Add auth headers support to OTEL metrics exporter by [@lukekim](https://github.com/lukekim) in [#10347](https://github.com/spiceai/spiceai/pull/10347)
+- fix(github): shrink GraphQL page size on gateway errors; lower comment defaults by [@lukekim](https://github.com/lukekim) in [#10355](https://github.com/spiceai/spiceai/pull/10355)
+- Relax apt mirror substitution failure to warning in CI action by [@ewgenius](https://github.com/ewgenius) in [#10361](https://github.com/spiceai/spiceai/pull/10361)
+- feat(http): Add OAuth2 refresh-token auth to HTTP connector by [@lukekim](https://github.com/lukekim) in [#10348](https://github.com/spiceai/spiceai/pull/10348)
+- Upgrade Rust toolchain to 1.94.1 by [@lukekim](https://github.com/lukekim) in [#10353](https://github.com/spiceai/spiceai/pull/10353)
+- Handle order by and sort in PartitionedTableScanRewrite by [@Jeadie](https://github.com/Jeadie) in [#9656](https://github.com/spiceai/spiceai/pull/9656)
+- Fix OTEL Exporter by [@krinart](https://github.com/krinart) in [#10363](https://github.com/spiceai/spiceai/pull/10363)
+- Pin spiceai candle / TEI forks to merged revs; drop local [patch] overrides by [@lukekim](https://github.com/lukekim) in [#10362](https://github.com/spiceai/spiceai/pull/10362)
+
+**Full Changelog**: [https://github.com/spiceai/spiceai/compare/v2.0.0-rc.2...v2.0.0-rc.3](https://github.com/spiceai/spiceai/compare/v2.0.0-rc.2...v2.0.0-rc.3)