Update dbt-to-datashare docs for S3 datashare

[TheEdgeOfRage]

No description provided.

[cursor]

PR Summary

Medium Risk
Prod post-hooks now run an extra metadata query and can flip incremental datashare runs to full refresh, affecting transfer cost and sync behavior after cleanup.

Overview
Documents S3 Export and multi-target datashare, and hardens incremental syncs when a destination was removed while the dbt model still exists.

The dune-datashares guide now treats destinations as Snowflake, BigQuery, and S3 Export (Iceberg in a Dune-managed bucket, IAM role or account access). It adds Selecting a target (target_type / target_region required when multiple targets exist), S3 setup and query notes, cross-region MERGE cost on S3, generated SQL with target args, and target-scoped delete_datashare.

The post-hook macro gains _datashare_active_sync_exists, which queries dune.datashare.table_syncs (optionally filtered by target). If an incremental run would sync but no active registration exists—e.g. after revoke—it forces full_refresh instead of a MERGE into a missing destination; probe failures default to assuming a sync exists to avoid surprise full refreshes.

^{Reviewed by Cursor Bugbot for commit 4913c99. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Partial target breaks sync probe
  • I fixed this by enforcing that meta.datashare.target_type and meta.datashare.target_region must be configured together, preventing partial target scoping from incorrectly skipping full refresh.

Or push these changes by commenting:

@cursor push 1b8e134555

Preview (1b8e134555)

diff --git a/docs/dune-datashares.md b/docs/dune-datashares.md
--- a/docs/dune-datashares.md
+++ b/docs/dune-datashares.md
@@ -1,6 +1,6 @@
 # Dune Datashares
 
-Datashares sync Dune tables to external data warehouses such as Snowflake and BigQuery so downstream consumers can query the data outside Dune.
+Datashares sync your Dune tables to external destinations such as Snowflake, BigQuery, and S3 Export, so you and your consumers can query the data outside Dune.
 
 ## Prerequisites
 
@@ -90,6 +90,47 @@
 
 Keep the sync window aligned with the `time_column` granularity. For example, if `time_column` is a `date`, use date-based expressions like `current_date - interval '1' day`, not hour-based timestamp windows.
 
+## Selecting a target
+
+If your team has a single datashare target set up, you do not need to set `target_type`/`target_region` — your data syncs to that target automatically.
+
+If your team has **multiple** targets set up (for example both Snowflake and S3 Export), choose which one to sync to by setting both `target_type` and `target_region` in `meta.datashare`. Without them, the sync cannot tell which destination you mean and will return an error:
+
+```sql
+{{ config(
+    materialized = 'incremental',
+    incremental_strategy = 'merge',
+    unique_key = ['block_number', 'block_date'],
+    meta = {
+        "datashare": {
+            "enabled": true,
+            "time_column": "block_date",
+            "time_start": time_start,
+            "time_start_incremental": time_start_incremental,
+            "time_end": time_end,
+            "target_type": "s3",
+            "target_region": "eu-west-1"
+        }
+    }
+) }}
+```
+
+Always set `target_type` and `target_region` together. Use the region your target was set up with:
+
+- Warehouse targets (Snowflake, BigQuery): `us` or `eu`.
+- S3 Export: the AWS region of your bucket, e.g. `eu-west-1`, `us-east-1`.
+
+Each table can have one active sync per target. If a table is already syncing to one target and you want to send it somewhere else, either point at the existing target or publish it under a different model `alias`.
+
+### S3 Export
+
+S3 Export delivers your data as an Iceberg table in a Dune-managed S3 bucket. Dune adds a bucket policy to your export bucket that grants an AWS principal you control read access. You can either:
+
+- create an IAM role in your own AWS account with read access to S3 and give Dune that role's ARN, so only that role can read the bucket; or
+- give Dune just your AWS account ID, and Dune grants the whole account access. You can then control which IAM users/roles have access to the S3 bucket by setting the appropriate IAM policy permissions.
+
+You query the data directly from S3 with your own engine — Athena, Spark, DuckDB, or your own Trino — using that principal, without going through Dune. S3 Export currently supports the **Iceberg** table format. To set up an S3 target, contact Dune with the bucket region you want, the table format, and either the IAM role ARN or the AWS account ID to grant read access.
+
 ## Cadence and sync windows
 
 The `time_start_incremental` → `time_end` window and your dbt **run cadence** are not independent knobs. Every incremental sync issues a `MERGE INTO` against the destination table, which re-reads the destination data covered by that window. On **S3 Export** targets where the destination bucket is in a different region from Trino, each run pays cross-region transfer for the entire window.
@@ -102,11 +143,11 @@
 
 Examples:
 
-| Cadence | `time_column` | Incremental window         | Multiplier | Notes |
-| ------- | ------------- | -------------------------- | ---------- | ----- |
-| Daily   | `date`        | `interval '1' day`         | 1x         | Safe default. The included example model uses this shape. |
-| Hourly  | `timestamp`   | `interval '2' hour`        | 2x         | Use only when `time_column` is timestamp-granular and the destination is partitioned/prunable on it. |
-| Hourly  | `date`        | `interval '1' day`         | 24x        | **Cost trap.** Every hourly run re-reads the full day's partition from the destination. |
+| Cadence | `time_column` | Incremental window  | Multiplier | Notes                                                                                                |
+| ------- | ------------- | ------------------- | ---------- | ---------------------------------------------------------------------------------------------------- |
+| Daily   | `date`        | `interval '1' day`  | 1x         | Safe default. The included example model uses this shape.                                            |
+| Hourly  | `timestamp`   | `interval '2' hour` | 2x         | Use only when `time_column` is timestamp-granular and the destination is partitioned/prunable on it. |
+| Hourly  | `date`        | `interval '1' day`  | 24x        | **Cost trap.** Every hourly run re-reads the full day's partition from the destination.              |
 
 Rules of thumb:
 
@@ -143,12 +184,12 @@
 
 The macro determines `full_refresh` automatically:
 
-| Context | `full_refresh` |
-| --- | --- |
-| Incremental post-hook on a normal incremental run | `false` |
-| Incremental post-hook on first run or `--full-refresh` | `true` |
-| Table materialization post-hook | `true` |
-| `run-operation` | `false` unless overridden |
+| Context                                                | `full_refresh`            |
+| ------------------------------------------------------ | ------------------------- |
+| Incremental post-hook on a normal incremental run      | `false`                   |
+| Incremental post-hook on first run or `--full-refresh` | `true`                    |
+| Table materialization post-hook                        | `true`                    |
+| `run-operation`                                        | `false` unless overridden |
 
 ## Generated SQL
 
@@ -164,6 +205,20 @@
 )

+When target_type/target_region are set in meta.datashare, the macro appends them:
+
+```sql
+ALTER TABLE dune..

EXECUTE datashare(

• time_column => '<column_name>',
• unique_key_columns => ARRAY['col1', 'col2'],
• time_start => CAST(<sql_expression> AS VARCHAR),
• time_end => CAST(<sql_expression> AS VARCHAR),
• full_refresh => true|false
• , target_type => 's3'
• , target_region => 'eu-west-1'
  +)
  +```

Manual Syncs

Use run-operation when you want to trigger a sync outside dbt run.
@@ -225,6 +280,16 @@
ALTER TABLE dune..

EXECUTE delete_datashare

+If the table has datashares to **multiple** targets, identify which one to remove by passing both arguments:
+
+```sql
+ALTER TABLE dune.<schema>.<table> EXECUTE delete_datashare(
+    target_type => 's3', target_region => 'eu-west-1'
+)
+```
+
+Cleanup stops the sync and revokes access to the destination — for S3 Export, your IAM role can no longer read the bucket.
+
## Example Workflow

1. Configure a model with `meta.datashare`.

diff --git a/macros/dune_dbt_overrides/datashare_table_sync_post_hook.sql b/macros/dune_dbt_overrides/datashare_table_sync_post_hook.sql
--- a/macros/dune_dbt_overrides/datashare_table_sync_post_hook.sql
+++ b/macros/dune_dbt_overrides/datashare_table_sync_post_hook.sql
@@ -20,6 +20,45 @@
{%- endmacro -%}

{#
+    Returns true if an active (non-deleted) datashare sync is registered for this
+    table. When target_type/target_region are provided, the check is scoped to
+    that target so revoking one target of a multi-target table is detected.
+
+    Used to force a full refresh when the source table already exists (so dbt
+    builds incrementally) but the destination sync was revoked: an incremental
+    MERGE would otherwise target a destination that no longer exists.
+
+    On probe failure (e.g. cluster cold), returns true so the caller keeps its
+    existing is_incremental() behavior rather than forcing an unwanted refresh.
+#}
+{% macro _datashare_active_sync_exists(schema_name, table_name, target_type=None, target_region=None) %}
+    {%- if not execute -%}
+        {{ return(true) }}
+    {%- endif -%}
+    {%- set where = [] -%}
+    {%- do where.append('source_schema = ' ~ _datashare_sql_string(schema_name)) -%}
+    {%- do where.append('source_table = ' ~ _datashare_sql_string(table_name)) -%}
+    {%- do where.append('deleted_at IS NULL') -%}
+    {%- if target_type is not none and target_type | string | trim != '' -%}
+        {%- do where.append('target_type = ' ~ _datashare_sql_string(target_type)) -%}
+    {%- endif -%}
+    {%- if target_region is not none and target_region | string | trim != '' -%}
+        {%- do where.append('target_region = ' ~ _datashare_sql_string(target_region)) -%}
+    {%- endif -%}
+    {%- set probe_sql = 'SELECT count(*) AS c FROM dune.datashare.table_syncs WHERE ' ~ (where | join(' AND ')) -%}
+    {%- set result = none -%}
+    {%- set probe = run_query(probe_sql) -%}
+    {%- if probe is not none and probe.columns | length > 0 and probe.columns[0].values() | length > 0 -%}
+        {%- set result = probe.columns[0].values()[0] -%}
+    {%- endif -%}
+    {%- if result is none -%}
+        {{ log('datashare sync probe for ' ~ schema_name ~ '.' ~ table_name ~ ' returned no rows; assuming sync exists.', info=True) }}
+        {{ return(true) }}
+    {%- endif -%}
+    {{ return(result | int > 0) }}
+{%- endmacro -%}
+
+{#
    Datashare sync macro - generates ALTER TABLE ... EXECUTE datashare() SQL.
    Config reference and usage: docs/dune-datashares.md
#}
@@ -55,7 +94,19 @@
    {%- set target_region = datashare.get('target_region') -%}
    {%- set include_target_type = target_type is not none and target_type | string | trim != '' -%}
    {%- set include_target_region = target_region is not none and target_region | string | trim != '' -%}
+    {%- if include_target_type != include_target_region -%}
+        {{ exceptions.raise_compiler_error('Invalid datashare target config for ' ~ model_ref ~ ': set both meta.datashare.target_type and meta.datashare.target_region, or omit both.') }}
+    {%- endif -%}

+    {#- An incremental sync targets an existing destination via MERGE. If the
+        destination sync was revoked while the source table still exists, dbt
+        builds incrementally but there is nothing to merge into. Force a full
+        refresh when no active sync is registered for this table/target. -#}
+    {%- if not full_refresh and not _datashare_active_sync_exists(schema_name, table_name, target_type, target_region) -%}
+        {{ log('No active datashare sync for ' ~ model_ref ~ '; forcing full_refresh.', info=True) }}
+        {%- set full_refresh = true -%}
+    {%- endif -%}
+
    {%- set sql -%}
ALTER TABLE {{ catalog_name }}.{{ schema_name }}.{{ table_name }} EXECUTE datashare(
    time_column => {{ _datashare_sql_string(time_column | default('', true)) }},

_{You can send follow-ups to the cloud agent here.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 4913c99. Configure here.}