Connecting to a legacy ducklake catalog with an incompatible duckdb extension version fails when it doesn't need to

[TomBurdge]

dlt version

1.27.0

Describe the problem

When attaching a v0.4 ducklake catalog with a v1.x version of the duckb ducklake extension with dlt, there is the following error: _duckdb.InvalidInputException: Invalid Input Error: DuckLake catalog version mismatch: catalog version is 0.4, but the extension requires version 1.0. To automatically migrate, set AUTOMATIC_MIGRATION to TRUE when attaching.

Expected behavior

I don't think there is a way to handle this currently within dlthub, but this doesn't need to fail. The ducklake docs and the error message are quite useful on this. You need to attach with the AUTOMATIC_MIGRATION flag as true: ATTACH '...' (AUTOMATIC_MIGRATION);

Proposed Solution: I think this can be solved by following the ducklake docs: add a flag to the destination which enables AUTOMATIC_MIGRATION to be True. I have implemented this in #3980

Steps to reproduce

• Attach to a ducklake source (in my case with a postgresql catalog db) with duckdb version <v1.5. (will create a <v0.4 version ducklake catalog - see the ducklake release calendar for reference ) and write any destination (you need to trigger a commit).
• Run the same pipeline again with duckdb >=v1.5.2. Currently, you will see: _duckdb.InvalidInputException: Invalid Input Error: DuckLake catalog version mismatch: catalog version is 0.4, but the extension requires version 1.0. To automatically migrate, set AUTOMATIC_MIGRATION to TRUE when attaching.

I'm happy to write a couple of scripts and a Dockerfile for more details to repro, just let me know. :)

Operating system

Linux

Runtime environment

Kubernetes

Python version

3.13

dlt data source

I don't think this makes a difference here as the failure is on attach, but in this case an API source that yields json objects.

dlt destination

DuckDB

Other deployment details

No response

Additional information

No response