Skip to content

Commit 480af4e

Browse files
bertilxiclaudeMaxteabag
authored
refactor(turso): migrate from libsql-client to libsql package (#37)
* refactor(turso): migrate from libsql-client to libsql package libsql-client is deprecated; switch to the new libsql package with updated API patterns (connect instead of create_client_sync, cursor-based query execution with fetchall/description instead of result.rows). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix(turso): use direct HTTP mode and add commit() for write persistence The libsql package has two connection modes: 1. Embedded replica mode (local file + sync_url) - for read-heavy caching 2. Direct HTTP mode (url only) - for immediate read/write operations The previous implementation used embedded replica mode which: - Writes to a local temp file that gets deleted on close - sync() only pulls FROM server, doesn't push writes back - Missing commit() meant writes weren't persisted This fix: - Uses direct HTTP mode: libsql.connect(url, auth_token=token) - Adds conn.commit() in execute_non_query() like all other adapters - Converts libsql:// URLs to https:// as required by the package Tested against both Turso Cloud and local Docker (libsql-server). * fix(tests): update Turso fixtures to use direct HTTP mode Update test fixtures to match the adapter's direct HTTP mode: - _create_turso_connection() now uses libsql.connect(url, auth_token=token) - _setup_turso_test_tables() uses commit() instead of sync() - _cleanup_turso_test_tables() uses commit() instead of sync() This ensures test setup/teardown works consistently with the adapter for both Turso Cloud and local Docker environments. * test(turso): add write persistence test and fix cloud mode handling - Add test_write_persistence_across_connections() to verify writes persist to the remote server across separate connections - Fix test_create_turso_connection() to handle cloud mode (tuple) - Fix test_delete_turso_connection() to handle cloud mode (tuple) - Add test_expand_tables_folder(), test_expand_table_node(), and test_expand_views_folder() to test tree expansion operations The persistence test catches the bug where embedded replica mode doesn't actually persist writes to the remote server. * chore: add .env.example and protect secrets in tests/.env - Add tests/.env to .gitignore to prevent accidental commit of secrets - Add tests/.env.example as template for Turso Cloud configuration To test against Turso Cloud instead of local Docker: 1. Copy tests/.env.example to tests/.env 2. Fill in your Turso Cloud URL and auth token 3. Run: pytest tests/test_turso.py -v --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Peter Adams <18162810+Maxteabag@users.noreply.github.com>
1 parent dbc42e4 commit 480af4e

9 files changed

Lines changed: 391 additions & 1086 deletions

File tree

.github/workflows/ci.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -576,7 +576,7 @@ jobs:
576576
run: |
577577
python -m pip install --upgrade pip
578578
pip install -e ".[test]"
579-
pip install libsql-client
579+
pip install libsql
580580
581581
- name: Start Turso (libsql-server)
582582
run: |

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,3 +57,6 @@ assets/favorites/*
5757
!assets/favorites/logo_sqlit.png
5858
.worktrees/
5959
.obsidian/
60+
61+
# Test environment files with secrets
62+
tests/.env

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -245,7 +245,7 @@ Most of the time you can just run `sqlit` and connect. If a Python driver is mis
245245
| Oracle | `oracledb` | `pipx inject sqlit-tui oracledb` | `python -m pip install oracledb` |
246246
| DuckDB | `duckdb` | `pipx inject sqlit-tui duckdb` | `python -m pip install duckdb` |
247247
| ClickHouse | `clickhouse-connect` | `pipx inject sqlit-tui clickhouse-connect` | `python -m pip install clickhouse-connect` |
248-
| Turso | `libsql-client` | `pipx inject sqlit-tui libsql-client` | `python -m pip install libsql-client` |
248+
| Turso | `libsql` | `pipx inject sqlit-tui libsql` | `python -m pip install libsql` |
249249
| Cloudflare D1 | `requests` | `pipx inject sqlit-tui requests` | `python -m pip install requests` |
250250
| Snowflake | `snowflake-connector-python` | `pipx inject sqlit-tui snowflake-connector-python` | `python -m pip install snowflake-connector-python` |
251251
| Firebird | `firebirdsql` | `pipx inject sqlit-tui firebirdsql` | `python -m pip install firebirdsql` |

pyproject.toml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ all = [
4444
"duckdb>=1.1.0", # min avoids known CVEs
4545
"clickhouse-connect>=0.7.0",
4646
"requests>=2.32.4", # min avoids known CVEs
47-
"libsql-client>=0.1.0",
47+
"libsql>=0.1.0",
4848
"firebirdsql>=1.3.5",
4949
"sshtunnel>=0.4.0",
5050
"paramiko>=2.0.0,<4.0.0",
@@ -59,7 +59,7 @@ oracle = ["oracledb>=2.0.0"]
5959
duckdb = ["duckdb>=1.1.0"] # min avoids known CVEs
6060
clickhouse = ["clickhouse-connect>=0.7.0"]
6161
d1 = ["requests>=2.32.4"] # min avoids known CVEs
62-
turso = ["libsql-client>=0.1.0"]
62+
turso = ["libsql>=0.1.0"]
6363
firebird = ["firebirdsql>=1.3.5"]
6464
snowflake = ["snowflake-connector-python>=3.7.0"]
6565
ssh = [
@@ -158,7 +158,7 @@ module = [
158158
"oracledb",
159159
"duckdb",
160160
"clickhouse_connect",
161-
"libsql_client",
161+
"libsql",
162162
"snowflake.connector",
163163
"sshtunnel",
164164
"docker",

sqlit/db/adapters/turso.py

Lines changed: 40 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Turso adapter using libsql-client."""
1+
"""Turso adapter using libsql."""
22

33
from __future__ import annotations
44

@@ -13,7 +13,7 @@
1313
class TursoAdapter(DatabaseAdapter):
1414
"""Adapter for Turso (libSQL) databases.
1515
16-
Turso is a distributed SQLite-compatible database. Uses the libsql_client
16+
Turso is a distributed SQLite-compatible database. Uses the libsql
1717
package for connections via HTTP/HTTPS URLs with optional token authentication.
1818
"""
1919

@@ -67,11 +67,11 @@ def install_extra(self) -> str:
6767

6868
@property
6969
def install_package(self) -> str:
70-
return "libsql-client"
70+
return "libsql"
7171

7272
@property
7373
def driver_import_names(self) -> tuple[str, ...]:
74-
return ("libsql_client",)
74+
return ("libsql",)
7575

7676
@property
7777
def supports_multiple_databases(self) -> bool:
@@ -92,69 +92,71 @@ def connect(self, config: ConnectionConfig) -> Any:
9292
"""Connect to Turso database.
9393
9494
Uses config.server for the database URL and config.password for the auth token.
95-
Supports libsql://, https://, and http:// URLs.
95+
Accepts libsql://, https://, and http:// URLs (libsql:// is converted to https://).
96+
Uses direct HTTP mode for immediate read/write operations.
9697
"""
97-
libsql_client = import_driver_module(
98-
"libsql_client",
98+
libsql = import_driver_module(
99+
"libsql",
99100
driver_name=self.name,
100101
extra_name=self.install_extra,
101102
package_name=self.install_package,
102103
)
103104

104105
url = config.server
105-
# Ensure URL has proper scheme
106-
if not url.startswith(("libsql://", "https://", "http://")):
107-
url = f"libsql://{url}"
106+
# Convert URL scheme (libsql package requires http:// or https://)
107+
if url.startswith("libsql://"):
108+
url = url.replace("libsql://", "https://", 1)
109+
elif not url.startswith(("https://", "http://")):
110+
url = f"https://{url}"
108111

109-
auth_token = config.password if config.password else None
110-
client = libsql_client.create_client_sync(url, auth_token=auth_token)
111-
return client
112+
auth_token = config.password if config.password else ""
113+
return libsql.connect(url, auth_token=auth_token)
112114

113115
def get_databases(self, conn: Any) -> list[str]:
114116
"""Turso doesn't support multiple databases - return empty list."""
115117
return []
116118

117119
def get_tables(self, conn: Any, database: str | None = None) -> list[TableInfo]:
118120
"""Get list of tables from Turso. Returns (schema, name) with empty schema."""
119-
result = conn.execute(
121+
rows = conn.execute(
120122
"SELECT name FROM sqlite_master WHERE type='table' "
121123
"AND name NOT LIKE 'sqlite_%' AND name NOT LIKE '_litestream_%' "
122124
"ORDER BY name"
123-
)
124-
return [("", row[0]) for row in result.rows]
125+
).fetchall()
126+
return [("", row[0]) for row in rows]
125127

126128
def get_views(self, conn: Any, database: str | None = None) -> list[TableInfo]:
127129
"""Get list of views from Turso. Returns (schema, name) with empty schema."""
128-
result = conn.execute("SELECT name FROM sqlite_master WHERE type='view' ORDER BY name")
129-
return [("", row[0]) for row in result.rows]
130+
rows = conn.execute("SELECT name FROM sqlite_master WHERE type='view' ORDER BY name").fetchall()
131+
return [("", row[0]) for row in rows]
130132

131133
def get_columns(
132134
self, conn: Any, table: str, database: str | None = None, schema: str | None = None
133135
) -> list[ColumnInfo]:
134136
"""Get columns for a table from Turso. Schema parameter is ignored."""
135137
quoted_table = self.quote_identifier(table)
136-
result = conn.execute(f"PRAGMA table_info({quoted_table})")
138+
rows = conn.execute(f"PRAGMA table_info({quoted_table})").fetchall()
137139
# PRAGMA table_info returns: cid, name, type, notnull, dflt_value, pk
138140
# pk > 0 indicates column is part of primary key
139-
return [ColumnInfo(name=row[1], data_type=row[2] or "TEXT", is_primary_key=row[5] > 0) for row in result.rows]
141+
return [ColumnInfo(name=row[1], data_type=row[2] or "TEXT", is_primary_key=row[5] > 0) for row in rows]
140142

141143
def get_procedures(self, conn: Any, database: str | None = None) -> list[str]:
142144
"""Turso doesn't support stored procedures - return empty list."""
143145
return []
144146

145147
def get_indexes(self, conn: Any, database: str | None = None) -> list[IndexInfo]:
146148
"""Get indexes from Turso (SQLite-compatible)."""
147-
result = conn.execute(
149+
rows = conn.execute(
148150
"SELECT name, tbl_name FROM sqlite_master "
149151
"WHERE type='index' AND name NOT LIKE 'sqlite_%' "
150152
"ORDER BY tbl_name, name"
151-
)
153+
).fetchall()
152154
results = []
153-
for row in result.rows:
155+
for row in rows:
154156
# Check if index is unique using PRAGMA
155-
idx_result = conn.execute(f"PRAGMA index_list({self.quote_identifier(row[1])})")
157+
idx_result = conn.execute(f"PRAGMA index_list({self.quote_identifier(row[1])})").fetchall()
156158
is_unique = False
157-
for idx_info in idx_result.rows:
159+
for idx_info in idx_result:
158160
if idx_info[1] == row[0]: # idx_info: seq, name, unique, origin, partial
159161
is_unique = idx_info[2] == 1
160162
break
@@ -163,12 +165,12 @@ def get_indexes(self, conn: Any, database: str | None = None) -> list[IndexInfo]
163165

164166
def get_triggers(self, conn: Any, database: str | None = None) -> list[TriggerInfo]:
165167
"""Get triggers from Turso (SQLite-compatible)."""
166-
result = conn.execute(
168+
rows = conn.execute(
167169
"SELECT name, tbl_name FROM sqlite_master "
168170
"WHERE type='trigger' "
169171
"ORDER BY tbl_name, name"
170-
)
171-
return [TriggerInfo(name=row[0], table_name=row[1]) for row in result.rows]
172+
).fetchall()
173+
return [TriggerInfo(name=row[0], table_name=row[1]) for row in rows]
172174

173175
def get_sequences(self, conn: Any, database: str | None = None) -> list[SequenceInfo]:
174176
"""Turso/SQLite doesn't support sequences - return empty list."""
@@ -188,16 +190,20 @@ def build_select_query(self, table: str, limit: int, database: str | None = None
188190

189191
def execute_query(self, conn: Any, query: str, max_rows: int | None = None) -> tuple[list[str], list[tuple], bool]:
190192
"""Execute a query on Turso with optional row limit."""
191-
result = conn.execute(query)
192-
if result.columns:
193-
columns = list(result.columns)
194-
rows = [tuple(row) for row in result.rows]
193+
cur = conn.cursor()
194+
rows = cur.execute(query).fetchall()
195+
columns = [col[0] for col in cur.description]
196+
if columns:
197+
rows = [tuple(row) for row in rows]
195198
if max_rows is not None and len(rows) > max_rows:
196199
return columns, rows[:max_rows], True
197200
return columns, rows, False
198201
return [], [], False
199202

200203
def execute_non_query(self, conn: Any, query: str) -> int:
201204
"""Execute a non-query on Turso."""
202-
result = conn.execute(query)
203-
return int(result.rows_affected or 0)
205+
cur = conn.cursor()
206+
cur.execute(query)
207+
rowcount = int(cur.rowcount or 0)
208+
conn.commit()
209+
return rowcount

tests/.env.example

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Turso Test Configuration
2+
# Copy this file to .env and fill in your values
3+
#
4+
# To run tests against Turso Cloud instead of local Docker:
5+
# 1. Copy this file to tests/.env
6+
# 2. Fill in your Turso Cloud URL and auth token
7+
# 3. Run: pytest tests/test_turso.py -v
8+
#
9+
# To run tests against local Docker (default):
10+
# 1. Start libsql-server: docker run -d --name turso -p 8080:8080 ghcr.io/tursodatabase/libsql-server:latest
11+
# 2. Run: TURSO_PORT=8080 pytest tests/test_turso.py -v
12+
13+
# Turso Cloud settings (uncomment and fill in to use cloud instead of Docker)
14+
# TURSO_CLOUD_URL=libsql://your-database.turso.io
15+
# TURSO_CLOUD_AUTH_TOKEN=your-auth-token-here
16+
17+
# Local Docker settings (defaults shown)
18+
# TURSO_HOST=localhost
19+
# TURSO_PORT=8081

0 commit comments

Comments
 (0)