Tusker rewrite in Rust

[bikeshedder]

This project is discontinued in favor of the Rust rewrite that just got released:

📦 https://crates.io/crates/tusker
https://github.com/bikeshedder/tusker/

---

Original post:

I'm currently in process of rewriting tusker in Rust.

Why?

1. I love writing Rust code.
2. I would prefer if I could just call a statically linked binary instead of having to bring an entire Python runtime when using Tusker in my devcontainer or deployment servers.
3. migra and schemainspect have some unsolved issues. The lack of types makes it really hard to reason about the code.
4. Tusker lacks a few features which I'd like to add but adding them to the current python code base doesn't feel right.

How is it going?

Head over to https://github.com/bikeshedder/tusker-rs and see for your own.

It doesn't work, yet. Don't bother trying it unless you want to start working on it. If that's your intention. Feel free to contact me and we can see what's the best way to collaborate.

[bikeshedder]

@dlight wrote at https://github.com/bikeshedder/tusker/issues/35#issuecomment-1885169258

Not sure if this thread is the right place to ask, but, did you check out postgres_migrator? It says in the readme:

Credits

• migra for making it possible to diff schemas.
• tusker was the inspiration for using temporary databases as diff targets. postgres_migrator adds the ability to generate and run versioned migrations and to perform compaction.

I've seen postgres_migrator but to me it is going in the wrong direction. It works just like tusker but runs migra and PostgreSQL inside a docker container. I want a lightweight tusker binary which can utilize my local running database. That also makes it compatible with a devcontainer without having to resort to awful solutions like DIND (Docker in Docker).

The most promising solution for me was renovate. I was not able to get it working and the whole idea of parsing the output of pgdump seams a bit weird to me. That project also seams to have stalled over the past few months.

Personally migra strikes a nice balance between sophistication and simplicity. PostgreSQL knows its schemas best. Accessing the pg_catalog.* tables for that purpose just makes sense.

The rewrite aims to keep the spirit of tusker while adding two new features:

• A simple migration manager which can run the files in the migrations directory against the database. Also supports embedding in existing rust applications so you don't have to ship two binaries when writing a Rust server.
• A simple type-safe query system. Just like the schemas you write SQL queries and tusker extracts the types from them via EXPLAIN queries. It works similar to sqlx but only takes care of the SQL queries and their types rather than providing a complete database driver.

[dlight]

Okay cool!

A simple type-safe query system. Just like the schemas you write SQL queries and tusker extracts the types from them via EXPLAIN queries. It works similar to sqlx but only takes care of the SQL queries and their types rather than providing a complete database driver.

Is this like cornucopia?

Cornucopia is really top notch; I think that you could use it in conjunction with tusker-rs.

[bikeshedder]

When I first heard about cornucopia a few weeks ago - I did my research before starting this project - I was very hyped about it. After some experiments with it I had two main problems with it:

• The CLI expects docker (or podman) to prepare the queries. Again, that's against my liking of using a local database. I'm using devcontainers for all my newer projects now and I have a fresh DB available. No need for DIND madness.
• The code seams overly complex for what it's doing. Finding the type of a query should be just a matter of preparing the query against a database, reading the params and columns from it and dumping all that information into a JSON file. Cornucopia opted for a custom SQL parser, replacing the $1, $2, $3... syntax by its own :foo, :bar, and adding all sorts of meta data in comments, etc. p.p.

The design I have in mind is a very small CLI tool and a single Query trait that can be derived:

-- get_post_by_id.sql, one file per query
SELECT id, author, text, created
FROM post
WHERE id = $1

#[derive(QueryOne)]
#[query(sql = "get_post_by_id.sql", row = Post)]
struct GetPostById {
    // order of params must match, types are checked
    pub id: i32,
}

struct Post {
    // column name and type are checked, order is irrelevant
    pub id: i32,
    pub author: String,
    pub text: String,
    pub created: time::OffsetDateTime,
}

And when using the query I'm aiming for this API:

let post = db.query(GetPostById { id: 1 }).await?;

I got a prototype working but it's still very rough around the edges. I'm still figuring out how to resolve the paths relative to the #[derive(QueryOne)] and how to type check the Post without having to add a #[derive(FromRow)] to the row structure as well. I might actually end up doing just that if I can't find another way. 🤷

tbh. working on this future is quite frustrating but I haven't given up, yet. I'm tempted to ditch this code and just use cornucopia or sqlx ... 🙈

[bheemvennapureddy]

Any updates on this ?

[bikeshedder]

I finally came around implementing basic diffing of tables and constraints. It doesn't support functions, views, etc.

For simple schemas that just use tables and constraints it's almost usable today.

I plan to release a 0.1.0 shortly. I need to get this in the hand of other developers to add the missing features and fix the remaining bugs which I'm sure there are still plenty of: https://github.com/bikeshedder/tusker-rs

[bheemvennapureddy]

I ended up using https://github.com/stripe/pg-schema-diff as migra is not getting much support now a days. thanks for insights.

[bikeshedder]

I plan to release a 0.1.0 shortly.

I was a bit too optimistic here.

The rewrite is finally here!
📦 https://crates.io/crates/tusker

It's not a 0.1 but a 0.6 to avoid confusion with previous Tusker releases on PyPI.

This marks the EOL of Tusker written in Python. That's why I renamed this repository to tusker-python-legacy and the tusker repository now contains the Rust source code. I will archive this repository in the near future and make a final release of tusker on python with a deprecation notice.

---

Head over to https://github.com/bikeshedder/tusker/ and check out the code. It might be a bit rough around the edges but I'm already using it for my own projects. Now it even comes with a migration management tool and a type extractor for SQL queries. 🥳

[dlight]

Congrats on the Tusker release!

When I first heard about cornucopia a few weeks ago - I did my research before starting this project - I was very hyped about it. After some experiments with it I had two main problems with it:

Just wanted to say that in the meantime Cornucopia went through a hiatus, a fork Clorinde appeared, and then in the last weeks the fork is being merged back into Cornucopia (Cornucopia 1.0 was just released, like, 3 days ago). I don't really use Cornucopia right now but you might want to take another look on it.