|
| 1 | +# sqlfluff vs sql-sop - when to use which |
| 2 | + |
| 3 | +sqlfluff is the big dog of Python SQL tooling. 800+ rules, dialect-aware, |
| 4 | +first-class dbt integration, active maintainer team, and years of |
| 5 | +production use across thousands of companies. If you asked me "which |
| 6 | +linter should my data team adopt?" my answer is sqlfluff, every time. |
| 7 | + |
| 8 | +So why does sql-sop exist? |
| 9 | + |
| 10 | +Because the two projects aren't solving the same problem, and when I |
| 11 | +needed the second shape I couldn't find a good one. This post is a |
| 12 | +plain-language comparison so you can pick the right tool for your |
| 13 | +situation - or use both. |
| 14 | + |
| 15 | +## The short answer |
| 16 | + |
| 17 | +> Use **sqlfluff** for dialect-aware formatting, dbt integration, and |
| 18 | +> comprehensive lint coverage of a SQL codebase. |
| 19 | +> |
| 20 | +> Use **sql-sop** as a pre-commit hook that catches the "this would |
| 21 | +> delete production" class of mistakes in under 0.1 seconds, on |
| 22 | +> every commit, zero config. |
| 23 | +
|
| 24 | +You can (and probably should) use both. sql-sop runs locally on every |
| 25 | +commit; sqlfluff runs in CI once a minute. They don't overlap much in |
| 26 | +practice. |
| 27 | + |
| 28 | +## Side-by-side |
| 29 | + |
| 30 | +| | sql-sop | sqlfluff | |
| 31 | +|---|---|---| |
| 32 | +| Rule count | 23 (focused) | 800+ (comprehensive) | |
| 33 | +| Config needed | Zero | `.sqlfluff` with dialect + profile | |
| 34 | +| Speed (200 files) | ~0.08s | ~45s | |
| 35 | +| Dialect-aware | No (dialect-neutral regex) | Yes (12+ dialects) | |
| 36 | +| Formatter (fix mode) | No | Yes | |
| 37 | +| dbt integration | No | First-class (`sqlfluff fix models/`) | |
| 38 | +| Pre-commit hook | Yes | Yes | |
| 39 | +| GitHub Action | Yes (in Marketplace) | Community-maintained | |
| 40 | +| Catches DELETE-without-WHERE | Yes | Via custom rules | |
| 41 | +| Catches SQL injection in Python | Yes (libCST scanner) | No (SQL-only) | |
| 42 | +| Language | Python | Python | |
| 43 | +| License | MIT | MIT | |
| 44 | +| Maintainer team | Solo (for now) | ~20 active | |
| 45 | +| Age | ~1 year | ~5 years | |
| 46 | + |
| 47 | +## The real mental model |
| 48 | + |
| 49 | +The way I think about them: |
| 50 | + |
| 51 | +**sqlfluff is a spell-checker for your entire SQL codebase.** Run it in |
| 52 | +CI on every PR. Accept the 45-second build time because you want dialect |
| 53 | +awareness, auto-formatting, and 800-rule coverage. |
| 54 | + |
| 55 | +**sql-sop is a smoke detector for the kitchen.** Wire it to your |
| 56 | +pre-commit hook. If you write `DELETE FROM orders;` and try to commit, |
| 57 | +it fires in under a second. It doesn't know Snowflake from Postgres - it |
| 58 | +doesn't need to. |
| 59 | + |
| 60 | +Smoke detectors and spell-checkers serve different purposes. No house |
| 61 | +ships only one. |
| 62 | + |
| 63 | +## Where sqlfluff is clearly better |
| 64 | + |
| 65 | +- **Formatting.** sql-sop only lints; sqlfluff fixes and reformats. |
| 66 | +- **Dialect accuracy.** sqlfluff's parser knows Snowflake's variant |
| 67 | + syntax, BigQuery's wildcards, MS SQL's `TOP`. sql-sop uses regex, so |
| 68 | + it makes trade-offs. |
| 69 | +- **dbt projects.** sqlfluff is de facto in the dbt ecosystem. Ten |
| 70 | + years from now it'll still be the right answer for dbt. |
| 71 | +- **Rule depth.** 800 rules vs 23. If you want every edge case covered, |
| 72 | + sqlfluff is the only answer. |
| 73 | +- **Large codebases.** 10,000+ SQL files? sqlfluff's parser-based |
| 74 | + approach scales better than regex on that order of magnitude. |
| 75 | +- **Team adoption.** sqlfluff has a mature config file and inherited- |
| 76 | + rule system. Teams need that. |
| 77 | + |
| 78 | +## Where sql-sop is useful (and sqlfluff isn't quite the right shape) |
| 79 | + |
| 80 | +- **Pre-commit hook speed.** Developers abandon hooks that take more than |
| 81 | + a couple of seconds. 0.08 seconds is imperceptible; 45 seconds is |
| 82 | + painful enough that people `--no-verify`. That's the main reason |
| 83 | + sql-sop exists. |
| 84 | +- **Zero-config starting point.** You can `pip install sql-sop` and get |
| 85 | + useful output in the next five seconds. sqlfluff requires picking a |
| 86 | + dialect and tuning a profile; that's the right investment for a team |
| 87 | + but an overhead for a first commit. |
| 88 | +- **Catching SQL injection patterns in Python.** sql-sop's |
| 89 | + `--include-python` mode walks your Python source with libCST and |
| 90 | + flags f-string interpolation inside `.execute()`, `text()`, |
| 91 | + `.read_sql()`, etc. sqlfluff doesn't do this because it's out of |
| 92 | + scope for a SQL linter. |
| 93 | +- **The "smoke detector" rule count.** 23 rules is small enough to |
| 94 | + read in one sitting. Every rule has a dedicated test. The whole |
| 95 | + ruleset fits in memory. For the pre-commit shape, that's a feature, |
| 96 | + not a limitation. |
| 97 | +- **Regex + AST hybrid, small surface.** sqlfluff's parser is brilliant |
| 98 | + but it's a lot to embed in a pre-commit hook. sql-sop is ~2000 lines |
| 99 | + of Python, most of it regex patterns. |
| 100 | + |
| 101 | +## Honest shortcomings of sql-sop |
| 102 | + |
| 103 | +- **Dialect-neutral means some false positives.** `W001 select-star` |
| 104 | + triggers even in a materialized-view definition where `SELECT *` is |
| 105 | + intentional. Future plan: `-- noqa: W001` inline disables. |
| 106 | +- **Structural rules are shallow.** `S001-S003` use sqlparse, which is |
| 107 | + decent for basic AST but not production-grade. Complex nested queries |
| 108 | + can defeat the depth check. |
| 109 | +- **No fix mode.** You can't run `sql-sop --fix` the way you can with |
| 110 | + `sqlfluff fix`. |
| 111 | +- **Small community.** One maintainer, ~300 downloads/month. If you |
| 112 | + need something in the next 24 hours, sqlfluff's 20-person team will |
| 113 | + get to you first. |
| 114 | +- **Newer.** A year of production use is not five years of production |
| 115 | + use. |
| 116 | + |
| 117 | +## When to use both together |
| 118 | + |
| 119 | +A concrete setup I run in a production data pipeline: |
| 120 | + |
| 121 | +```yaml |
| 122 | +# .pre-commit-config.yaml |
| 123 | +repos: |
| 124 | + - repo: https://github.com/Pawansingh3889/sql-guard |
| 125 | + rev: v0.4.1 |
| 126 | + hooks: |
| 127 | + - id: sql-guard |
| 128 | + args: [--severity, error] # fast, block on real dangers |
| 129 | +``` |
| 130 | +
|
| 131 | +```yaml |
| 132 | +# .github/workflows/ci.yml (excerpt) |
| 133 | +jobs: |
| 134 | + sqlfluff: |
| 135 | + runs-on: ubuntu-latest |
| 136 | + steps: |
| 137 | + - uses: actions/checkout@v4 |
| 138 | + - run: | |
| 139 | + pip install sqlfluff |
| 140 | + sqlfluff lint --dialect postgres . |
| 141 | +``` |
| 142 | +
|
| 143 | +Pre-commit catches the dangerous stuff in under a second; CI runs |
| 144 | +sqlfluff at leisure and enforces the style guide. Developers never feel |
| 145 | +the cost of either. |
| 146 | +
|
| 147 | +## If you're picking one |
| 148 | +
|
| 149 | +- **You maintain a dbt project.** Use sqlfluff. Don't look back. |
| 150 | +- **You have a SQL-heavy Python backend and want pre-commit safety.** |
| 151 | + Use sql-sop. Add sqlfluff later if you expand the SQL codebase. |
| 152 | +- **You're a solo dev who just wants a lint in your next commit.** |
| 153 | + Use sql-sop for five seconds. Add sqlfluff when it starts paying off. |
| 154 | +- **You have a mature data team across multiple dialects.** Use |
| 155 | + sqlfluff. Everything else is a distraction. |
| 156 | +
|
| 157 | +## Honest footnote |
| 158 | +
|
| 159 | +I'm the author of sql-sop, so take the comparison with that bias in |
| 160 | +mind. The benchmark number (0.08s vs 45s) was run on my machine |
| 161 | +against a 200-file corpus; your mileage varies. sqlfluff's 800 rules |
| 162 | +are mostly opt-in - the default rule set is more like 80, which is still |
| 163 | +vastly more than sql-sop's 23. |
| 164 | +
|
| 165 | +None of this is a zero-sum argument. sqlfluff is excellent. sql-sop |
| 166 | +exists because I wanted a smaller, faster shape for the pre-commit |
| 167 | +surface, and writing it was cheaper than bending sqlfluff into that |
| 168 | +shape. If you end up picking sqlfluff after reading this, that's a |
| 169 | +reasonable outcome. |
| 170 | +
|
| 171 | +--- |
| 172 | +
|
| 173 | +*The sql-sop repo and PyPI package are here:* |
| 174 | +
|
| 175 | +- *Repo: <https://github.com/Pawansingh3889/sql-guard>* |
| 176 | +- *PyPI: `pip install sql-sop`* |
| 177 | + |
| 178 | +*sqlfluff's home:* |
| 179 | + |
| 180 | +- *Repo: <https://github.com/sqlfluff/sqlfluff>* |
| 181 | +- *PyPI: `pip install sqlfluff`* |
0 commit comments