Skip to content

Commit 1811ad0

Browse files
PabloRMiraPabloRMira
authored
[DOC] Add docs for formatting logic (#123)
1 parent b703245 commit 1811ad0

3 files changed

Lines changed: 226 additions & 11 deletions

File tree

‎README.md‎

Lines changed: 69 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ print(format_sql(example_sql))
8686
and b.qwer = 2
8787
and a.asdf <= 1 --comment that
8888
or b.qwer >= 5
89-
GROUP BY a.asdf;
89+
GROUP BY a.asdf
9090

9191

9292
It can even deal with subqueries and it will correct my favourite simple careless mistake (comma at the end of SELECT statement before of FROM) for you on the flow :-)
@@ -116,7 +116,7 @@ where qwer1 >= 0
116116
FROM table2
117117
WHERE qwer2 = 1) as b
118118
ON a.asdf = b.asdf
119-
WHERE qwer1 >= 0;
119+
WHERE qwer1 >= 0
120120

121121

122122
The formatter is also robust against nested subqueries
@@ -138,7 +138,7 @@ field3 from table1 where a=1 and b>=100))
138138
field3
139139
FROM table1
140140
WHERE a = 1
141-
and b >= 100));
141+
and b >= 100))
142142

143143

144144
If you do not want to get some query formatted in your SQL file then you can use the marker `/*skip-formatter*/` in your query to disable formatting for just the corresponding query
@@ -198,6 +198,70 @@ This package is just a SQL formatter and therefore
198198
* cannot parse your SQL queries into e.g. dictionaries
199199
* cannot validate your SQL queries to be valid for the corresponding database system / provider
200200

201+
Up to now it only formats queries of the form
202+
203+
* `CREATE TABLE / VIEW ...`
204+
* `SELECT ...`
205+
206+
Every other SQL commands will remain unformatted, e.g. `INSERT INTO` ...
207+
208+
## Formatting Logic
209+
210+
The main goal of the `sql_formatter` is to enhance readability and quick understanding of SQL queries via proper formatting. We use **indentation** and **lowercasing / uppercasing** as means to arrange statements / clauses and parameters into context. By **programmatically standardizing** the way to write SQL queries we help the user understand its queries faster.
211+
212+
As a by-product of using the `sql_formatter`, developer teams can focus on the query logic itself and save time by not incurring into styling decisions, this then begin accomplished by the `sql_formatter`. This is similar to the goal accomplished by the [black package](https://github.com/psf/black) for the Python language, which was also an inspiration for the development of this package for SQL.
213+
214+
We can summarize the main steps of the formatter as follows:
215+
216+
1. Each query is separated from above by two newlines.
217+
2. Everything but **main statements\* / clauses** is lowercased
218+
219+
\* Main statements:
220+
221+
* CREATE ... TABLE / VIEW table_name AS
222+
* SELECT (DISTINCT)
223+
* FROM
224+
* (LEFT / INNER / RIGHT / OUTER) JOIN
225+
* UNION
226+
* ON
227+
* WHERE
228+
* GROUP BY
229+
* ORDER BY
230+
* OVER
231+
* PARTITION BY
232+
233+
3. Indentation is used to put parameters into context. Here an easy example:
234+
235+
```sql
236+
SELECT field1,
237+
case when field2 > 1 and
238+
field2 <= 10 and
239+
field1 = 'a' then 1
240+
else 0 end as case_field,
241+
...
242+
FROM table1
243+
WHERE field1 = 1
244+
and field2 <= 2
245+
or field3 = 5
246+
ORDER BY field1
247+
```
248+
> This is a very nice, easy example but things can become more complicated if comments come into play
249+
250+
4. Subqueries are also properly indented, e.g.:```sqlSELECT a.field1,
251+
a.field2,
252+
b.field3
253+
FROM (SELECT field1,
254+
field2
255+
FROM table1
256+
WHERE field1 = 1) as a
257+
LEFT JOIN (SELECT field1,
258+
field3
259+
FROM table2) as b
260+
ON a.field1 = b.field1:
261+
```
262+
263+
5. Everything not being a query of the form `CREATE ... TABLE / VIEW` or `SELECT ...` is left unchanged
264+
201265
## How to contribute
202266
203267
You can contribute to this project:
@@ -282,3 +346,5 @@ For development we follow these steps:
282346
Thank you very much to Jeremy Howard and all the [nbdev](https://github.com/fastai/nbdev) team for enabling the *fast* and delightful development of this library via the `nbdev` framework.
283347

284348
For more details on `nbdev`, see its official [tutorial](https://nbdev.fast.ai/tutorial.html)
349+
350+
Thank you very much for the developers of the [black](https://github.com/psf/black) package, which was also an inspiration for the development of this package

‎docs/index.html‎

Lines changed: 77 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,7 @@ <h3 id="Usage-in-Python">Usage in Python<a class="anchor-link" href="#Usage-in-P
165165
and b.qwer = 2
166166
and a.asdf &lt;= 1 --comment that
167167
or b.qwer &gt;= 5
168-
GROUP BY a.asdf;
168+
GROUP BY a.asdf
169169
</pre>
170170
</div>
171171
</div>
@@ -224,7 +224,7 @@ <h3 id="Usage-in-Python">Usage in Python<a class="anchor-link" href="#Usage-in-P
224224
FROM table2
225225
WHERE qwer2 = 1) as b
226226
ON a.asdf = b.asdf
227-
WHERE qwer1 &gt;= 0;
227+
WHERE qwer1 &gt;= 0
228228
</pre>
229229
</div>
230230
</div>
@@ -275,7 +275,7 @@ <h3 id="Usage-in-Python">Usage in Python<a class="anchor-link" href="#Usage-in-P
275275
field3
276276
FROM table1
277277
WHERE a = 1
278-
and b &gt;= 100));
278+
and b &gt;= 100))
279279
</pre>
280280
</div>
281281
</div>
@@ -375,6 +375,79 @@ <h3 id="What-sql_formatter-does-not-do">What <code>sql_formatter</code> does not
375375
<li>cannot parse your SQL queries into e.g. dictionaries</li>
376376
<li>cannot validate your SQL queries to be valid for the corresponding database system / provider</li>
377377
</ul>
378+
<p>Up to now it only formats queries of the form</p>
379+
<ul>
380+
<li><code>CREATE TABLE / VIEW ...</code></li>
381+
<li><code>SELECT ...</code></li>
382+
</ul>
383+
<p>Every other SQL commands will remain unformatted, e.g. <code>INSERT INTO</code> ...</p>
384+
385+
</div>
386+
</div>
387+
</div>
388+
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
389+
<div class="text_cell_render border-box-sizing rendered_html">
390+
<h2 id="Formatting-Logic">Formatting Logic<a class="anchor-link" href="#Formatting-Logic"> </a></h2><p>The main goal of the <code>sql_formatter</code> is to enhance readability and quick understanding of SQL queries via proper formatting. We use <strong>indentation</strong> and <strong>lowercasing / uppercasing</strong> as means to arrange statements / clauses and parameters into context. By <strong>programmatically standardizing</strong> the way to write SQL queries we help the user understand its queries faster.</p>
391+
<p>As a by-product of using the <code>sql_formatter</code>, developer teams can focus on the query logic itself and save time by not incurring into styling decisions, this then begin accomplished by the <code>sql_formatter</code>. This is similar to the goal accomplished by the <a href="https://github.com/psf/black">black package</a> for the Python language, which was also an inspiration for the development of this package for SQL.</p>
392+
<p>We can summarize the main steps of the formatter as follows:</p>
393+
<ol>
394+
<li>Each query is separated from above by two newlines.</li>
395+
<li>Everything but <strong>main statements* / clauses</strong> is lowercased</li>
396+
</ol>
397+
<p>* Main statements:</p>
398+
<ul>
399+
<li>CREATE ... TABLE / VIEW table_name AS</li>
400+
<li>SELECT (DISTINCT)</li>
401+
<li>FROM</li>
402+
<li>(LEFT / INNER / RIGHT / OUTER) JOIN</li>
403+
<li>UNION</li>
404+
<li>ON</li>
405+
<li>WHERE</li>
406+
<li>GROUP BY</li>
407+
<li>ORDER BY</li>
408+
<li>OVER</li>
409+
<li>PARTITION BY</li>
410+
</ul>
411+
<ol>
412+
<li>Indentation is used to put parameters into context. Here an easy example:</li>
413+
</ol>
414+
<div class="highlight"><pre><span></span><span class="k">SELECT</span> <span class="n">field1</span><span class="p">,</span>
415+
<span class="k">case</span> <span class="k">when</span> <span class="n">field2</span> <span class="o">&gt;</span> <span class="mi">1</span> <span class="k">and</span>
416+
<span class="n">field2</span> <span class="o">&lt;=</span> <span class="mi">10</span> <span class="k">and</span>
417+
<span class="n">field1</span> <span class="o">=</span> <span class="s1">&#39;a&#39;</span> <span class="k">then</span> <span class="mi">1</span>
418+
<span class="k">else</span> <span class="mi">0</span> <span class="k">end</span> <span class="k">as</span> <span class="n">case_field</span><span class="p">,</span>
419+
<span class="p">...</span>
420+
<span class="k">FROM</span> <span class="n">table1</span>
421+
<span class="k">WHERE</span> <span class="n">field1</span> <span class="o">=</span> <span class="mi">1</span>
422+
<span class="k">and</span> <span class="n">field2</span> <span class="o">&lt;=</span> <span class="mi">2</span>
423+
<span class="k">or</span> <span class="n">field3</span> <span class="o">=</span> <span class="mi">5</span>
424+
<span class="k">ORDER</span> <span class="k">BY</span> <span class="n">field1</span>
425+
</pre></div>
426+
<blockquote><p>This is a very nice, easy example but things can become more complicated if comments come into play</p>
427+
</blockquote>
428+
<ol>
429+
<li><p>Subqueries are also properly indented, e.g.:```sqlSELECT a.field1,</p>
430+
431+
<pre><code>a.field2,
432+
b.field3
433+
</code></pre>
434+
<p>FROM (SELECT field1,</p>
435+
436+
<pre><code> field2
437+
FROM table1
438+
WHERE field1 = 1) as a
439+
</code></pre>
440+
<p>LEFT JOIN (SELECT field1,</p>
441+
442+
<pre><code> field3
443+
FROM table2) as b
444+
ON a.field1 = b.field1:
445+
</code></pre>
446+
<p>```</p>
447+
</li>
448+
<li><p>Everything not being a query of the form <code>CREATE ... TABLE / VIEW</code> or <code>SELECT ...</code> is left unchanged</p>
449+
</li>
450+
</ol>
378451

379452
</div>
380453
</div>
@@ -472,6 +545,7 @@ <h2 id="Acknowledgements">Acknowledgements<a class="anchor-link" href="#Acknowle
472545
<div class="text_cell_render border-box-sizing rendered_html">
473546
<p>Thank you very much to Jeremy Howard and all the <a href="https://github.com/fastai/nbdev">nbdev</a> team for enabling the <em>fast</em> and delightful development of this library via the <code>nbdev</code> framework.</p>
474547
<p>For more details on <code>nbdev</code>, see its official <a href="https://nbdev.fast.ai/tutorial.html">tutorial</a></p>
548+
<p>Thank you very much for the developers of the <a href="https://github.com/psf/black">black</a> package, which was also an inspiration for the development of this package</p>
475549

476550
</div>
477551
</div>

‎nbs/index.ipynb‎

Lines changed: 80 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -152,7 +152,7 @@
152152
" and b.qwer = 2\n",
153153
" and a.asdf <= 1 --comment that\n",
154154
" or b.qwer >= 5\n",
155-
"GROUP BY a.asdf;\n"
155+
"GROUP BY a.asdf\n"
156156
]
157157
}
158158
],
@@ -189,7 +189,7 @@
189189
" FROM table2\n",
190190
" WHERE qwer2 = 1) as b\n",
191191
" ON a.asdf = b.asdf\n",
192-
"WHERE qwer1 >= 0;\n"
192+
"WHERE qwer1 >= 0\n"
193193
]
194194
}
195195
],
@@ -231,7 +231,7 @@
231231
" field3\n",
232232
" FROM table1\n",
233233
" WHERE a = 1\n",
234-
" and b >= 100));\n"
234+
" and b >= 100))\n"
235235
]
236236
}
237237
],
@@ -322,7 +322,80 @@
322322
"This package is just a SQL formatter and therefore\n",
323323
"\n",
324324
"* cannot parse your SQL queries into e.g. dictionaries\n",
325-
"* cannot validate your SQL queries to be valid for the corresponding database system / provider"
325+
"* cannot validate your SQL queries to be valid for the corresponding database system / provider\n",
326+
"\n",
327+
"Up to now it only formats queries of the form\n",
328+
"\n",
329+
"* `CREATE TABLE / VIEW ...`\n",
330+
"* `SELECT ...`\n",
331+
"\n",
332+
"Every other SQL commands will remain unformatted, e.g. `INSERT INTO` ..."
333+
]
334+
},
335+
{
336+
"cell_type": "markdown",
337+
"metadata": {},
338+
"source": [
339+
"## Formatting Logic\n",
340+
"\n",
341+
"The main goal of the `sql_formatter` is to enhance readability and quick understanding of SQL queries via proper formatting. We use **indentation** and **lowercasing / uppercasing** as means to arrange statements / clauses and parameters into context. By **programmatically standardizing** the way to write SQL queries we help the user understand its queries faster.\n",
342+
"\n",
343+
"As a by-product of using the `sql_formatter`, developer teams can focus on the query logic itself and save time by not incurring into styling decisions, this then begin accomplished by the `sql_formatter`. This is similar to the goal accomplished by the [black package](https://github.com/psf/black) for the Python language, which was also an inspiration for the development of this package for SQL. \n",
344+
"\n",
345+
"We can summarize the main steps of the formatter as follows:\n",
346+
"\n",
347+
"1. Each query is separated from above by two newlines.\n",
348+
"2. Everything but **main statements\\* / clauses** is lowercased\n",
349+
"\n",
350+
"\\* Main statements:\n",
351+
"\n",
352+
"* CREATE ... TABLE / VIEW table_name AS\n",
353+
"* SELECT (DISTINCT)\n",
354+
"* FROM\n",
355+
"* (LEFT / INNER / RIGHT / OUTER) JOIN\n",
356+
"* UNION\n",
357+
"* ON\n",
358+
"* WHERE\n",
359+
"* GROUP BY\n",
360+
"* ORDER BY\n",
361+
"* OVER\n",
362+
"* PARTITION BY\n",
363+
"\n",
364+
"3. Indentation is used to put parameters into context. Here an easy example:\n",
365+
"\n",
366+
"```sql\n",
367+
"SELECT field1,\n",
368+
" case when field2 > 1 and\n",
369+
" field2 <= 10 and\n",
370+
" field1 = 'a' then 1\n",
371+
" else 0 end as case_field,\n",
372+
" ...\n",
373+
"FROM table1\n",
374+
"WHERE field1 = 1\n",
375+
" and field2 <= 2\n",
376+
" or field3 = 5\n",
377+
"ORDER BY field1\n",
378+
"```\n",
379+
"\n",
380+
"> This is a very nice, easy example but things can become more complicated if comments come into play\n",
381+
"\n",
382+
"4. Subqueries are also properly indented, e.g.:\n",
383+
"\n",
384+
"```sql\n",
385+
"SELECT a.field1,\n",
386+
" a.field2,\n",
387+
" b.field3\n",
388+
"FROM (SELECT field1,\n",
389+
" field2\n",
390+
" FROM table1\n",
391+
" WHERE field1 = 1) as a\n",
392+
" LEFT JOIN (SELECT field1,\n",
393+
" field3\n",
394+
" FROM table2) as b\n",
395+
" ON a.field1 = b.field1:\n",
396+
"```\n",
397+
"\n",
398+
"5. Everything not being a query of the form `CREATE ... TABLE / VIEW` or `SELECT ...` is left unchanged"
326399
]
327400
},
328401
{
@@ -427,7 +500,9 @@
427500
"source": [
428501
"Thank you very much to Jeremy Howard and all the [nbdev](https://github.com/fastai/nbdev) team for enabling the *fast* and delightful development of this library via the `nbdev` framework.\n",
429502
"\n",
430-
"For more details on `nbdev`, see its official [tutorial](https://nbdev.fast.ai/tutorial.html)"
503+
"For more details on `nbdev`, see its official [tutorial](https://nbdev.fast.ai/tutorial.html)\n",
504+
"\n",
505+
"Thank you very much for the developers of the [black](https://github.com/psf/black) package, which was also an inspiration for the development of this package"
431506
]
432507
}
433508
],

0 commit comments

Comments
 (0)