Skip to content

Commit fb923c0

Browse files
authored
feature: Implement FORMAT and WIDTH in EXPLAIN/EXPLAIN ANALYZE (#229)
* Implement compact and wrap using spannerplan * Update spannerplan to include latest changes * Bump spanemuboost to v0.2.10
1 parent 1597fa3 commit fb923c0

17 files changed

Lines changed: 327 additions & 115 deletions

README.md

Lines changed: 79 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ There are differences between spanner-mycli and spanner-cli that include not onl
1515

1616
* [More concise `EXPLAIN` and `EXPLAIN ANALYZE` output](#more-concise-format-of-explain-and-explain-analyze), making efficient use of limited display space.
1717
* Optimized for narrower terminals and code blocks.
18+
* Compact format and query plan wrapping to fit into more limited space.
1819
* Respects my minor use cases
1920
* Protocol Buffers support as `SHOW LOCAL PROTO`, `SHOW REMOTE PROTO`, `SYNC PROTO BUNDLE` statement
2021
* Can use embedded emulator (`--embedded-emulator`)
@@ -1857,9 +1858,9 @@ spanner-mycli
18571858

18581859
```text
18591860
+----+-----------------------------------------------------------------------------+------+-------+---------------+
1860-
| ID | Query_Execution_Plan <execution_method> (metadata, ...) | Rows | Exec. | Total Latency |
1861+
| ID | Operator <execution_method> (metadata, ...) | Rows | Exec. | Total Latency |
18611862
+----+-----------------------------------------------------------------------------+------+-------+---------------+
1862-
| 0 | Distributed Union on Singers <Row> (split_ranges_aligned: false) | 0 | 1 | 7.58 msecs |
1863+
| 0 | Distributed Union on Singers <Row> | 0 | 1 | 7.58 msecs |
18631864
| 1 | +- Local Distributed Union <Row> | 0 | 3 | 0.78 msecs |
18641865
| 2 | +- Serialize Result <Row> | 0 | 3 | 0.75 msecs |
18651866
| *3 | +- Filter Scan <Row> (seekable_key_size: 0) | | | |
@@ -1871,11 +1872,84 @@ Predicates(identified by ID):
18711872
```
18721873

18731874
- `execution_method: {Row|Batch}` metadata is simply displayed as `<Row>` or `<Batch>` after display name of operator.
1874-
- Target metadata, `distribution_table: <target>` and `scan_target: <target>`, are displayed as `on <target>` after display name of operator.
1875-
- `Full scan: true` is shortened as `Full scan`.
1875+
- Target metadata, `distribution_table: <target>`, `scan_target: <target>`, and `table: <target>` are displayed as `on <target>` after display name of operator.
1876+
- Some metadata with boolean value is printed as label.
1877+
- For example, `Full scan: true` and `split_ranges_aligned: true` are shortened as `Full scan` and `split_ranges_aligned`, and they are not printed if the value is `false`.
18761878
- Column names of `EXPLAIN ANALYZE` are shorter than before.
18771879

1878-
Note: These changes except column names can be controlled using `SET CLI_SPANNER_CLI_COMPATIBLE_PLAN=TRUE`.
1880+
Note: These changes except column names can be controlled using `EXPLAIN [ANALYZE] FORMAT=TRADITIONAL` or `SET CLI_EXPLAIN_FORMAT=TRADITIONAL`.
1881+
1882+
To support more limited width environment, there are further options.
1883+
#### Compact format
1884+
1885+
`FORMAT=COMPACT` option removes characters that use width for readability, as long as it doesn't compromise information.
1886+
1887+
- Whitespace is generally not inserted.
1888+
- The characters used for the tree will only be one character wide per level.
1889+
1890+
```
1891+
spanner> EXPLAIN FORMAT=COMPACT
1892+
SELECT * FROM Singers
1893+
JOIN Albums USING (SingerId)
1894+
WHERE LastName LIKE "%son";
1895+
+-----+-------------------------------------------------------------------+
1896+
| ID | Operator <execution_method> (metadata, ...) |
1897+
+-----+-------------------------------------------------------------------+
1898+
| 0 | Distributed Union on Albums<Row> |
1899+
| 1 | +Serialize Result<Row> |
1900+
| 2 | +Cross Apply<Row> |
1901+
| 3 | +[Input]Distributed Union on Singers<Row> |
1902+
| 4 | |+Local Distributed Union<Row> |
1903+
| *5 | | +Filter Scan<Row>(seekable_key_size:0) |
1904+
| 6 | | +Table Scan on Singers<Row>(Full scan,scan_method:Automatic) |
1905+
| 17 | +[Map]Local Distributed Union<Row> |
1906+
| 18 | +Filter Scan<Row>(seekable_key_size:0) |
1907+
| *19 | +Table Scan on Albums<Row>(scan_method:Row) |
1908+
+-----+-------------------------------------------------------------------+
1909+
Predicates(identified by ID):
1910+
5: Residual Condition: ($LastName LIKE '%son')
1911+
19: Seek Condition: ($SingerId_1 = $SingerId)
1912+
```
1913+
1914+
#### Wrapped plans
1915+
1916+
When you want to adjust the width, such as when displaying an execution plan on media where horizontal scrolling is not possible and content might be truncated or wrapped,
1917+
you can use the `WIDTH=<width>` option to wrap the content of the `Operator` column at the specified width.
1918+
1919+
For example, in the illustration below, by setting the width of the ASCII tree drawn in the Operator column to 39 characters,
1920+
the entire output, including the ASCII table, can fit within 80 characters.
1921+
1922+
```
1923+
spanner> EXPLAIN ANALYZE FORMAT=COMPACT WIDTH=39
1924+
SELECT * FROM Singers
1925+
JOIN Albums USING (SingerId)
1926+
WHERE LastName LIKE "%r";
1927+
+-----+-----------------------------------------+------+-------+---------------+
1928+
| ID | Operator | Rows | Exec. | Total Latency |
1929+
+-----+-----------------------------------------+------+-------+---------------+
1930+
| 0 | Distributed Union on Albums<Row> | 1 | 1 | 3.82 msecs |
1931+
| 1 | +Serialize Result<Row> | 1 | 3 | 2.2 msecs |
1932+
| 2 | +Cross Apply<Row> | 1 | 3 | 2.19 msecs |
1933+
| 3 | +[Input]Distributed Union on Singers< | 1 | 3 | 2.16 msecs |
1934+
| | |Row> | | | |
1935+
| 4 | |+Local Distributed Union<Row> | 1 | 4 | 0.25 msecs |
1936+
| *5 | | +Filter Scan<Row>(seekable_key_size | | | |
1937+
| | | :0) | | | |
1938+
| 6 | | +Table Scan on Singers<Row>(Full s | 1 | 4 | 0.21 msecs |
1939+
| | | can,scan_method:Automatic) | | | |
1940+
| 17 | +[Map]Local Distributed Union<Row> | 1 | 1 | 0.03 msecs |
1941+
| 18 | +Filter Scan<Row>(seekable_key_size: | | | |
1942+
| | 0) | | | |
1943+
| *19 | +Table Scan on Albums<Row>(scan_met | 1 | 1 | 0.03 msecs |
1944+
| | hod:Row) | | | |
1945+
+-----+-----------------------------------------+------+-------+---------------+
1946+
Predicates(identified by ID):
1947+
5: Residual Condition: ($LastName LIKE '%r')
1948+
19: Seek Condition: ($SingerId_1 = $SingerId)
1949+
```
1950+
1951+
Note: Since the widths of columns other than the Operator column are not deterministic, there may be cases where the output does not fit within 80 characters even with the same settings, such as when the number of Rows is large.
1952+
18791953

18801954
### Tab character handling
18811955

cli_test.go

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -648,9 +648,9 @@ func TestRenderPlanTree(t *testing.T) {
648648
},
649649
resultSetStats: lo.Must(protojsonUnmarshal[sppb.ResultSetStats, *sppb.ResultSetStats](dcaStatsJSON)),
650650
want: `+-----+-------------------------------------------------------------------------------------------+------+---------+----------+
651-
| ID | Query_Execution_Plan <execution_method> (metadata, ...) | Rows | Scanned | Filtered |
651+
| ID | Operator <execution_method> (metadata, ...) | Rows | Scanned | Filtered |
652652
+-----+-------------------------------------------------------------------------------------------+------+---------+----------+
653-
| 0 | Distributed Union on AlbumsByAlbumTitle <Row> (split_ranges_aligned: false) | 33 | | |
653+
| 0 | Distributed Union on AlbumsByAlbumTitle <Row> | 33 | | |
654654
| *1 | +- Distributed Cross Apply <Row> | 33 | | |
655655
| 2 | +- [Input] Create Batch <Row> | | | |
656656
| 3 | | +- Local Distributed Union <Row> | 7 | | |
@@ -679,7 +679,7 @@ optimizer statistics: auto_20250421_21_29_41UTC
679679
for _, tcase := range tcases {
680680
t.Run(tcase.desc, func(t *testing.T) {
681681
stats := protostruct.DecodeToMap(tcase.resultSetStats.QueryStats)
682-
result, err := generateExplainAnalyzeResult(tcase.sysVars, tcase.resultSetStats.QueryPlan, stats)
682+
result, err := generateExplainAnalyzeResult(tcase.sysVars, tcase.resultSetStats.QueryPlan, stats, explainFormatUnspecified, 0)
683683
if err != nil {
684684
t.Errorf("shouldn't fail, but: %v", err)
685685
}

client_side_statement_def.go

Lines changed: 66 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -291,24 +291,46 @@ var clientSideStatementDefs = []*clientSideStatementDef{
291291
Descriptions: []clientSideStatementDescription{
292292
{
293293
Usage: `Show execution plan without execution`,
294-
Syntax: `EXPLAIN <sql>`,
294+
Syntax: `EXPLAIN [FORMAT=<format>] [WIDTH=<width>] <sql>`,
295+
Note: "Options can be in any order. Spaces are not allowed before or after the `=`.",
295296
},
296297
{
297298
Usage: `Execute query and show execution plan with profile`,
298-
Syntax: `EXPLAIN ANALYZE <sql>`,
299+
Syntax: `EXPLAIN ANALYZE [FORMAT=<format>] [WIDTH=<width>] <sql>`,
300+
Note: "Options can be in any order. Spaces are not allowed before or after the `=`.",
299301
},
300302
},
301-
Pattern: regexp.MustCompile(`(?is)^EXPLAIN\s+(ANALYZE\s+)?(.+)$`),
303+
// To prevent ReDoS, repetition is limit to 10. Maybe it will be enhanced.
304+
Pattern: regexp.MustCompile(`(?is)^EXPLAIN\s+(ANALYZE\s+)?((?:(?:FORMAT|WIDTH)(?:|=\S+)\s+){0,10})(.+)$`),
302305
HandleSubmatch: func(matched []string) (Statement, error) {
303306
isAnalyze := matched[1] != ""
304-
isDML := stmtkind.IsDMLLexical(matched[2])
307+
options, err := parseExplainOptions(matched[2])
308+
if err != nil {
309+
return nil, fmt.Errorf("invalid EXPLAIN%s: %w", lo.Ternary(isAnalyze, " ANALYZE", ""), err)
310+
}
311+
312+
format, err := parseExplainFormat(options["FORMAT"])
313+
if err != nil {
314+
return nil, fmt.Errorf("invalid EXPLAIN%s: %w", lo.Ternary(isAnalyze, " ANALYZE", ""), err)
315+
}
316+
317+
var width int64
318+
if widthStr := options["WIDTH"]; widthStr != "" {
319+
width, err = strconv.ParseInt(widthStr, 10, 64)
320+
if err != nil {
321+
return nil, fmt.Errorf("invalid WIDTH: %v, expected a positive integer, err: %w", widthStr, err)
322+
}
323+
}
324+
325+
sql := matched[3]
326+
isDML := stmtkind.IsDMLLexical(sql)
305327
switch {
306328
case isAnalyze && isDML:
307-
return &ExplainAnalyzeDmlStatement{Dml: matched[2]}, nil
329+
return &ExplainAnalyzeDmlStatement{Dml: sql, Format: format, Width: width}, nil
308330
case isAnalyze:
309-
return &ExplainAnalyzeStatement{Query: matched[2]}, nil
331+
return &ExplainAnalyzeStatement{Query: sql, Format: format, Width: width}, nil
310332
default:
311-
return &ExplainStatement{Explain: matched[2], IsDML: isDML}, nil
333+
return &ExplainStatement{Explain: sql, IsDML: isDML, Format: format, Width: width}, nil
312334
}
313335
},
314336
},
@@ -769,6 +791,31 @@ var clientSideStatementDefs = []*clientSideStatementDef{
769791
},
770792
}
771793

794+
type explainFormat string
795+
796+
const (
797+
explainFormatUnspecified explainFormat = ""
798+
explainFormatCurrent explainFormat = "CURRENT"
799+
explainFormatTraditional explainFormat = "TRADITIONAL"
800+
explainFormatCompact explainFormat = "COMPACT"
801+
)
802+
803+
func parseExplainFormat(s string) (explainFormat, error) {
804+
switch strings.ToUpper(s) {
805+
case "COMPACT":
806+
return explainFormatCompact, nil
807+
case "CURRENT":
808+
return explainFormatCurrent, nil
809+
case "TRADITIONAL":
810+
return explainFormatTraditional, nil
811+
case "":
812+
return explainFormatUnspecified, nil
813+
default:
814+
return "", fmt.Errorf("parse error: unknown explain format: %s", s)
815+
}
816+
817+
}
818+
772819
// Helper functions for HandleSubmatch implementations
773820

774821
func parseTransaction(s string) (isReadOnly bool, err error) {
@@ -867,3 +914,15 @@ func parseIsolationLevel(isolationLevel string) (sppb.TransactionOptions_Isolati
867914
}
868915
return sppb.TransactionOptions_IsolationLevel(p), nil
869916
}
917+
918+
func parseExplainOptions(ss string) (map[string]string, error) {
919+
m := make(map[string]string)
920+
for s := range strings.FieldsSeq(ss) {
921+
before, after, _ := strings.Cut(s, "=")
922+
if before == "" {
923+
return nil, fmt.Errorf("invalid EXPLAIN option, expect <key>[=<value>], but: %s", s)
924+
}
925+
m[strings.ToUpper(before)] = after
926+
}
927+
return m, nil
928+
}

go.mod

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,8 @@ require (
1414
github.com/apstndb/gsqlutils v0.0.0-20250517013444-d2334c88d6ae
1515
github.com/apstndb/lox v0.0.0-20241212132733-62f24606dc82
1616
github.com/apstndb/memebridge v0.5.0
17-
github.com/apstndb/spanemuboost v0.2.8
18-
github.com/apstndb/spannerplanviz v0.6.0
17+
github.com/apstndb/spanemuboost v0.2.10
18+
github.com/apstndb/spannerplan v0.1.1
1919
github.com/apstndb/spantype v0.3.8
2020
github.com/apstndb/spanvalue v0.1.6
2121
github.com/bufbuild/protocompile v0.14.1
@@ -37,7 +37,7 @@ require (
3737
github.com/olekukonko/tablewriter v1.0.6
3838
github.com/samber/lo v1.49.1
3939
github.com/sourcegraph/conc v0.3.0
40-
github.com/testcontainers/testcontainers-go/modules/gcloud v0.36.0
40+
github.com/testcontainers/testcontainers-go/modules/gcloud v0.37.0
4141
github.com/vbauerster/mpb/v8 v8.9.3
4242
go.uber.org/zap v1.27.0
4343
golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0
@@ -65,6 +65,7 @@ require (
6565
github.com/Microsoft/go-winio v0.6.2 // indirect
6666
github.com/VividCortex/ewma v1.2.0 // indirect
6767
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d // indirect
68+
github.com/apstndb/treeprint v0.0.0-20250529153958-e82576b37da6 // indirect
6869
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
6970
github.com/cespare/xxhash/v2 v2.3.0 // indirect
7071
github.com/cncf/xds/go v0.0.0-20250326154945-ae57f3c0d45f // indirect
@@ -97,7 +98,7 @@ require (
9798
github.com/hashicorp/golang-lru v1.0.2 // indirect
9899
github.com/klauspost/compress v1.17.11 // indirect
99100
github.com/lufia/plan9stats v0.0.0-20240909124753-873cd0166683 // indirect
100-
github.com/magiconair/properties v1.8.9 // indirect
101+
github.com/magiconair/properties v1.8.10 // indirect
101102
github.com/mattn/go-colorable v0.1.14 // indirect
102103
github.com/mattn/go-isatty v0.0.20 // indirect
103104
github.com/mattn/go-tty v0.0.7 // indirect
@@ -125,10 +126,9 @@ require (
125126
github.com/stretchr/testify v1.10.0 // indirect
126127
github.com/swaggest/jsonschema-go v0.3.73 // indirect
127128
github.com/swaggest/refl v1.3.0 // indirect
128-
github.com/testcontainers/testcontainers-go v0.36.0 // indirect
129+
github.com/testcontainers/testcontainers-go v0.37.0 // indirect
129130
github.com/tklauser/go-sysconf v0.3.14 // indirect
130131
github.com/tklauser/numcpus v0.9.0 // indirect
131-
github.com/xlab/treeprint v1.2.0 // indirect
132132
github.com/yusufpapurcu/wmi v1.2.4 // indirect
133133
go.opencensus.io v0.24.0 // indirect
134134
go.opentelemetry.io/auto/sdk v1.1.0 // indirect

0 commit comments

Comments
 (0)