feat: add TOON output format (--format toon)#3147
Open
davidfierro wants to merge 1 commit into
Open
Conversation
Adds TOON (Token-Oriented Object Notation, https://github.com/toon-format) as a new structured output format. TOON is a token-efficient text format for LLM consumption (30-60% fewer tokens than JSON). - New OutputFormat.TOON enum value and is_structured property that consolidates the previously triplicated "structured format" check (printing, intermediate-output muting, streamlit log streaming). - TOON printer mirrors the CSV dispatch; rows are pre-converted with _to_encodable_value to preserve data fidelity (Decimal as exact string, bytes/bytearray as hex, non-finite floats as NaN/Infinity tokens matching JSON output, terminal-sanitized keys and values). Empty single-query results print `null`, matching JSON semantics. - Related fix: commands gating structured returns on is_json (stage diff, app diff, app version create, release-channel list, apps init, helpers write-project-definition-schema) now use is_structured, so they emit structured results for CSV and TOON instead of no output at all. - Dependency: toon-format==0.9.0b1 (MIT, zero runtime deps, Python>=3.8). Note: 0.9.0b1 is a beta pin; the 0.1.0 release on PyPI is a stale placeholder.
davidfierro
force-pushed
the
feature/toon-output-format
branch
from
July 18, 2026 18:41
44da427 to
16e5f9a
Compare
davidfierro
marked this pull request as ready for review
July 18, 2026 18:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre-review checklist
tests/output/test_printing.py, round-trip viatoon_format.decode)main)RELEASE-NOTES.md(see when and how).Changes description
Adds TOON (Token-Oriented Object Notation, spec) as a new structured output format:
snow sql -q "..." --format toon(available on every command, like CSV/JSON).Why TOON
TOON is a compact, lossless text format designed for LLM consumption — YAML-style indentation plus CSV-style tabular arrays, which is exactly the shape of
snow sqlquery results. Numbers below are from the project's published benchmarks:snow sqlresult shape). E-commerce −33.3%, nested config −31.9%.snow sqloutput consumed by agents/LLM pipelines — materially lower context cost with no loss of parseability (encode/decode round-trips are lossless; asserted in the unit tests).Implementation
OutputFormat.TOONenum value and anis_structuredproperty that consolidates the previously-triplicated "structured format" check (printing dispatch, intermediate-output muting, streamlit log streaming)._to_encodable_valueto preserve data fidelity:Decimalas exact string (neverfloat),bytes/bytearrayas hex, non-finite floats asNaN/Infinitytokens (matching JSON output, since the toon encoder would collapse them tonull), dates as ISO-8601, keys and string values passed throughsanitize_for_terminal. Empty single-row results printnull, matching JSON semantics.output_format.is_json(stage diff,app diff,app validate,app version create,app release-channel list,apps init,helpers write-project-definition-schema). Under--format csvthese emitted no output at all (console muted +Nonereturn). They now useis_structured, so CSV and TOON get structured results.toon-format==0.9.0b1(MIT, zero runtime dependencies, Python ≥3.8). Note: this pins a beta — the0.1.0release on PyPI is a stale placeholder;0.9.0b1is the spec-aligned implementation. Flagging explicitly for dependency review.Design sign-off requested
Per adding-commands.md, a new output format needs maintainer sign-off — requesting it here:
--format toon(case-insensitive, like the existing values).FeatureFlag(e.g.ENABLE_TOON_OUTPUT_FORMAT) in this PR.null(mirrors JSON).Proposed follow-ups (out of scope here)
printing.py(CustomJSONEncoder,StreamingJSONEncoder,_write_csv_row,_to_encodable_value) into one shared converter._stream_collection_as_csv(raises after the.keys()call that already crashes).