Skip to content

Commit 8122ed1

Browse files
authored
Merge pull request rarestype#127 from rarestype/tutorial
[minor]: write a tutorial for jq, and use insights from the tutorial to refine the jq API
2 parents 556c9d7 + 0938a7c commit 8122ed1

6 files changed

Lines changed: 288 additions & 31 deletions

File tree

README.md

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,172 @@ The swift-json library requires Swift 6.2 or later.
2626
<!-- DO NOT EDIT ABOVE! AUTOSYNC CONTENT [STATUS TABLE] -->
2727

2828
[Check deployment minimums](https://swiftinit.org/docs/swift-json#ss:platform-requirements)
29+
30+
31+
## Getting started
32+
33+
Many users only need to parse simple JSON messages, which you can do with the `JSON.Node.init(parsing:)` initializer:
34+
35+
```swift
36+
import JSON
37+
38+
let string: String = """
39+
{"success": true, "value": 0.1}
40+
"""
41+
42+
let root: JSON.Node = try .init(parsing: string)
43+
```
44+
45+
If you have UTF-8 data in array form, you can skip the string entirely, and bind your native `UInt8` buffer to an instance of [`JSON`](https://swiftinit.org/docs/swift-json/jsonast/json).
46+
47+
```swift
48+
let json: JSON = .init(utf8: buffer[...])
49+
let root: JSON.Node = try .init(parsing: json)
50+
```
51+
52+
The difference between `JSON` and [`JSON.Node`](https://swiftinit.org/docs/swift-json/jsonast/json/node) is the former is a unparsed buffer wrapper while the latter is a fully-parsed JSON abstract syntax tree (AST). The separation allows you to strongly-type JSON data without necessarily paying the cost of parsing it up front.
53+
54+
`JSON` is backed by `ArraySlice<UInt8>` to help you avoid unnecessary buffer copies.
55+
56+
Depending on what you want to do with JSON, you will either want to use the **query API** (`JQ`, named for the iconic command line tool it was inspired by), or the **modeling API**.
57+
58+
59+
### Using the query API
60+
61+
The **query API** is good for when you want to extract data from JSON payloads at known locations, or make surgical modifications to the JSON without having to model, or even decode, irrelevant portions of the payload. The syntax should look instantly familiar if you have ever used the `jq` command line tool.
62+
63+
```swift
64+
import JSON
65+
import JQ
66+
67+
let string: String = """
68+
{"id": 1, "scores": []}
69+
"""
70+
71+
var root: JSON.Node = try .init(parsing: string)
72+
try root["name"] &= "Barbara"
73+
try root["scores"][0] &= .number(5)
74+
75+
// {"id":1,"scores":[5],"name":"Barbara"}
76+
```
77+
78+
Assigning to JSON paths can throw an error if data already exists in that location, and the node in there is not compatible with the query path.
79+
80+
```swift
81+
do {
82+
try root["scores"]["banana"] &= true
83+
} catch let error {
84+
// cannot write to protected json node 'scores'
85+
print(error)
86+
}
87+
```
88+
89+
Alternatively, if you just want to read data without modifying it, you can do that by calling the `node` property on the accessor.
90+
91+
```swift
92+
if let score: JSON.Node = try root["scores"][0].node {
93+
print("read back score: \(score)")
94+
}
95+
```
96+
97+
The `JQ` setters are *vivifying*, in other words, they will create objects and arrays if they do not already exist in the AST.
98+
99+
```swift
100+
try root["profile"]["address"]["city"] &= "Malibu"
101+
// {"id":1,"scores":[5],"name":"Barbara","profile":{"address":{"city":"Malibu"}}}
102+
```
103+
104+
You can also delete nodes by assigning `nil` to the node accessor, although `JQ` will not automatically clean up empty containers after deletion.
105+
106+
```swift
107+
try root["profile"]["address"]["city"] &= nil
108+
// {"id":1,"scores":[5],"name":"Barbara","profile":{"address":{}}}
109+
try root["profile"] &= nil
110+
// {"id":1,"scores":[5],"name":"Barbara"}
111+
```
112+
113+
For this reason, more-sophisticated create-modify-delete operations are often better-served by the yielding accessor APIs, which take a closure argument and supply the caller with the preimage of the node being modified.
114+
115+
The yielding accessors are spelled with the `&`, `&?`, and `&!` operators.
116+
117+
```swift
118+
/// this only creates the wrapper objects if the node is
119+
/// assigned a non-nil value
120+
try root["profile"]["address"]["city"] & {
121+
if Bool.random() {
122+
$0 = .string("Tehran")
123+
}
124+
}
125+
126+
/// this only calls the closure if the node already exists
127+
root["profile"]["address"]["city"] &? {
128+
if case .string(let string) = $0 {
129+
$0 = .string(string.value.uppercased())
130+
}
131+
}
132+
```
133+
134+
The most general form of `&` yields the accessed node as `(inout JSON.Node?)`, but it has a variant, `&!`, that passes the caller an `(inout JSON.Node)` binding. `&!` can be thought of as a special case of `&` that initializes the node with a default value of `null` if it does not already exist.
135+
136+
Do note that this `null` is a “hard” `null`, thus if you use `&!`, all wrapper objects will be created, and unlike the fully-general `&`, the update will fail if that `null` can’t be safely written back to the AST.
137+
138+
```swift
139+
/// this initializes the node with a default value (of `null`)
140+
/// if it does not already exist
141+
try root["x"]["y"]["z"] &! {
142+
if Bool.random() {
143+
$0 = true
144+
}
145+
}
146+
```
147+
148+
Empty brackets are used to bind a node to an array type. The array version of `&!` is just like the general version of `&!`, except it initializes missing fields to empty arrays instead of `null`. There is also `&?`, if the convenience of receiving a non-optional `(inout JSON.Array)` is more important to you than the flexibility to conditionally create or delete the array.
149+
150+
```swift
151+
/// this appends a value to the array only if it already exists
152+
try root["scores"][] & {
153+
$0?.elements.append(.number(8))
154+
}
155+
156+
/// this initializes the field to an empty array if it does not already exist
157+
try root["friends"][] &! {
158+
$0.elements.append("Paris")
159+
}
160+
// {"id":1,"scores":[5,8],"friends":["Paris"]}
161+
```
162+
163+
Finally, it’s worth highlighting the powerful `|` and `|?` operators, which provide a concise means of expressing map operations over array fields.
164+
165+
```swift
166+
let scores: [Int]? = try root["scores"][] | { try Int.init(json: $0) }
167+
```
168+
169+
The only difference between `|` and `|?` is that the latter ignores invalid access paths and simply returns nil if the path is incompatible with the container.
170+
171+
There is no syntactical sugar for modifying array elements in place, to do that, you can just compose other library APIs with native Swift loops.
172+
173+
```swift
174+
/// this adds 1 to each score in the 'scores' array
175+
try root["scores"][] &! {
176+
for i: Int in $0.indices {
177+
let score: Int = try $0[i].decode()
178+
$0.elements[i] = .number(score + 1)
179+
}
180+
}
181+
// {"id":1,"scores":[6,9],"friends":["Paris"]}
182+
```
183+
184+
Most developers using the query API import `JSON` alongside `JQ`, to take advantage of the library’s built-in error handling for casting JSON types. That is where the `try $0[i].decode()` API that we used to cast each `score` to `Int` came from. But `JQ` doesn’t actually depend on the full JSON parser, encoder, or decoder. If you are aggressively stripping down dependencies, you can get away with just importing `JSONAST`, which provides the minimal set of tools for working with JSON trees.
185+
186+
```swift
187+
import JSONAST
188+
import JQ
189+
```
190+
191+
### Using the modeling API
192+
193+
Unlike the query API, the **modeling API** is an indexed API — it builds random-access indexes of the JSON, which makes it more efficient if you are trying to destructure a large portion of the data in a payload as opposed to a small subset.
194+
195+
As the name suggests, using the modeling API involves defining the full schema of the JSON you expect to receive. It requires writing more code, but is also significantly more type safe (as it involves reifying the schema into Swift structures), and also enables blazing fast JSON encoding performance, as modeled types know how to serialize themselves to raw JSON buffer output without building syntax trees at all.
196+
197+
Getting the most out of the modeling API will require learning and internalizing a set of reusable code patterns that can be composed to build roundtripping logic for sophisticated JSON data models. We suggest [reading the tutorial](https://swiftinit.org/docs/swift-json/json/decoding) to get started.
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
import JSON
2+
import JQ
3+
4+
let string: String = """
5+
{"id": 1, "scores": []}
6+
"""
7+
8+
var root: JSON.Node = try .init(parsing: string)
9+
try root["name"] &= "Barbara"
10+
try root["scores"][0] &= .number(5)
11+
print("\(root)")
12+
13+
do {
14+
try root["scores"]["banana"] &= true
15+
} catch let error {
16+
print(error)
17+
}
18+
19+
if let score: JSON.Node = try root["scores"][0].node {
20+
print("read back score: \(score)")
21+
}
22+
23+
24+
try root["profile"]["address"]["city"] &= "Malibu"
25+
26+
print("\(root)")
27+
28+
try root["profile"]["address"]["city"] &= nil
29+
30+
print("\(root)")
31+
32+
try root["profile"] &= nil
33+
34+
print("\(root)")
35+
36+
/// this only creates the wrapper objects if the node is assigned a non-nil value
37+
try root["profile"]["address"]["city"] & {
38+
if Bool.random() {
39+
$0 = .string("Tehran")
40+
}
41+
}
42+
43+
/// this only calls the closure if the node already exists
44+
root["profile"]["address"]["city"] &? {
45+
if case .string(let string) = $0 {
46+
$0 = .string(string.value.uppercased())
47+
}
48+
}
49+
print("\(root)")
50+
51+
/// this initializes the node with a default value (of `null`)
52+
/// if it does not already exist
53+
try root["x"]["y"]["z"] &! {
54+
if Bool.random() {
55+
$0 = true
56+
}
57+
}
58+
print("\(root)")
59+
60+
try root["x"] &= nil
61+
try root["name"] &= nil
62+
try root["profile"] &= nil
63+
64+
try root["scores"][] & {
65+
$0?.elements.append(.number(8))
66+
}
67+
68+
print("\(root)")
69+
70+
/// this initializes the field to an empty array if it does not already exist
71+
try root["friends"][] &! {
72+
$0.elements.append("Paris")
73+
}
74+
75+
print("\(root)")
76+
77+
let scores: [Int]? = try root["scores"][] | { try Int.init(json: $0) }
78+
print(scores ?? [])
79+
80+
/// this adds 1 to each score in the 'scores' array
81+
try root["scores"][] &! {
82+
for i: Int in $0.indices {
83+
let score: Int = try $0[i].decode()
84+
$0.elements[i] = .number(score + 1)
85+
}
86+
}
87+
88+
print("\(root)")

Sources/JQ/JSON.ArrayAccessor.swift

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ extension JSON.ArrayAccessor {
5252
///
5353
/// Use this when you are confident that the array must be written, and expect to receive
5454
/// an error if incompatible data already exists in the accessed location.
55-
@inlinable public static func & (
55+
@inlinable public static func &! (
5656
self: inout Self,
5757
yield: (inout JSON.Array) throws -> ()
5858
) throws {
@@ -135,14 +135,14 @@ extension JSON.ArrayAccessor {
135135
@inlinable public static func | <T>(
136136
self: borrowing Self,
137137
yield: (JSON.Node) throws -> T
138-
) throws -> [T] {
138+
) throws -> [T]? {
139139
switch self.state {
140140
case .protected:
141141
throw JSON.NodeAccessError.protected(self.crumb)
142142
case .reserved(let offender):
143143
throw JSON.NodeAccessError.reserved(self.crumb, offender)
144144
case .writable:
145-
return []
145+
return nil
146146
case .occupied(let array):
147147
return try array.elements.map(yield)
148148
}

Sources/JQ/JSON.NodeAccessor.swift

Lines changed: 17 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,22 @@ extension JSON.NodeAccessor {
159159
}
160160
}
161161
}
162+
extension JSON.NodeAccessor {
163+
@inlinable public var node: JSON.Node? {
164+
get throws(JSON.NodeAccessError) {
165+
switch self.state {
166+
case .protected:
167+
throw JSON.NodeAccessError.protected(self.crumb)
168+
case .reserved(let offender):
169+
throw JSON.NodeAccessError.reserved(self.crumb, offender)
170+
case .writable:
171+
return nil
172+
case .occupied(let value):
173+
return value
174+
}
175+
}
176+
}
177+
}
162178
extension JSON.NodeAccessor {
163179
/// Delete the node at the accessed path, throwing a ``NodeAccessError`` if the location is
164180
/// not writable and data already exists there.
@@ -218,7 +234,7 @@ extension JSON.NodeAccessor {
218234
///
219235
/// Use this when you are confident that the node must be written, and expect to receive
220236
/// an error if incompatible data already exists in the accessed location.
221-
@inlinable public static func & (
237+
@inlinable public static func &! (
222238
self: inout Self,
223239
yield: (inout JSON.Node) throws -> ()
224240
) throws {
@@ -281,31 +297,4 @@ extension JSON.NodeAccessor {
281297
try yield(&value)
282298
}
283299
}
284-
285-
@inlinable public static func |? <E, T>(
286-
self: borrowing Self,
287-
yield: (JSON.Node) throws(E) -> T
288-
) throws(E) -> T? {
289-
if case .occupied(let value) = self.state {
290-
return try yield(value)
291-
} else {
292-
return nil
293-
}
294-
}
295-
296-
@inlinable public static func | <T>(
297-
self: borrowing Self,
298-
yield: (JSON.Node) throws -> T
299-
) throws -> T {
300-
switch self.state {
301-
case .protected:
302-
throw JSON.NodeAccessError.protected(self.crumb)
303-
case .reserved(let offender):
304-
throw JSON.NodeAccessError.reserved(self.crumb, offender)
305-
case .writable:
306-
return try yield(.null)
307-
case .occupied(let value):
308-
return try yield(value)
309-
}
310-
}
311300
}

Sources/JQ/lexemes.swift

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
// these precedence groups chosen for consistency with `|` and `&`
22
infix operator |? : AdditionPrecedence
33
infix operator &? : MultiplicationPrecedence
4+
infix operator &! : MultiplicationPrecedence

Sources/JQTests/AccessArrayTests.swift

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -264,4 +264,14 @@ import Testing
264264
let nested: JSON.Node = [[:]]
265265
#expect(nil == (nested[0][] |? { "\($0)" }))
266266
}
267+
268+
@Test static func Append() throws {
269+
var node: JSON.Node = .null
270+
try node[] &! { $0.elements.append("x") }
271+
#expect("\(node)" == "\(["x"] as JSON.Node)")
272+
273+
var nested: JSON.Node = [:]
274+
try nested["a"][] &! { $0.elements.append("x") }
275+
#expect("\(nested)" == "\(["a": ["x"]] as JSON.Node)")
276+
}
267277
}

0 commit comments

Comments
 (0)