Skip to content

Commit cdda68c

Browse files
authored
add SPL to PPL user guide (#207)
1 parent 79ab493 commit cdda68c

5 files changed

Lines changed: 321 additions & 0 deletions

File tree

docs/starlight-docs/astro.config.mjs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -197,6 +197,7 @@ export default defineConfig({
197197
{ label: 'Function Reference', link: '/ppl/functions/' },
198198
{ label: 'Observability Examples', link: '/ppl/examples/' },
199199
{ label: 'PPL for DQL/Lucene Users', link: '/ppl/dql-lucene-users/' },
200+
{ label: 'PPL for SPL Users', link: '/ppl/spl-users/' },
200201
],
201202
},
202203
{
160 KB
Loading
161 KB
Loading
178 KB
Loading
Lines changed: 320 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,320 @@
1+
---
2+
title: "PPL for SPL Users"
3+
description: "A transition guide for users familiar with Splunk Search Processing Language (SPL) - learn the PPL equivalents for the commands and patterns you already know."
4+
---
5+
6+
import { Aside } from '@astrojs/starlight/components';
7+
import PlaygroundLink from '../../../components/PlaygroundLink.astro';
8+
9+
This cheat sheet helps Splunk users transition to OpenSearch's PPL. It maps common Splunk Search Processing Language (SPL) commands to their PPL equivalents with examples using OpenTelemetry observability data.
10+
11+
<PlaygroundLink query="" text="Try PPL in the live playground →" />
12+
13+
## Structure and Concepts
14+
15+
| Aspect | Splunk SPL | OpenSearch PPL | Notes |
16+
|--------|------------|---------------|-------|
17+
| Query structure | `search terms \| command` | `search term source = index \| command` | PPL requires explicit source at the beginning |
18+
| Index reference | `index=name*` | `source=name*` | Different command to specify data source, PPL support referring to multiple indices |
19+
| Raw field | Special `_raw` field | Identify a field in your OpenSearch data that contains the text content you want to work with (often `message` or `content` fields in log data) | Default field configured by the index.query.default_field setting (defaults to * which searches all fields) |
20+
| Time field | Special `_time` field | User-specified timestamp field | PPL uses @timestamp by default |
21+
22+
## Command Reference
23+
24+
This table provides a mapping between Splunk SPL commands and their OpenSearch PPL equivalents:
25+
26+
| Splunk SPL | OpenSearch PPL | Purpose |
27+
|------------|---------------|---------|
28+
| dedup | [dedup](/docs/ppl/commands/dedup/) | Remove duplicate results |
29+
| eval | [eval](/docs/ppl/commands/eval/) | Calculate and create new fields |
30+
| eventstats | [eventstats](/docs/ppl/commands/eventstats/) | Calculate statistics while preserving events |
31+
| mvexpand | [expand](/docs/ppl/commands/expand/) | Expand multi-value fields |
32+
| fields | [fields](/docs/ppl/commands/fields/) | Include or exclude fields |
33+
| fillnull | [fillnull](/docs/ppl/commands/fillnull/) | Replace null values |
34+
| head | [head](/docs/ppl/commands/head/) | Retrieve the first N results |
35+
| join | [join](/docs/ppl/commands/join/) | Combine results from multiple sources |
36+
| lookup | [lookup](/docs/ppl/commands/lookup/) | Enrich data with lookups |
37+
| rare | [rare](/docs/ppl/commands/rare/) | Find the least common values |
38+
| rename | [rename](/docs/ppl/commands/rename/) | Rename fields in results |
39+
| rex | [rex](/docs/ppl/commands/rex/) | Extract with regular expression pattern |
40+
| search | [search](/docs/ppl/commands/search/) | Basic searching of data |
41+
| sort | [sort](/docs/ppl/commands/sort/) | Sort results by specified fields |
42+
| spath | [spath](/docs/ppl/commands/spath/) | Extracting fields from structured text data |
43+
| stats | [stats](/docs/ppl/commands/stats/) | Statistical aggregation of data |
44+
| timechart | [timechart](/docs/ppl/commands/timechart/) | Statistical aggregation of time-series data |
45+
| top | [top](/docs/ppl/commands/top/) | Find the most common values |
46+
| trendline | [trendline](/docs/ppl/commands/trendline/) | Calculate moving averages of fields |
47+
| where | [where](/docs/ppl/commands/where/) | Filter results based on conditions |
48+
49+
## Example Query Conversions
50+
51+
**Simple search:**
52+
- SPL: `error failed`
53+
- PPL: `source=logs-otel-v1* error failed`
54+
55+
**Aggregation:**
56+
- SPL: `index=logs-otel-v1* | stats count by resource.attributes.service.name | sort -count`
57+
- PPL: ``source=logs-otel-v1* | stats count by `resource.attributes.service.name` | sort -count``
58+
59+
**Time-based query:**
60+
- SPL: `index=logs-otel-v1* | stats count by span(@timestamp, 5m)`
61+
- PPL: `source=logs-otel-v1* | stats count by span(@timestamp, 5m)`
62+
63+
**Complex calculation:**
64+
- SPL: `index=logs-otel-v1* | eval severity_category=case(severityNumber >= 17, "high", 1==1, "low")`
65+
- PPL: `source=logs-otel-v1* | eval severity_category=case(severityNumber >= 17 then "high", else "low")`
66+
67+
68+
## Basic search and filtering
69+
70+
### Simple text search
71+
Search for log entries containing specific text terms.
72+
73+
**SPL:**
74+
```
75+
error failed
76+
```
77+
78+
**PPL:**
79+
```sql
80+
source=logs-otel-v1* error failed
81+
```
82+
83+
<PlaygroundLink query="source%3Dlogs-otel-v1*%20error%20failed%0A%7C%20head%2010" />
84+
85+
![Basic search results showing error logs with highlighted terms](/docs/images/ppl/spl-basic-search-example.png)
86+
87+
### Filter by severity level
88+
Find logs with a specific severity level.
89+
90+
**SPL:**
91+
```
92+
index=logs-otel-v1* severityText="ERROR"
93+
```
94+
95+
**PPL:**
96+
```sql
97+
source=logs-otel-v1* severityText="ERROR"
98+
```
99+
100+
<PlaygroundLink query="source%3Dlogs-otel-v1*%20severityText%3D%22ERROR%22%0A%7C%20head%2010" />
101+
102+
### Filter by service and severity
103+
Combine multiple conditions to find errors from a specific service.
104+
105+
**SPL:**
106+
```
107+
index=logs-otel-v1* resource.attributes.service.name="load-generator" AND severityText="ERROR"
108+
```
109+
110+
**PPL:**
111+
```sql
112+
source=logs-otel-v1* `resource.attributes.service.name`="load-generator" AND severityText="ERROR"
113+
```
114+
115+
<PlaygroundLink query="source%3Dlogs-otel-v1*%20%60resource.attributes.service.name%60%3D%22load-generator%22%20AND%20severityText%3D%22ERROR%22%0A%7C%20head%2010" />
116+
117+
### Multiple severity levels
118+
Find logs matching multiple severity levels using OR.
119+
120+
**SPL:**
121+
```
122+
index=logs-otel-v1* severityText="ERROR" OR severityText="WARN"
123+
```
124+
125+
**PPL:**
126+
```sql
127+
source=logs-otel-v1* severityText="ERROR" OR severityText="WARN"
128+
```
129+
130+
<PlaygroundLink query="source%3Dlogs-otel-v1*%20severityText%3D%22ERROR%22%20OR%20severityText%3D%22WARN%22%0A%7C%20head%2010" />
131+
132+
## Aggregation and statistics
133+
134+
### Count logs by service
135+
Count the number of log entries grouped by service name.
136+
137+
**SPL:**
138+
```
139+
index=logs-otel-v1* | stats count by resource.attributes.service.name
140+
```
141+
142+
**PPL:**
143+
```sql
144+
source=logs-otel-v1* | stats count by `resource.attributes.service.name`
145+
```
146+
147+
<PlaygroundLink query="source%3Dlogs-otel-v1*%0A%7C%20stats%20count%20by%20%60resource.attributes.service.name%60" />
148+
149+
![Count logs by service visualization showing bar chart of log counts grouped by service name](/docs/images/ppl/count-logs-by-service-visualization.png)
150+
151+
### Multiple statistics per service
152+
Calculate both count and average severity for each service.
153+
154+
**SPL:**
155+
```
156+
index=logs-otel-v1* | stats count as log_count, avg(severityNumber) as avg_severity by resource.attributes.service.name
157+
```
158+
159+
**PPL:**
160+
```sql
161+
source=logs-otel-v1* | stats count as log_count, avg(severityNumber) as avg_severity by `resource.attributes.service.name`
162+
```
163+
164+
<PlaygroundLink query="source%3Dlogs-otel-v1*%0A%7C%20stats%20count%20as%20log_count%2C%20avg%28severityNumber%29%20as%20avg_severity%20by%20%60resource.attributes.service.name%60" />
165+
166+
### Sort results by count
167+
Sort the aggregated results by log count in descending order.
168+
169+
**SPL:**
170+
```
171+
index=logs-otel-v1* | stats count as log_count by resource.attributes.service.name | sort -log_count
172+
```
173+
174+
**PPL:**
175+
```sql
176+
source=logs-otel-v1* | stats count as log_count by `resource.attributes.service.name` | sort -log_count
177+
```
178+
179+
<PlaygroundLink query="source%3Dlogs-otel-v1*%0A%7C%20stats%20count%20as%20log_count%20by%20%60resource.attributes.service.name%60%0A%7C%20sort%20-log_count" />
180+
181+
![Sort results by count visualization showing bar chart with log counts sorted in descending order](/docs/images/ppl/sort-results-by-count-visualization.png)
182+
183+
## Conditional logic
184+
185+
### Categorize severity levels
186+
Create human-readable categories from numeric severity values.
187+
188+
**SPL:**
189+
```
190+
index=logs-otel-v1* | eval severity_category=case(severityNumber >= 17, "high", severityNumber >= 9, "medium", 1==1, "low")
191+
```
192+
193+
**PPL:**
194+
```sql
195+
source=logs-otel-v1* | eval severity_category=case(severityNumber >= 17, "high", severityNumber >= 9, "medium" else "low")
196+
```
197+
198+
<PlaygroundLink query="source%3Dlogs-otel-v1*%20%7C%20eval%20severity_category%3Dcase%28severityNumber%20%3E%3D%2017%2C%20%22high%22%2C%20severityNumber%20%3E%3D%209%2C%20%22medium%22%20else%20%22low%22%29" />
199+
200+
## Field operations
201+
202+
### Select specific fields
203+
Return only specific fields in the results.
204+
205+
**SPL:**
206+
```
207+
index=logs-otel-v1* | fields @timestamp, resource.attributes.service.name, severityText
208+
```
209+
210+
**PPL:**
211+
```sql
212+
source=logs-otel-v1* | fields @timestamp, `resource.attributes.service.name`, severityText
213+
```
214+
215+
<PlaygroundLink query="source%3Dlogs-otel-v1*%0A%7C%20fields%20%40timestamp%2C%20%60resource.attributes.service.name%60%2C%20severityText%0A%7C%20head%2010" />
216+
217+
218+
## Time-based operations
219+
220+
221+
### Time bucketing
222+
Group logs into 5-minute time intervals and count by service.
223+
224+
**SPL:**
225+
```
226+
index=logs-otel-v1* | stats count by span(@timestamp, 5m) as time_bucket, resource.attributes.service.name
227+
```
228+
229+
**PPL:**
230+
```sql
231+
source=logs-otel-v1* | stats count by span(@timestamp, 5m) as time_bucket, `resource.attributes.service.name`
232+
```
233+
234+
<PlaygroundLink query="source%3Dlogs-otel-v1*%0A%7C%20stats%20count%20by%20span%28%40timestamp%2C%205m%29%20as%20time_bucket%2C%20%60resource.attributes.service.name%60%0A%7C%20sort%20%2B%20time_bucket" />
235+
236+
### Time series visualization
237+
Create time-based charts showing log volume over time.
238+
239+
**SPL:**
240+
```
241+
index=logs-otel-v1* | timechart span=5m count by resource.attributes.service.name
242+
```
243+
244+
**PPL:**
245+
```sql
246+
source=logs-otel-v1* | timechart span=5m count by `resource.attributes.service.name`
247+
```
248+
249+
<PlaygroundLink query="source%3Dlogs-otel-v1*%0A%7C%20timechart%20span%3D5m%20count%20by%20%60resource.attributes.service.name%60" />
250+
251+
## String operations
252+
253+
### Search within field
254+
Use pattern matching to find logs where the body field contains "error".
255+
256+
**SPL:**
257+
```
258+
index=logs-otel-v1* | where like(body, "%error%")
259+
```
260+
261+
**PPL:**
262+
```sql
263+
source=logs-otel-v1* | where like(body, "%error%")
264+
```
265+
266+
<PlaygroundLink query="source%3Dlogs-otel-v1*%0A%7C%20where%20like%28body%2C%20%22%25error%25%22%29%0A%7C%20head%2010" />
267+
268+
### Create descriptive labels
269+
Combine service name and severity into a readable label.
270+
271+
**SPL:**
272+
```
273+
index=logs-otel-v1* | eval service_info=resource.attributes.service.name + " (" + severityText + ")"
274+
```
275+
276+
**PPL:**
277+
```sql
278+
source=logs-otel-v1* | eval service_info=concat(`resource.attributes.service.name`, " (", severityText, ")")
279+
```
280+
281+
<PlaygroundLink query="source%3Dlogs-otel-v1*%0A%7C%20eval%20service_info%3Dconcat%28%60resource.attributes.service.name%60%2C%20%22%20%28%22%2C%20severityText%2C%20%22%29%22%29%0A%7C%20fields%20%60resource.attributes.service.name%60%2C%20severityText%2C%20service_info%0A%7C%20head%2010" />
282+
283+
## Deduplication and sampling
284+
285+
### Remove duplicates
286+
Get only one log entry per unique service name.
287+
288+
**SPL:**
289+
```
290+
index=logs-otel-v1* | dedup resource.attributes.service.name
291+
```
292+
293+
**PPL:**
294+
```sql
295+
source=logs-otel-v1* | dedup `resource.attributes.service.name`
296+
```
297+
298+
<PlaygroundLink query="source%3Dlogs-otel-v1*%0A%7C%20dedup%20%60resource.attributes.service.name%60%0A%7C%20head%2010" />
299+
300+
### Handle missing service names
301+
Replace empty service names with a default value for cleaner reporting.
302+
303+
**SPL:**
304+
```
305+
index=logs-otel-v1* | fillnull value="unknown" resource.attributes.service.name
306+
```
307+
308+
**PPL:**
309+
```sql
310+
source=logs-otel-v1* | fillnull with "unknown" in `resource.attributes.service.name`
311+
```
312+
313+
<PlaygroundLink query="source%3Dlogs-otel-v1*%0A%7C%20fillnull%20with%20%22unknown%22%20in%20%60resource.attributes.service.name%60%0A%7C%20fields%20%60resource.attributes.service.name%60%0A%7C%20head%2010" />
314+
315+
## See also
316+
317+
- [PPL Overview](/docs/ppl/) - Complete PPL language reference
318+
- [`search` command](/docs/ppl/commands/search/) - Full search syntax
319+
- [`stats` command](/docs/ppl/commands/stats/) - Aggregation functions
320+
- [Observability Examples](/docs/ppl/examples/) - Real-world PPL queries for observability data

0 commit comments

Comments
 (0)