Skip to content

Commit 950695c

Browse files
ritvibhattritvibhatt
authored
update ppl doc examples to use otel data (opensearch-project#5303)
Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>
1 parent 6421658 commit 950695c

51 files changed

Lines changed: 2776 additions & 3629 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/category.json‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,14 @@
6666
"user/ppl/functions/system.md",
6767
"user/ppl/general/comments.md",
6868
"user/ppl/general/datatypes.md",
69-
"user/ppl/general/identifiers.md"
69+
"user/ppl/general/identifiers.md",
70+
"user/ppl/cmd/appendcol.md",
71+
"user/ppl/cmd/appendpipe.md",
72+
"user/ppl/cmd/reverse.md",
73+
"user/ppl/cmd/table.md",
74+
"user/ppl/cmd/convert.md",
75+
"user/ppl/cmd/expand.md",
76+
"user/ppl/cmd/flatten.md"
7077
],
7178
"sql_cli": [
7279
"user/dql/expressions.rst",
@@ -85,4 +92,4 @@
8592
"bash_settings": [
8693
"user/ppl/admin/settings.md"
8794
]
88-
}
95+
}

‎docs/user/ppl/cmd/addcoltotals.md‎

Lines changed: 53 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -24,76 +24,86 @@ The `addcoltotals` command supports the following parameters.
2424
| `labelfield` | Optional | The field in which the label is placed. If the field does not exist, it is created and the label is shown in the summary row (last row) of the new field. |
2525
| `label` | Optional | The text that appears in the summary row (last row) to identify the computed totals. When used with `labelfield`, this text is placed in the specified field in the summary row. Default is `Total`. |
2626

27-
## Example 1: Basic example
27+
## Example 1: Adding column totals to a severity breakdown
2828

29-
The following query places the label in an existing field:
29+
The following query adds a total row to a severity breakdown, showing the grand total of all log entries:
3030

3131
```ppl
32-
source=accounts
33-
| fields firstname, balance
34-
| head 3
35-
| addcoltotals labelfield='firstname'
32+
source=otellogs
33+
| stats count() as log_count by severityText
34+
| sort severityText
35+
| fields severityText, log_count
36+
| addcoltotals labelfield='severityText'
3637
```
3738

3839
The query returns the following results:
3940

4041
```text
41-
fetched rows / total rows = 4/4
42-
+-----------+---------+
43-
| firstname | balance |
44-
|-----------+---------|
45-
| Amber | 39225 |
46-
| Hattie | 5686 |
47-
| Nanette | 32838 |
48-
| Total | 77749 |
49-
+-----------+---------+
42+
fetched rows / total rows = 5/5
43+
+--------------+-----------+
44+
| severityText | log_count |
45+
|--------------+-----------|
46+
| DEBUG | 3 |
47+
| ERROR | 7 |
48+
| INFO | 6 |
49+
| WARN | 4 |
50+
| Total | 20 |
51+
+--------------+-----------+
5052
```
5153

52-
## Example 2: Adding column totals with a custom summary label
54+
## Example 2: Adding column totals with a custom label
5355

54-
The following query adds totals after a `stats` command where the final summary event label is `Sum`. It also creates a new field specified by `labelfield` because this field does not exist in the data:
56+
The following query adds totals to error counts per service with a custom summary label:
5557

5658
```ppl
57-
source=accounts
58-
| stats count() by gender
59-
| addcoltotals `count()` label='Sum' labelfield='Total'
59+
source=otellogs
60+
| where severityText = 'ERROR'
61+
| stats count() as errors by `resource.attributes.service.name`
62+
| sort `resource.attributes.service.name`
63+
| addcoltotals errors label='Grand Total' labelfield='Summary'
6064
```
6165

6266
The query returns the following results:
6367

6468
```text
65-
fetched rows / total rows = 3/3
66-
+---------+--------+-------+
67-
| count() | gender | Total |
68-
|---------+--------+-------|
69-
| 1 | F | null |
70-
| 3 | M | null |
71-
| 4 | null | Sum |
72-
+---------+--------+-------+
69+
fetched rows / total rows = 6/6
70+
+--------+----------------------------------+-------------+
71+
| errors | resource.attributes.service.name | Summary |
72+
|--------+----------------------------------+-------------|
73+
| 2 | checkout | null |
74+
| 1 | frontend-proxy | null |
75+
| 2 | payment | null |
76+
| 1 | product-catalog | null |
77+
| 1 | recommendation | null |
78+
| 7 | null | Grand Total |
79+
+--------+----------------------------------+-------------+
7380
```
7481

7582
## Example 3: Using all options
7683

77-
The following query uses the `addcoltotals` command with all options set:
84+
The following query uses the `addcoltotals` command with all options set, totaling only the specified numeric fields and placing the summary label in a new column:
7885

7986
```ppl
80-
source=accounts
81-
| where age > 30
82-
| stats avg(balance) as avg_balance, count() as count by state
83-
| head 3
84-
| addcoltotals avg_balance, count label='Sum' labelfield='Column Total'
87+
source=otellogs
88+
| where severityText IN ('ERROR', 'WARN')
89+
| eval error_count = IF(severityText = 'ERROR', 1, 0), warn_count = IF(severityText = 'WARN', 1, 0)
90+
| stats sum(error_count) as errors, sum(warn_count) as warnings by `resource.attributes.service.name`
91+
| sort `resource.attributes.service.name`
92+
| addcoltotals errors, warnings label='Sum' labelfield='Column Total'
8593
```
8694

8795
The query returns the following results:
8896

8997
```text
90-
fetched rows / total rows = 4/4
91-
+-------------+-------+-------+--------------+
92-
| avg_balance | count | state | Column Total |
93-
|-------------+-------+-------+--------------|
94-
| 39225.0 | 1 | IL | null |
95-
| 4180.0 | 1 | MD | null |
96-
| 5686.0 | 1 | TN | null |
97-
| 49091.0 | 3 | null | Sum |
98-
+-------------+-------+-------+--------------+
98+
fetched rows / total rows = 6/6
99+
+--------+----------+----------------------------------+--------------+
100+
| errors | warnings | resource.attributes.service.name | Column Total |
101+
|--------+----------+----------------------------------+--------------|
102+
| 2 | 0 | checkout | null |
103+
| 1 | 2 | frontend-proxy | null |
104+
| 2 | 0 | payment | null |
105+
| 1 | 2 | product-catalog | null |
106+
| 1 | 0 | recommendation | null |
107+
| 7 | 4 | null | Sum |
108+
+--------+----------+----------------------------------+--------------+
99109
```

‎docs/user/ppl/cmd/addtotals.md‎

Lines changed: 58 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -27,103 +27,91 @@ The `addtotals` command supports the following parameters.
2727
| `label` | Optional | The text that appears in the summary row (last row) to identify the computed totals. When used with `labelfield`, this text is placed in the specified field in the summary row. Default is `Total`. Applicable when `col=true`. This parameter has no effect when the `labelfield` and `fieldname` parameters specify the same field name. |
2828
| `fieldname` | Optional | The field used to store row totals. Applicable when `row=true`. |
2929

30-
## Example 1: Basic example
30+
## Example 1: Adding column totals
3131

32-
The following query places the label in an existing field:
32+
The following query counts errors and warnings per service, then adds a column total row showing the grand totals:
3333

3434
```ppl
35-
source=accounts
36-
| head 3
37-
| fields firstname, balance
38-
| addtotals col=true labelfield='firstname' label='Total'
35+
source=otellogs
36+
| where severityText IN ('ERROR', 'WARN')
37+
| eval error_count = IF(severityText = 'ERROR', 1, 0), warn_count = IF(severityText = 'WARN', 1, 0)
38+
| stats sum(error_count) as errors, sum(warn_count) as warnings by `resource.attributes.service.name`
39+
| sort `resource.attributes.service.name`
40+
| fields `resource.attributes.service.name`, errors, warnings
41+
| addtotals col=true labelfield='resource.attributes.service.name' label='Total'
3942
```
4043

4144
The query returns the following results:
4245

4346
```text
44-
fetched rows / total rows = 4/4
45-
+-----------+---------+-------+
46-
| firstname | balance | Total |
47-
|-----------+---------+-------|
48-
| Amber | 39225 | 39225 |
49-
| Hattie | 5686 | 5686 |
50-
| Nanette | 32838 | 32838 |
51-
| Total | 77749 | null |
52-
+-----------+---------+-------+
53-
```
54-
55-
## Example 2: Adding column totals with a custom summary label
56-
57-
The following query adds totals after a `stats` command, with the final summary event labeled `Sum`. It also creates a new field specified by `labelfield` because the field does not exist in the data:
58-
59-
60-
```ppl
61-
source=accounts
62-
| fields account_number, firstname , balance , age
63-
| addtotals col=true row=false label='Sum' labelfield='Total'
64-
```
65-
66-
The query returns the following results:
67-
68-
```text
69-
fetched rows / total rows = 5/5
70-
+----------------+-----------+---------+-----+-------+
71-
| account_number | firstname | balance | age | Total |
72-
|----------------+-----------+---------+-----+-------|
73-
| 1 | Amber | 39225 | 32 | null |
74-
| 6 | Hattie | 5686 | 36 | null |
75-
| 13 | Nanette | 32838 | 28 | null |
76-
| 18 | Dale | 4180 | 33 | null |
77-
| 38 | null | 81929 | 129 | Sum |
78-
+----------------+-----------+---------+-----+-------+
47+
fetched rows / total rows = 6/6
48+
+----------------------------------+--------+----------+-------+
49+
| resource.attributes.service.name | errors | warnings | Total |
50+
|----------------------------------+--------+----------+-------|
51+
| checkout | 2 | 0 | 2 |
52+
| frontend-proxy | 1 | 2 | 3 |
53+
| payment | 2 | 0 | 2 |
54+
| product-catalog | 1 | 2 | 3 |
55+
| recommendation | 1 | 0 | 1 |
56+
| Total | 7 | 4 | null |
57+
+----------------------------------+--------+----------+-------+
7958
```
8059

81-
If you set `row=true` in the preceding example, both row totals and column totals try to use the same field name (`Total`), creating a conflict. When this happens, the summary row label displays as `null` instead of `Sum` because the field becomes numeric (for row totals) and cannot display string values:
60+
## Example 2: Adding row totals
8261

62+
The following query counts errors and warnings separately per service, then adds a row total showing the combined count of actionable issues per service:
8363

8464
```ppl
85-
source=accounts
86-
| fields account_number, firstname , balance , age
87-
| addtotals col=true row=true label='Sum' labelfield='Total'
65+
source=otellogs
66+
| where severityText IN ('ERROR', 'WARN')
67+
| eval error_count = IF(severityText = 'ERROR', 1, 0), warn_count = IF(severityText = 'WARN', 1, 0)
68+
| stats sum(error_count) as errors, sum(warn_count) as warnings by `resource.attributes.service.name`
69+
| sort `resource.attributes.service.name`
70+
| fields `resource.attributes.service.name`, errors, warnings
71+
| addtotals row=true fieldname='total_issues'
8872
```
8973

9074
The query returns the following results:
9175

9276
```text
9377
fetched rows / total rows = 5/5
94-
+----------------+-----------+---------+-----+-------+
95-
| account_number | firstname | balance | age | Total |
96-
|----------------+-----------+---------+-----+-------|
97-
| 1 | Amber | 39225 | 32 | 39258 |
98-
| 6 | Hattie | 5686 | 36 | 5728 |
99-
| 13 | Nanette | 32838 | 28 | 32879 |
100-
| 18 | Dale | 4180 | 33 | 4231 |
101-
| 38 | null | 81929 | 129 | null |
102-
+----------------+-----------+---------+-----+-------+
78+
+----------------------------------+--------+----------+--------------+
79+
| resource.attributes.service.name | errors | warnings | total_issues |
80+
|----------------------------------+--------+----------+--------------|
81+
| checkout | 2 | 0 | 2 |
82+
| frontend-proxy | 1 | 2 | 3 |
83+
| payment | 2 | 0 | 2 |
84+
| product-catalog | 1 | 2 | 3 |
85+
| recommendation | 1 | 0 | 1 |
86+
+----------------------------------+--------+----------+--------------+
10387
```
10488

10589
## Example 3: Using all options
10690

107-
The following query uses the `addtotals` command with all options set:
91+
The following query uses the `addtotals` command with all options set, combining both row totals and column totals in a single report:
10892

10993
```ppl
110-
source=accounts
111-
| where age > 30
112-
| stats avg(balance) as avg_balance, count() as count by state
113-
| head 3
114-
| addtotals avg_balance, count row=true col=true fieldname='Row Total' label='Sum' labelfield='Column Total'
94+
source=otellogs
95+
| where severityText IN ('ERROR', 'WARN')
96+
| eval error_count = IF(severityText = 'ERROR', 1, 0), warn_count = IF(severityText = 'WARN', 1, 0)
97+
| stats sum(error_count) as errors, sum(warn_count) as warnings by `resource.attributes.service.name`
98+
| sort `resource.attributes.service.name`
99+
| fields `resource.attributes.service.name`, errors, warnings
100+
| addtotals errors, warnings row=true col=true fieldname='Row Total' label='Sum' labelfield='Column Total'
115101
```
116102

117103
The query returns the following results:
118104

119105
```text
120-
fetched rows / total rows = 4/4
121-
+-------------+-------+-------+-----------+--------------+
122-
| avg_balance | count | state | Row Total | Column Total |
123-
|-------------+-------+-------+-----------+--------------|
124-
| 39225.0 | 1 | IL | 39226.0 | null |
125-
| 4180.0 | 1 | MD | 4181.0 | null |
126-
| 5686.0 | 1 | TN | 5687.0 | null |
127-
| 49091.0 | 3 | null | null | Sum |
128-
+-------------+-------+-------+-----------+--------------+
129-
```
106+
fetched rows / total rows = 6/6
107+
+----------------------------------+--------+----------+-----------+--------------+
108+
| resource.attributes.service.name | errors | warnings | Row Total | Column Total |
109+
|----------------------------------+--------+----------+-----------+--------------|
110+
| checkout | 2 | 0 | 2 | null |
111+
| frontend-proxy | 1 | 2 | 3 | null |
112+
| payment | 2 | 0 | 2 | null |
113+
| product-catalog | 1 | 2 | 3 | null |
114+
| recommendation | 1 | 0 | 1 | null |
115+
| null | 7 | 4 | null | Sum |
116+
+----------------------------------+--------+----------+-----------+--------------+
117+
```

0 commit comments

Comments
 (0)