1111
1212Description
1313============
14- | The ``rex`` command extracts fields from a raw text field using regular expression named capture groups and optionally filters events that match the extraction pattern .
14+ | The ``rex`` command extracts fields from a raw text field using regular expression named capture groups.
1515
16+ Version
17+ =======
18+ 3.3.0
1619
1720Syntax
1821============
1922rex [mode=<mode>] field=<field> <pattern> [max_match=<int>] [offset_field=<string>]
2023
21- * field: mandatory. The field must be a text field to extract data from.
24+ * field: mandatory. The field must be a string field to extract data from.
2225* pattern: mandatory string. The regular expression pattern with named capture groups used to extract new fields. Pattern must contain at least one named capture group using ``(?<name>pattern) `` syntax.
2326* mode: optional. Either ``extract `` (default) or ``sed ``. In extract mode, creates new fields from named capture groups. In sed mode, performs text substitution on the field.
27+
28+ **Sed mode syntax: **
29+
30+ - **Substitution: ** ``s/pattern/replacement/[flags] ``
31+
32+ - ``s/pattern/replacement/ `` - Replace first occurrence
33+ - ``s/pattern/replacement/g `` - Replace all occurrences (global)
34+ - ``s/pattern/replacement/n `` - Replace only the nth occurrence (where n is a number)
35+ - Backreferences: Use ``\1 ``, ``\2 ``, etc. to reference captured groups in replacement
36+
37+ - **Transliteration: ** ``y/from_chars/to_chars/ `` - Character-by-character mapping
38+
2439* max_match: optional integer (default=1). Maximum number of matches to extract. If greater than 1, extracted fields become arrays.
2540* offset_field: optional string. Field name to store the character offset positions of matches.
2641
@@ -50,13 +65,13 @@ PPL query::
5065
5166 os> source=accounts | rex field=email "(?<username>[^@]+)@(?<domain>[^.]+)" | fields email, username, domain | head 3 ;
5267 fetched rows / total rows = 3/3
53- +-----------------------+ -------------+---------+
54- | email | username | domain |
55- |-----------------------+-------------| ---------|
56- | amberduke@pyrami.com | amberduke | pyrami |
57- | hattiebond@netagy.com | hattiebond | netagy |
68+ +--------------------------+- -------------+---------+
69+ | email | username | domain |
70+ |-------------------------- +--------------+ ---------|
71+ | amberduke@pyrami.com | amberduke | pyrami |
72+ | hattiebond@netagy.com | hattiebond | netagy |
5873 | nanettebates@quility.com | nanettebates | quility |
59- +-----------------------+ -------------+---------+
74+ +--------------------------+- -------------+---------+
6075
6176
6277Example 2: Extract with Filtering
@@ -86,13 +101,13 @@ PPL query::
86101
87102 os> source=accounts | rex field=address "(?<words>[A-Za-z]+)" max_match=3 | fields address, words | head 3 ;
88103 fetched rows / total rows = 3/3
89- +------------------+ ------------------+
90- | address | words |
91- |------------------| ------------------|
92- | 880 Holmes Lane | [Holmes, Lane] |
104+ +--------------------+- ------------------+
105+ | address | words |
106+ |--------------------|- ------------------|
107+ | 880 Holmes Lane | [Holmes, Lane] |
93108 | 671 Bristol Street | [Bristol, Street] |
94109 | 789 Madison Street | [Madison, Street] |
95- +------------------+ ------------------+
110+ +--------------------+- ------------------+
96111
97112
98113Example 4: Text Replacement with mode=sed
@@ -104,13 +119,13 @@ PPL query::
104119
105120 os> source=accounts | rex field=email mode=sed "s/@.*/@company.com/" | fields email | head 3 ;
106121 fetched rows / total rows = 3/3
107- +----------------------+
108- | email |
109- |----------------------|
110- | amberduke@company.com |
111- | hattiebond@company.com |
112- | nanettebates@company.com |
113- +----------------------+
122+ +--------------------------- +
123+ | email |
124+ |--------------------------- |
125+ | amberduke@company.com |
126+ | hattiebond@company.com |
127+ | nanettebates@company.com |
128+ +--------------------------- +
114129
115130
116131Example 5: Using offset_field
@@ -122,13 +137,13 @@ PPL query::
122137
123138 os> source=accounts | rex field=email "(?<username>[^@]+)@(?<domain>[^.]+)" offset_field=matchpos | fields email, username, domain, matchpos | head 3 ;
124139 fetched rows / total rows = 3/3
125- +-----------------------+-------------+---------+------------------------+
126- | email | username | domain | matchpos |
127- |-----------------------+-------------|- --------| ------------------------|
128- | amberduke@pyrami.com | amberduke | pyrami | username=0-8,domain=10-15 |
129- | hattiebond@netagy.com | hattiebond | netagy | username=0-9,domain=11-16 |
130- | nanettebates@quility.com | nanettebates | quility | username=0-11,domain=13-19 |
131- +-----------------------+-------------+---------+------------------------+
140+ +-------------------------- +-------------- +---------+----- ------------------------+
141+ | email | username | domain | matchpos |
142+ |-------------------------- +--------------+ ---------+----- ------------------------|
143+ | amberduke@pyrami.com | amberduke | pyrami | username=0-8,domain=10-15 |
144+ | hattiebond@netagy.com | hattiebond | netagy | username=0-9,domain=11-16 |
145+ | nanettebates@quility.com | nanettebates | quility | username=0-11,domain=13-19 |
146+ +-------------------------- +-------------- +---------+----- ------------------------+
132147
133148
134149Example 6: Complex Email Pattern
@@ -140,13 +155,13 @@ PPL query::
140155
141156 os> source=accounts | rex field=email "(?<user>[a-zA-Z0-9._%+-]+)@(?<domain>[a-zA-Z0-9.-]+)\\.(?<tld>[a-zA-Z]{2,})" | fields email, user, domain, tld | head 3 ;
142157 fetched rows / total rows = 3/3
143- +-----------------------+ -------------+---------+-----+
144- | email | user | domain | tld |
145- |-----------------------+-------------|- --------| -----|
146- | amberduke@pyrami.com | amberduke | pyrami | com |
147- | hattiebond@netagy.com | hattiebond | netagy | com |
158+ +--------------------------+- -------------+---------+-----+
159+ | email | user | domain | tld |
160+ |-------------------------- +--------------+ ---------+ -----|
161+ | amberduke@pyrami.com | amberduke | pyrami | com |
162+ | hattiebond@netagy.com | hattiebond | netagy | com |
148163 | nanettebates@quility.com | nanettebates | quility | com |
149- +-----------------------+ -------------+---------+-----+
164+ +--------------------------+- -------------+---------+-----+
150165
151166
152167Example 7: Chaining Multiple rex Commands
0 commit comments