Skip to content

Commit da3c4e9

Browse files
ilyannnclaude
ilyannn
and
claude
committed
[citrix_waf] Support RFC5424 syslog header for NetScaler NS14.1+
NetScaler NS14.1 and later prepend an RFC5424 syslog header to both CEF and native messages over TCP/UDP, which the pipeline never stripped. The whole <PRI>VER ... envelope leaked into citrix.detail, so grok_citrix_detail failed, and CEF events were mis-routed to the native sub-pipeline because the <PRI>VER prefix defeats the startsWith("CEF:") check. - default.yml: parse the RFC5424 envelope with the standard SYSLOG5424BASE / SYSLOG5424PRI grok building blocks (matching symantec_endpoint, fortinet, etc.), capturing log.syslog.{priority,version,hostname,appname,procid,msgid} and the envelope timestamp -> @timestamp, and surfacing the appliance hostname as observer.hostname. Stripping the envelope lets the existing CEF/native routing work. - native.yml: make the NetScaler header optional and reconstruct citrix.device_event_class_id from the RFC5424 APP-NAME (log.syslog.appname) for native bodies that omit the module token. Classic NS10/NS11 parsing is unchanged. - Declare the log.syslog.* ECS fields and add NS14.1 pipeline test fixtures (CEF + native). Relates to elastic/sdh-beats#7312. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent cf28f96 commit da3c4e9

8 files changed

Lines changed: 167 additions & 5 deletions

File tree

‎packages/citrix_waf/changelog.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
11
# newer versions go on top
2+
- version: "1.21.0"
3+
changes:
4+
- description: Support RFC5424 syslog header (NetScaler NS14.1 and later) for CEF and native formats over TCP/UDP.
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/19891
27
- version: "1.20.1"
38
changes:
49
- description: Remove top level note from docs

‎packages/citrix_waf/data_stream/log/_dev/test/pipeline/test-citrix-waf-rfc5424.log‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
<134>1 2026-06-15T10:18:26Z DEVICE APPFW 0-PPE-3 - - default APPFW_STARTURL 18309968 0 : 10.10.10.2 1334598-PPE3 EXAMPLEsessionID0000000000000000000000000000000001== waf_prof_webservers_3 Disallow Illegal URL: https://url.example/Microsoft-Server-ActiveSync?Cmd=Sync&User=username2%40domainname&DeviceId=0123456789ABCDEF0123456789ABCDEF&DeviceType=iPhone <not blocked>
2+
<134>1 2026-06-22T14:34:06Z netscaler01 APPFW 0-PPE-3 - - CEF:0|Citrix|NetScaler|NS14.1|APPFW|APPFW_STARTURL|6|src=10.10.10.2 geolocation=Unknown spt=27699 method=POST request=https://url.local/Microsoft-Server-ActiveSync?Cmd\=Ping&User\=username%40domainname&DeviceId\=FEDCBA9876543210FEDCBA9876543210&DeviceType\=iPhone msg=Disallow Illegal URL. cn1=19960246 cn2=1458719 cs1=waf_prof_webservers_3 cs2=PPE3 cs3=EXAMPLEsessionID0000000000000000000000000000000002== cs4=ALERT cs5=2026 act=not blocked

‎packages/citrix_waf/data_stream/log/_dev/test/pipeline/test-citrix-waf-rfc5424.log-expected.json‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
{
2+
"expected": [
3+
{
4+
"@timestamp": "2026-06-15T10:18:26.000Z",
5+
"citrix": {
6+
"cef_format": false,
7+
"default_class": true,
8+
"detail": "default APPFW_STARTURL 18309968 0 : 10.10.10.2 1334598-PPE3 EXAMPLEsessionID0000000000000000000000000000000001== waf_prof_webservers_3 Disallow Illegal URL: https://url.example/Microsoft-Server-ActiveSync?Cmd=Sync&User=username2%40domainname&DeviceId=0123456789ABCDEF0123456789ABCDEF&DeviceType=iPhone <not blocked>",
9+
"device_event_class_id": "APPFW",
10+
"extended": {
11+
"message": "10.10.10.2 1334598-PPE3 EXAMPLEsessionID0000000000000000000000000000000001== waf_prof_webservers_3 Disallow Illegal URL: https://url.example/Microsoft-Server-ActiveSync?Cmd=Sync&User=username2%40domainname&DeviceId=0123456789ABCDEF0123456789ABCDEF&DeviceType=iPhone <not blocked>"
12+
},
13+
"name": "APPFW_STARTURL"
14+
},
15+
"ecs": {
16+
"version": "8.17.0"
17+
},
18+
"event": {
19+
"id": "18309968",
20+
"original": "<134>1 2026-06-15T10:18:26Z DEVICE APPFW 0-PPE-3 - - default APPFW_STARTURL 18309968 0 : 10.10.10.2 1334598-PPE3 EXAMPLEsessionID0000000000000000000000000000000001== waf_prof_webservers_3 Disallow Illegal URL: https://url.example/Microsoft-Server-ActiveSync?Cmd=Sync&User=username2%40domainname&DeviceId=0123456789ABCDEF0123456789ABCDEF&DeviceType=iPhone <not blocked>",
21+
"severity": 0,
22+
"timezone": "Z"
23+
},
24+
"log": {
25+
"syslog": {
26+
"appname": "APPFW",
27+
"hostname": "DEVICE",
28+
"priority": 134,
29+
"procid": "0-PPE-3",
30+
"version": "1"
31+
}
32+
},
33+
"observer": {
34+
"hostname": "DEVICE"
35+
}
36+
},
37+
{
38+
"@timestamp": "2026-06-22T14:34:06.000Z",
39+
"citrix": {
40+
"cef_format": true,
41+
"cef_version": "0",
42+
"detail": "CEF:0|Citrix|NetScaler|NS14.1|APPFW|APPFW_STARTURL|6|src=10.10.10.2 geolocation=Unknown spt=27699 method=POST request=https://url.local/Microsoft-Server-ActiveSync?Cmd\\=Ping&User\\=username%40domainname&DeviceId\\=FEDCBA9876543210FEDCBA9876543210&DeviceType\\=iPhone msg=Disallow Illegal URL. cn1=19960246 cn2=1458719 cs1=waf_prof_webservers_3 cs2=PPE3 cs3=EXAMPLEsessionID0000000000000000000000000000000002== cs4=ALERT cs5=2026 act=not blocked",
43+
"device_event_class_id": "APPFW",
44+
"device_product": "NetScaler",
45+
"device_vendor": "Citrix",
46+
"device_version": "NS14.1",
47+
"extended": {
48+
"geolocation": "Unknown"
49+
},
50+
"name": "APPFW_STARTURL",
51+
"ppe_id": "PPE3",
52+
"profile_name": "waf_prof_webservers_3",
53+
"session_id": "EXAMPLEsessionID0000000000000000000000000000000002==",
54+
"severity": "ALERT"
55+
},
56+
"ecs": {
57+
"version": "8.17.0"
58+
},
59+
"event": {
60+
"action": "not blocked",
61+
"id": "19960246",
62+
"original": "<134>1 2026-06-22T14:34:06Z netscaler01 APPFW 0-PPE-3 - - CEF:0|Citrix|NetScaler|NS14.1|APPFW|APPFW_STARTURL|6|src=10.10.10.2 geolocation=Unknown spt=27699 method=POST request=https://url.local/Microsoft-Server-ActiveSync?Cmd\\=Ping&User\\=username%40domainname&DeviceId\\=FEDCBA9876543210FEDCBA9876543210&DeviceType\\=iPhone msg=Disallow Illegal URL. cn1=19960246 cn2=1458719 cs1=waf_prof_webservers_3 cs2=PPE3 cs3=EXAMPLEsessionID0000000000000000000000000000000002== cs4=ALERT cs5=2026 act=not blocked",
63+
"severity": 6,
64+
"timezone": "Z"
65+
},
66+
"http": {
67+
"request": {
68+
"id": "1458719",
69+
"method": "POST"
70+
}
71+
},
72+
"log": {
73+
"syslog": {
74+
"appname": "APPFW",
75+
"hostname": "netscaler01",
76+
"priority": 134,
77+
"procid": "0-PPE-3",
78+
"version": "1"
79+
}
80+
},
81+
"message": "Disallow Illegal URL.",
82+
"observer": {
83+
"hostname": "netscaler01"
84+
},
85+
"source": {
86+
"ip": "10.10.10.2",
87+
"port": 27699
88+
},
89+
"url": {
90+
"domain": "url.local",
91+
"original": "https://url.local/Microsoft-Server-ActiveSync?Cmd\\=Ping&User\\=username%40domainname&DeviceId\\=FEDCBA9876543210FEDCBA9876543210&DeviceType\\=iPhone",
92+
"path": "/Microsoft-Server-ActiveSync",
93+
"query": "Cmd\\=Ping&User\\=username%40domainname&DeviceId\\=FEDCBA9876543210FEDCBA9876543210&DeviceType\\=iPhone",
94+
"scheme": "https"
95+
}
96+
}
97+
]
98+
}

‎packages/citrix_waf/data_stream/log/elasticsearch/ingest_pipeline/default.yml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,26 @@ processors:
1616
description: Extract header details and message from log line.
1717
field: event.original
1818
patterns:
19+
# Classic BSD syslog framing (NetScaler NS10/NS11).
1920
- '^%{SYSLOG_TIMESTAMP} %{LEVEL} %{IP:client.ip:ip} %{GREEDYDATA:citrix.detail}'
21+
# RFC5424 syslog envelope prepended by NetScaler NS14.1+ over TCP/UDP.
22+
- '^%{SYSLOG5424BASE} +%{GREEDYDATA:citrix.detail}'
2023
- '^%{GREEDYDATA:citrix.detail}'
2124
pattern_definitions:
2225
LEVEL: '<?%{IDENT:citrix.facility:keyword}\.%{IDENT:citrix.priority:keyword}>?'
2326
IDENT: '[a-zA-Z][a-zA-Z0-9]*'
2427
SYSLOG_TIMESTAMP: '(?:%{SYSLOGTIMESTAMP:_tmp.timestamp}|%{TIMESTAMP_ISO8601:_tmp.timestamp8601})'
2528
TIMESTAMP_ISO8601: '%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE:event.timezone}?'
29+
# RFC5424 header: <PRI>VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID STRUCTURED-DATA.
30+
# PROCID/APP-NAME use SYSLOG5424PRINTASCII because NetScaler emits e.g. "0-PPE-3".
31+
SYSLOG5424PRI: '<%{NONNEGINT:log.syslog.priority:long}>'
32+
SYSLOG5424BASE: '%{SYSLOG5424PRI}%{NONNEGINT:log.syslog.version} +(?:-|%{TIMESTAMP_ISO8601:_tmp.timestamp8601}) +(?:-|%{IPORHOST:log.syslog.hostname}) +(?:-|%{SYSLOG5424PRINTASCII:log.syslog.appname}) +(?:-|%{SYSLOG5424PRINTASCII:log.syslog.procid}) +(?:-|%{SYSLOG5424PRINTASCII:log.syslog.msgid}) +(?:-|%{SYSLOG5424SD})?'
33+
- set:
34+
tag: set_observer_hostname_from_log_syslog_hostname_8b1f2c3d
35+
description: Surface the RFC5424 syslog envelope hostname (the NetScaler appliance) as the observer.
36+
field: observer.hostname
37+
copy_from: log.syslog.hostname
38+
if: ctx.log?.syslog?.hostname != null && ctx.observer?.hostname == null
2639
- pipeline:
2740
tag: pipeline_89a9f5e0
2841
name: '{{ IngestPipeline "cef" }}'

‎packages/citrix_waf/data_stream/log/elasticsearch/ingest_pipeline/native.yml‎

Lines changed: 30 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,22 +7,48 @@ processors:
77
value: false
88
- grok:
99
tag: grok_citrix_detail_21fa0653
10-
description: Extract native header and message.
10+
description: Extract native header and message. The header is optional because the RFC5424 syslog envelope (NetScaler NS14.1 and later) is already stripped in the entry pipeline.
1111
field: citrix.detail
1212
patterns:
13-
- '^%{HEADER} : %{GREEDYDATA:_tmp.details} : +"%{GREEDYDATA:citrix.extended.message}"'
14-
- '^%{HEADER} : %{GREEDYDATA:_tmp.details} : +%{GREEDYDATA:citrix.extended.message}'
13+
- '^(?:%{HEADER} : )?%{GREEDYDATA:_tmp.details} : +"%{GREEDYDATA:citrix.extended.message}"'
14+
- '^(?:%{HEADER} : )?%{GREEDYDATA:_tmp.details} : +%{GREEDYDATA:citrix.extended.message}'
1515
pattern_definitions:
1616
HEADER: '(?:<%{NUMBER}>%{SPACE})?%{NATIVE_TIMESTAMP:_tmp.timestamp_native} %{WORD:event.timezone} (?:%{SYSLOGHOST:citrix.host} )?%{INT}-PPE-%{INT}'
1717
NATIVE_TIMESTAMP: '(?:%{MONTHNUM}/%{MONTHDAY}/%{YEAR}|%{YEAR}/%{MONTHNUM}/%{MONTHDAY}):%{HOUR}:%{MINUTE}:%{SECOND}'
1818
- grok:
1919
tag: grok__tmp_details_7fe39004
20-
description: Parse out details.
20+
description: >-
21+
Parse out details for the classic native body
22+
(`default? <module> <event_class> <event_id> <severity>`). Skipped for
23+
RFC5424 events, whose body lacks the leading module token.
2124
field: _tmp.details
25+
if: ctx.log?.syslog?.appname == null
2226
patterns:
2327
- '^%{DEFAULT:_tmp.default}?%{WORD:citrix.device_event_class_id} %{GREEDYDATA:citrix.name} %{INT:event.id} %{INT:event.severity}$'
2428
pattern_definitions:
2529
DEFAULT: 'default '
30+
- grok:
31+
tag: grok__tmp_details_rfc5424_2b8d4f1a
32+
description: >-
33+
Parse out details for the RFC5424 native body
34+
(`default? <event_class> <event_id> <severity>`). The module token is
35+
absent here because it is carried in the RFC5424 APP-NAME, which is
36+
reconstructed into `citrix.device_event_class_id` below.
37+
field: _tmp.details
38+
if: ctx.log?.syslog?.appname != null
39+
patterns:
40+
- '^%{DEFAULT:_tmp.default}?%{GREEDYDATA:citrix.name} %{INT:event.id} %{INT:event.severity}$'
41+
pattern_definitions:
42+
DEFAULT: 'default '
43+
- set:
44+
tag: set_citrix_device_event_class_id_from_rfc5424_appname_5f1c2a7e
45+
description: >-
46+
Reconstruct the device event class ID from the RFC5424 APP-NAME
47+
(log.syslog.appname), for RFC5424 native events whose body lacks the
48+
leading module token.
49+
field: citrix.device_event_class_id
50+
copy_from: log.syslog.appname
51+
if: ctx.citrix?.device_event_class_id == null && ctx.log?.syslog?.appname != null
2652
- set:
2753
tag: set_citrix_default_class_7414b158
2854
field: citrix.default_class

‎packages/citrix_waf/data_stream/log/fields/ecs.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,18 @@
6060
name: log.file.path
6161
- external: ecs
6262
name: log.level
63+
- external: ecs
64+
name: log.syslog.appname
65+
- external: ecs
66+
name: log.syslog.hostname
67+
- external: ecs
68+
name: log.syslog.msgid
69+
- external: ecs
70+
name: log.syslog.priority
71+
- external: ecs
72+
name: log.syslog.procid
73+
- external: ecs
74+
name: log.syslog.version
6375
- external: ecs
6476
name: message
6577
- external: ecs

‎packages/citrix_waf/docs/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -411,6 +411,12 @@ The following fields are exported by this data stream:
411411
| log.level | Original log level of the log event. If the source of the event provides a log level or textual severity, this is the one that goes in `log.level`. If your source doesn't specify one, you may put your event transport's severity here (e.g. Syslog severity). Some examples are `warn`, `err`, `i`, `informational`. | keyword |
412412
| log.offset | Offset of the entry in the log file. | long |
413413
| log.source.address | Source address from which the log event was read / sent from. | keyword |
414+
| log.syslog.appname | The device or application that originated the Syslog message, if available. | keyword |
415+
| log.syslog.hostname | The hostname, FQDN, or IP of the machine that originally sent the Syslog message. This is sourced from the hostname field of the syslog header. Depending on the environment, this value may be different from the host that handled the event, especially if the host handling the events is acting as a collector. | keyword |
416+
| log.syslog.msgid | An identifier for the type of Syslog message, if available. Only applicable for RFC 5424 messages. | keyword |
417+
| log.syslog.priority | Syslog numeric priority of the event, if available. According to RFCs 5424 and 3164, the priority is 8 \* facility + severity. This number is therefore expected to contain a value between 0 and 191. | long |
418+
| log.syslog.procid | The process name or ID that originated the Syslog message, if available. | keyword |
419+
| log.syslog.version | The version of the Syslog protocol specification. Only applicable for RFC 5424 messages. | keyword |
414420
| message | For log events the message field contains the log message, optimized for viewing in a log viewer. For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event. If multiple messages exist, they can be combined into one message. | match_only_text |
415421
| network.bytes | Total bytes transferred in both directions. If `source.bytes` and `destination.bytes` are known, `network.bytes` is their sum. | long |
416422
| network.community_id | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. Learn more at https://github.com/corelight/community-id-spec. | keyword |

‎packages/citrix_waf/manifest.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
format_version: "3.0.3"
22
name: citrix_waf
33
title: "Citrix Web App Firewall"
4-
version: "1.20.1"
4+
version: "1.21.0"
55
description: Ingest events from Citrix Systems Web App Firewall.
66
type: integration
77
categories:

0 commit comments

Comments
 (0)