Skip to content

Commit b2bd593

Browse files
ilyannnclaude
ilyannn
and
claude
committed
[citrix_waf] Support RFC5424 syslog header for NetScaler NS14.1+
NetScaler NS14.1 and later prepend an RFC5424 syslog header to both CEF and native messages over TCP/UDP, which the pipeline never stripped. The whole <PRI>VER ... envelope leaked into citrix.detail, so grok_citrix_detail failed, and CEF events were mis-routed to the native sub-pipeline because the <PRI>VER prefix defeats the startsWith("CEF:") check. - default.yml: add an RFC5424 envelope pattern that strips the header so the existing CEF/native routing works, mapping the envelope timestamp to @timestamp and the hostname to observer.hostname. - native.yml: make the NetScaler header optional and reconstruct citrix.device_event_class_id from the RFC5424 APP-NAME for native bodies that omit the module token. Classic NS10/NS11 parsing is unchanged. - Add NS14.1 pipeline test fixtures (CEF + native). Relates to elastic/sdh-beats#7312. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent cf28f96 commit b2bd593

6 files changed

Lines changed: 122 additions & 5 deletions

File tree

‎packages/citrix_waf/changelog.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
11
# newer versions go on top
2+
- version: "1.21.0"
3+
changes:
4+
- description: Support RFC5424 syslog header (NetScaler NS14.1 and later) for CEF and native formats over TCP/UDP.
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/19891
27
- version: "1.20.1"
38
changes:
49
- description: Remove top level note from docs

‎packages/citrix_waf/data_stream/log/_dev/test/pipeline/test-citrix-waf-rfc5424.log‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
<134>1 2026-06-15T10:18:26Z DEVICE APPFW 0-PPE-3 - - default APPFW_STARTURL 18309968 0 : 10.10.10.2 1334598-PPE3 EXAMPLEsessionID0000000000000000000000000000000001== waf_prof_webservers_3 Disallow Illegal URL: https://url.example/Microsoft-Server-ActiveSync?Cmd=Sync&User=username2%40domainname&DeviceId=0123456789ABCDEF0123456789ABCDEF&DeviceType=iPhone <not blocked>
2+
<134>1 2026-06-22T14:34:06Z netscaler01 APPFW 0-PPE-3 - - CEF:0|Citrix|NetScaler|NS14.1|APPFW|APPFW_STARTURL|6|src=10.10.10.2 geolocation=Unknown spt=27699 method=POST request=https://url.local/Microsoft-Server-ActiveSync?Cmd\=Ping&User\=username%40domainname&DeviceId\=FEDCBA9876543210FEDCBA9876543210&DeviceType\=iPhone msg=Disallow Illegal URL. cn1=19960246 cn2=1458719 cs1=waf_prof_webservers_3 cs2=PPE3 cs3=EXAMPLEsessionID0000000000000000000000000000000002== cs4=ALERT cs5=2026 act=not blocked

‎packages/citrix_waf/data_stream/log/_dev/test/pipeline/test-citrix-waf-rfc5424.log-expected.json‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
{
2+
"expected": [
3+
{
4+
"@timestamp": "2026-06-15T10:18:26.000Z",
5+
"citrix": {
6+
"cef_format": false,
7+
"default_class": true,
8+
"detail": "default APPFW_STARTURL 18309968 0 : 10.10.10.2 1334598-PPE3 EXAMPLEsessionID0000000000000000000000000000000001== waf_prof_webservers_3 Disallow Illegal URL: https://url.example/Microsoft-Server-ActiveSync?Cmd=Sync&User=username2%40domainname&DeviceId=0123456789ABCDEF0123456789ABCDEF&DeviceType=iPhone <not blocked>",
9+
"device_event_class_id": "APPFW",
10+
"extended": {
11+
"message": "10.10.10.2 1334598-PPE3 EXAMPLEsessionID0000000000000000000000000000000001== waf_prof_webservers_3 Disallow Illegal URL: https://url.example/Microsoft-Server-ActiveSync?Cmd=Sync&User=username2%40domainname&DeviceId=0123456789ABCDEF0123456789ABCDEF&DeviceType=iPhone <not blocked>"
12+
},
13+
"name": "APPFW_STARTURL"
14+
},
15+
"ecs": {
16+
"version": "8.17.0"
17+
},
18+
"event": {
19+
"id": "18309968",
20+
"original": "<134>1 2026-06-15T10:18:26Z DEVICE APPFW 0-PPE-3 - - default APPFW_STARTURL 18309968 0 : 10.10.10.2 1334598-PPE3 EXAMPLEsessionID0000000000000000000000000000000001== waf_prof_webservers_3 Disallow Illegal URL: https://url.example/Microsoft-Server-ActiveSync?Cmd=Sync&User=username2%40domainname&DeviceId=0123456789ABCDEF0123456789ABCDEF&DeviceType=iPhone <not blocked>",
21+
"severity": 0,
22+
"timezone": "UTC"
23+
},
24+
"observer": {
25+
"hostname": "DEVICE"
26+
}
27+
},
28+
{
29+
"@timestamp": "2026-06-22T14:34:06.000Z",
30+
"citrix": {
31+
"cef_format": true,
32+
"cef_version": "0",
33+
"detail": "CEF:0|Citrix|NetScaler|NS14.1|APPFW|APPFW_STARTURL|6|src=10.10.10.2 geolocation=Unknown spt=27699 method=POST request=https://url.local/Microsoft-Server-ActiveSync?Cmd\\=Ping&User\\=username%40domainname&DeviceId\\=FEDCBA9876543210FEDCBA9876543210&DeviceType\\=iPhone msg=Disallow Illegal URL. cn1=19960246 cn2=1458719 cs1=waf_prof_webservers_3 cs2=PPE3 cs3=EXAMPLEsessionID0000000000000000000000000000000002== cs4=ALERT cs5=2026 act=not blocked",
34+
"device_event_class_id": "APPFW",
35+
"device_product": "NetScaler",
36+
"device_vendor": "Citrix",
37+
"device_version": "NS14.1",
38+
"extended": {
39+
"geolocation": "Unknown"
40+
},
41+
"name": "APPFW_STARTURL",
42+
"ppe_id": "PPE3",
43+
"profile_name": "waf_prof_webservers_3",
44+
"session_id": "EXAMPLEsessionID0000000000000000000000000000000002==",
45+
"severity": "ALERT"
46+
},
47+
"ecs": {
48+
"version": "8.17.0"
49+
},
50+
"event": {
51+
"action": "not blocked",
52+
"id": "19960246",
53+
"original": "<134>1 2026-06-22T14:34:06Z netscaler01 APPFW 0-PPE-3 - - CEF:0|Citrix|NetScaler|NS14.1|APPFW|APPFW_STARTURL|6|src=10.10.10.2 geolocation=Unknown spt=27699 method=POST request=https://url.local/Microsoft-Server-ActiveSync?Cmd\\=Ping&User\\=username%40domainname&DeviceId\\=FEDCBA9876543210FEDCBA9876543210&DeviceType\\=iPhone msg=Disallow Illegal URL. cn1=19960246 cn2=1458719 cs1=waf_prof_webservers_3 cs2=PPE3 cs3=EXAMPLEsessionID0000000000000000000000000000000002== cs4=ALERT cs5=2026 act=not blocked",
54+
"severity": 6,
55+
"timezone": "UTC"
56+
},
57+
"http": {
58+
"request": {
59+
"id": "1458719",
60+
"method": "POST"
61+
}
62+
},
63+
"message": "Disallow Illegal URL.",
64+
"observer": {
65+
"hostname": "netscaler01"
66+
},
67+
"source": {
68+
"ip": "10.10.10.2",
69+
"port": 27699
70+
},
71+
"url": {
72+
"domain": "url.local",
73+
"original": "https://url.local/Microsoft-Server-ActiveSync?Cmd\\=Ping&User\\=username%40domainname&DeviceId\\=FEDCBA9876543210FEDCBA9876543210&DeviceType\\=iPhone",
74+
"path": "/Microsoft-Server-ActiveSync",
75+
"query": "Cmd\\=Ping&User\\=username%40domainname&DeviceId\\=FEDCBA9876543210FEDCBA9876543210&DeviceType\\=iPhone",
76+
"scheme": "https"
77+
}
78+
}
79+
]
80+
}

‎packages/citrix_waf/data_stream/log/elasticsearch/ingest_pipeline/default.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,12 +17,16 @@ processors:
1717
field: event.original
1818
patterns:
1919
- '^%{SYSLOG_TIMESTAMP} %{LEVEL} %{IP:client.ip:ip} %{GREEDYDATA:citrix.detail}'
20+
- '^%{RFC5424_HEADER}%{SPACE}%{GREEDYDATA:citrix.detail}'
2021
- '^%{GREEDYDATA:citrix.detail}'
2122
pattern_definitions:
2223
LEVEL: '<?%{IDENT:citrix.facility:keyword}\.%{IDENT:citrix.priority:keyword}>?'
2324
IDENT: '[a-zA-Z][a-zA-Z0-9]*'
2425
SYSLOG_TIMESTAMP: '(?:%{SYSLOGTIMESTAMP:_tmp.timestamp}|%{TIMESTAMP_ISO8601:_tmp.timestamp8601})'
2526
TIMESTAMP_ISO8601: '%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE:event.timezone}?'
27+
RFC5424_TIMESTAMP: '%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?'
28+
RFC5424_PRI: '<%{NONNEGINT}>'
29+
RFC5424_HEADER: '%{RFC5424_PRI}%{NONNEGINT}%{SPACE}%{RFC5424_TIMESTAMP:_tmp.timestamp8601}%{SPACE}%{SYSLOGHOST:observer.hostname}%{SPACE}%{NOTSPACE:_tmp.rfc5424_appname}%{SPACE}%{NOTSPACE}%{SPACE}%{NOTSPACE}%{SPACE}(?:-|\[%{DATA}\])'
2630
- pipeline:
2731
tag: pipeline_89a9f5e0
2832
name: '{{ IngestPipeline "cef" }}'

‎packages/citrix_waf/data_stream/log/elasticsearch/ingest_pipeline/native.yml‎

Lines changed: 30 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,22 +7,48 @@ processors:
77
value: false
88
- grok:
99
tag: grok_citrix_detail_21fa0653
10-
description: Extract native header and message.
10+
description: Extract native header and message. The header is optional because the RFC5424 syslog envelope (NetScaler NS14.1 and later) is already stripped in the entry pipeline.
1111
field: citrix.detail
1212
patterns:
13-
- '^%{HEADER} : %{GREEDYDATA:_tmp.details} : +"%{GREEDYDATA:citrix.extended.message}"'
14-
- '^%{HEADER} : %{GREEDYDATA:_tmp.details} : +%{GREEDYDATA:citrix.extended.message}'
13+
- '^(?:%{HEADER} : )?%{GREEDYDATA:_tmp.details} : +"%{GREEDYDATA:citrix.extended.message}"'
14+
- '^(?:%{HEADER} : )?%{GREEDYDATA:_tmp.details} : +%{GREEDYDATA:citrix.extended.message}'
1515
pattern_definitions:
1616
HEADER: '(?:<%{NUMBER}>%{SPACE})?%{NATIVE_TIMESTAMP:_tmp.timestamp_native} %{WORD:event.timezone} (?:%{SYSLOGHOST:citrix.host} )?%{INT}-PPE-%{INT}'
1717
NATIVE_TIMESTAMP: '(?:%{MONTHNUM}/%{MONTHDAY}/%{YEAR}|%{YEAR}/%{MONTHNUM}/%{MONTHDAY}):%{HOUR}:%{MINUTE}:%{SECOND}'
1818
- grok:
1919
tag: grok__tmp_details_7fe39004
20-
description: Parse out details.
20+
description: >-
21+
Parse out details for the classic native body
22+
(`default? <module> <event_class> <event_id> <severity>`). Skipped for
23+
RFC5424 events, whose body lacks the leading module token.
2124
field: _tmp.details
25+
if: ctx._tmp?.rfc5424_appname == null
2226
patterns:
2327
- '^%{DEFAULT:_tmp.default}?%{WORD:citrix.device_event_class_id} %{GREEDYDATA:citrix.name} %{INT:event.id} %{INT:event.severity}$'
2428
pattern_definitions:
2529
DEFAULT: 'default '
30+
- grok:
31+
tag: grok__tmp_details_rfc5424_2b8d4f1a
32+
description: >-
33+
Parse out details for the RFC5424 native body
34+
(`default? <event_class> <event_id> <severity>`). The module token is
35+
absent here because it is carried in the RFC5424 APP-NAME, which is
36+
reconstructed into `citrix.device_event_class_id` below.
37+
field: _tmp.details
38+
if: ctx._tmp?.rfc5424_appname != null
39+
patterns:
40+
- '^%{DEFAULT:_tmp.default}?%{GREEDYDATA:citrix.name} %{INT:event.id} %{INT:event.severity}$'
41+
pattern_definitions:
42+
DEFAULT: 'default '
43+
- set:
44+
tag: set_citrix_device_event_class_id_from_rfc5424_appname_5f1c2a7e
45+
description: >-
46+
Reconstruct the device event class ID from the RFC5424 APP-NAME captured
47+
in the entry pipeline, for RFC5424 native events whose body lacks the
48+
leading module token.
49+
field: citrix.device_event_class_id
50+
copy_from: _tmp.rfc5424_appname
51+
if: ctx.citrix?.device_event_class_id == null && ctx._tmp?.rfc5424_appname != null
2652
- set:
2753
tag: set_citrix_default_class_7414b158
2854
field: citrix.default_class

‎packages/citrix_waf/manifest.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
format_version: "3.0.3"
22
name: citrix_waf
33
title: "Citrix Web App Firewall"
4-
version: "1.20.1"
4+
version: "1.21.0"
55
description: Ingest events from Citrix Systems Web App Firewall.
66
type: integration
77
categories:

0 commit comments

Comments
 (0)