Skip to content

Commit f2675b2

Browse files
patrickbirchpatrickbirch
authored
Merge pull request #694 from percona/ps-alf-8.4
Update the audit log filter 8.4
2 parents 2cc34fe + 0bc8d4a commit f2675b2

4 files changed

Lines changed: 131 additions & 66 deletions

File tree

‎docs/audit-log-filter-formats.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
When an auditable event occurs, the component writes a record to the log file.
44

5-
After the component starts, the first record lists the description of the server and the options at startup. After the first record, the auditable events are connections, disconnections, SQL statements executed, and so on. Statements within stored procedures or triggers are not logged, only the top-level statements.
5+
After the component starts, the first record marks audit logging start. With `audit_log_filter.format=NEW`, from Percona Server for MySQL 8.4.8-8 onward on the {{vers}} line (this docs build: {{release}}), that record’s `<NAME>` is `Startup` and the record includes `SERVER_ID` and `COMMAND_CLASS` (among the mandatory elements). Later records cover connections, disconnections, SQL statements executed, and so on. Statements within stored procedures or triggers are not logged, only the top-level statements. See [XML (new style)](audit-log-filter-new.md) for the full field list.
66

77
If files are referenced by `LOAD_DATA`, the contents are not logged.
88

‎docs/audit-log-filter-new.md‎

Lines changed: 124 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -1,47 +1,69 @@
11
# Audit Log Filter format - XML (new style)
22

3-
The filter writes the audit log filter file in XML. The XML file uses
4-
UTF-8.
3+
Starting with Percona Server for MySQL 8.4.8-8 on the {{vers}} line, the following describes new-style XML (`audit_log_filter.format=NEW`) from the Audit Log Filter component: element names, typical fields, and formatter behavior were aligned to the server for that release. This documentation build targets {{release}} (bump on each publish). If you run an older {{vers}} build than 8.4.8-8, verify against your own audit log in case output differs. Implementation reference: `components/audit_log_filter/log_record_formatter/new.cc` and `base.cc`.
54

6-
The <AUDIT> is the root element and this element contains
7-
&lt;AUDIT_RECORD&gt; elements. Each &lt;AUDIT_RECORD&gt; element contains specific
8-
information about an event that is audited.
5+
The Audit Log Filter component can write the audit log file as new-style XML
6+
(`audit_log_filter.format=NEW`). The file uses UTF-8.
97

10-
For each new file, the Audit Log Filter component writes the XML
11-
declaration and the root element tag. The component writes the closing
12-
</AUDIT> root element when closing the file. If the file is open, this
13-
closing element is not available.
8+
The root element is `<AUDIT>`. It contains `<AUDIT_RECORD>` elements. Each
9+
`<AUDIT_RECORD>` describes one audited event.
1410

15-
```xml
11+
For each new file, the component writes the XML declaration and the opening
12+
`<AUDIT>` tag. When the file is closed, the component writes the closing
13+
`</AUDIT>` tag. If the file is still open, that closing tag is not present yet.
14+
15+
Element order inside `<AUDIT_RECORD>` is not guaranteed (the writer may
16+
emit fields in a fixed order in practice, but consumers should not depend on it).
17+
18+
Timestamps use the server local time zone, in `YYYY-MM-DDTHH:MM:SS`
19+
form. They do not append a ` UTC` suffix to the timestamp string.
20+
21+
!!! note "NEW XML behavior (from 8.4.8-8, docs {{release}})"
22+
23+
Audit logging on/off is recorded with `<NAME>` values `Startup` and
24+
`Shutdown`. The NEW formatter does not emit `STATUS_CODE`, and
25+
does not write `VERSION`, `STARTUP_OPTIONS`, `MYSQL_VERSION`, or
26+
`OS_VERSION` on the startup or shutdown audit record. Disconnect events
27+
use the name `Disconnect`.
28+
29+
## Example (illustrative)
1630

31+
The snippet below shows the shape of several record types. Exact sets of
32+
elements depend on the event, filters, and server configuration.
33+
34+
```xml
1735
<?xml version="1.0" encoding="utf-8"?>
1836
<AUDIT>
1937
<AUDIT_RECORD>
20-
<NAME>Audit</NAME>
38+
<NAME>Startup</NAME>
2139
<RECORD_ID>0_2023-03-29T11:11:43</RECORD_ID>
2240
<TIMESTAMP>2023-03-29T11:11:43</TIMESTAMP>
41+
<COMMAND_CLASS>Audit</COMMAND_CLASS>
2342
<SERVER_ID>1</SERVER_ID>
2443
</AUDIT_RECORD>
2544
<AUDIT_RECORD>
26-
<NAME>Command Start</NAME>
27-
<RECORD_ID>1_2023-03-29T11:11:45</RECORD_ID>
28-
<TIMESTAMP>2023-03-29T11:11:45</TIMESTAMP>
45+
<NAME>Connect</NAME>
46+
<RECORD_ID>1_2023-03-29T11:11:44</RECORD_ID>
47+
<TIMESTAMP>2023-03-29T11:11:44</TIMESTAMP>
48+
<COMMAND_CLASS>Connection</COMMAND_CLASS>
49+
<CONNECTION_ID>11</CONNECTION_ID>
50+
<HOST>localhost</HOST>
51+
<IP>127.0.0.1</IP>
52+
<USER>root</USER>
53+
<OS_LOGIN></OS_LOGIN>
54+
<PRIV_USER>root</PRIV_USER>
55+
<PROXY_USER></PROXY_USER>
56+
<DB>test</DB>
2957
<STATUS>0</STATUS>
30-
<CONNECTION_ID>1</CONNECTION_ID>
31-
<COMMAND_CLASS>query</COMMAND_CLASS>
58+
<CONNECTION_TYPE>TCP/IP</CONNECTION_TYPE>
3259
</AUDIT_RECORD>
3360
<AUDIT_RECORD>
34-
<NAME>Query</NAME>
61+
<NAME>Command Start</NAME>
3562
<RECORD_ID>2_2023-03-29T11:11:45</RECORD_ID>
3663
<TIMESTAMP>2023-03-29T11:11:45</TIMESTAMP>
37-
<COMMAND_CLASS>create_table</COMMAND_CLASS>
38-
<CONNECTION_ID>11</CONNECTION_ID>
39-
<HOST>localhost</HOST>
40-
<IP></IP>
41-
<USER>root[root] @ localhost []</USER>
42-
<OS_LOGIN></OS_LOGIN>
43-
<SQLTEXT>CREATE TABLE t1 (c1 INT)</SQLTEXT>
4464
<STATUS>0</STATUS>
65+
<CONNECTION_ID>1</CONNECTION_ID>
66+
<COMMAND_CLASS>query</COMMAND_CLASS>
4567
</AUDIT_RECORD>
4668
<AUDIT_RECORD>
4769
<NAME>Query Start</NAME>
@@ -53,17 +75,13 @@ closing element is not available.
5375
<SQLTEXT>CREATE TABLE t1 (c1 INT)</SQLTEXT>
5476
</AUDIT_RECORD>
5577
<AUDIT_RECORD>
56-
<NAME>Query</NAME>
78+
<NAME>Query Status End</NAME>
5779
<RECORD_ID>4_2023-03-29T11:11:45</RECORD_ID>
5880
<TIMESTAMP>2023-03-29T11:11:45</TIMESTAMP>
59-
<COMMAND_CLASS>create_table</COMMAND_CLASS>
81+
<STATUS>0</STATUS>
6082
<CONNECTION_ID>11</CONNECTION_ID>
61-
<HOST>localhost</HOST>
62-
<IP></IP>
63-
<USER>root[root] @ localhost []</USER>
64-
<OS_LOGIN></OS_LOGIN>
83+
<COMMAND_CLASS>create_table</COMMAND_CLASS>
6584
<SQLTEXT>CREATE TABLE t1 (c1 INT)</SQLTEXT>
66-
<STATUS>0</STATUS>
6785
</AUDIT_RECORD>
6886
<AUDIT_RECORD>
6987
<NAME>Command End</NAME>
@@ -73,40 +91,81 @@ closing element is not available.
7391
<CONNECTION_ID>1</CONNECTION_ID>
7492
<COMMAND_CLASS>query</COMMAND_CLASS>
7593
</AUDIT_RECORD>
94+
<AUDIT_RECORD>
95+
<NAME>Disconnect</NAME>
96+
<RECORD_ID>6_2023-03-29T11:11:50</RECORD_ID>
97+
<TIMESTAMP>2023-03-29T11:11:50</TIMESTAMP>
98+
<COMMAND_CLASS>Connection</COMMAND_CLASS>
99+
<CONNECTION_ID>11</CONNECTION_ID>
100+
<HOST>localhost</HOST>
101+
<IP>127.0.0.1</IP>
102+
<USER>root</USER>
103+
<OS_LOGIN></OS_LOGIN>
104+
<PRIV_USER>root</PRIV_USER>
105+
<PROXY_USER></PROXY_USER>
106+
<DB>test</DB>
107+
<STATUS>0</STATUS>
108+
<CONNECTION_TYPE>TCP/IP</CONNECTION_TYPE>
109+
</AUDIT_RECORD>
110+
<AUDIT_RECORD>
111+
<NAME>Shutdown</NAME>
112+
<RECORD_ID>7_2023-03-29T11:12:00</RECORD_ID>
113+
<TIMESTAMP>2023-03-29T11:12:00</TIMESTAMP>
114+
<COMMAND_CLASS>Audit</COMMAND_CLASS>
115+
<SERVER_ID>1</SERVER_ID>
116+
</AUDIT_RECORD>
76117
</AUDIT>
77118
```
78119

79-
The order of the attributes within an &lt;AUDIT_RECORD&gt; can vary. Certain attributes are in every element. Other attributes are optional and depend on the type of audit record.
80-
81-
The attributes in every element are the following:
82-
83-
| Attribute Name | Description |
84-
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
85-
| `<NAME>` | The action that generated the audit record. |
86-
| `<RECORD_ID>` | The `<RECORD_ID>` consists of a sequence number and a timestamp value. The sequence number is initialized when the component opens the audit log filter file. |
87-
| `<TIMESTAMP>` | Displays the date and time when the audit event happened. |
88-
89-
The optional attributes are the following:
90-
91-
| Attribute Name | Description |
92-
| ----------------------- | ---------------------- |
93-
| `<COMMAND_CLASS>` | Contains the type of performed action. |
94-
| `<CONNECTION_ID>` | Contains the client connection identifier. |
95-
| `<CONNECTION_ATTRIBUTES>`| Contains the client connection attributes. Each attribute has a `<NAME>` and `<VALUE>` pair. |
96-
| `<CONNECTION_TYPE>` | Contains the type of connection security. |
97-
| `<DB>` | Contains the database name. |
98-
| `<HOST>` | Contains the client's hostname. |
99-
| `<IP>` | Contains the client's IP address. |
100-
| `<MYSQL_VERSION>` | Contains the MySQL server version. |
101-
| `<OS_LOGIN>` | Contains the user name used during an external authentication, for example, if the user is authenticated through an LDAP component. If the authentication component does not set a value or the user is authenticated using MySQL authentication, this value is empty. |
102-
| `<OS_VERSION>` | Contains the server's operating system. |
103-
| `<PRIV_USER>` | Contains the user name used by the server when checking privileges. This name may be different than `<USER>`. |
104-
| `<PROXY_USER>` | Contains the proxy user. If a proxy is not used, the value is empty. |
105-
| `<SERVER_ID>` | Contains the server ID. |
106-
| `<SQLTEXT>` | Contains the text of the SQL statement. |
107-
| `<STARTUP_OPTIONS>` | Contains the startup options. These options may be provided by the command line or files. |
108-
| `<STATUS>` | Contains the status of a command. A 0 (zero) is a success. A nonzero value is an error. |
109-
| `<STATUS_CODE>` | Contains the status of a command, which either succeeds (0) or an error occurred (1). |
110-
| `<TABLE>` | Contains the table name. |
111-
| `<USER>` | Contains the user name from the client. This name may be different than `<PRIV_USER>`. |
112-
| `<VERSION>` | Contains the audit log filter format. |
120+
Query-class events (`Query Start`, `Query Status End`, nested variants, and
121+
so on) include `STATUS`, `CONNECTION_ID`, `COMMAND_CLASS` (SQL
122+
command name from the event), and often `SQLTEXT` (or digest text from
123+
extended info). They do not include `HOST`, `IP`, `USER`, or
124+
`OS_LOGIN` in NEW XML from 8.4.8-8 onward—those appear on connection
125+
records (and on general records, which use subclasses such as `Log`,
126+
`Error`, `Result`, `Status`, not the string `Query`).
127+
128+
Connection records use `COMMAND_CLASS` with the value `Connection`
129+
(the event class label).
130+
131+
If the client supplies connection attributes and the event carries them,
132+
`CONNECTION_ATTRIBUTES` holds one `ATTRIBUTE` per attribute, each with a
133+
`NAME` and `VALUE` child element.
134+
135+
## Mandatory elements
136+
137+
These appear on every `<AUDIT_RECORD>` in this format:
138+
139+
| Element | Description |
140+
| --- | --- |
141+
| `<NAME>` | Event subclass string (for example `Startup`, `Connect`, `Query Start`, `TableRead`). |
142+
| `<RECORD_ID>` | Sequence number and timestamp (see [`audit_log_filter` file handling](reading-audit-log-filter-files.md)); format `SEQ_TIMESTAMP` where the timestamp part matches the formatter’s timestamp string. |
143+
| `<TIMESTAMP>` | Local date and time for the event. |
144+
145+
## Optional elements (by record category)
146+
147+
Many elements appear only for specific event classes. The following table lists
148+
elements used by the NEW XML formatter from Percona Server for MySQL 8.4.8-8 onward for at least one
149+
event type. It is not a promise that every field appears in every record.
150+
151+
| Element | Description |
152+
| --- | --- |
153+
| `<COMMAND_CLASS>` | Meaning depends on the record: connection events use `Connection`; table-access events use `Table Access`; command events use the `COM_*` command text (`query`, and so on); query events use the SQL command name (for example `select`, `create_table`); general events use `General`. |
154+
| `<CONNECTION_ID>` | Client connection ID. |
155+
| `<CONNECTION_ATTRIBUTES>` | Nested `ATTRIBUTE` elements, each with `NAME` and `VALUE`. Omitted if there are no attributes. |
156+
| `<CONNECTION_TYPE>` | Connection security / transport (for example `TCP/IP`, `SSL`, `Socket`). |
157+
| `<STATUS>` | Status code for the event (for `Query` / `Command` / connection records, `0` success and non-zero for failure where applicable). |
158+
| `<SQLTEXT>` | Statement or digest text when the event carries SQL text. |
159+
| `<HOST>`, `<IP>`, `<USER>` | Client context on connection and general records (and on authentication records where applicable). Not emitted on `Query Start` / `Query Status End` style records in NEW XML. |
160+
| `<OS_LOGIN>` | External user from authentication (`external_user`); on connection records from 8.4.8-8 onward (documented behavior). |
161+
| `<PRIV_USER>`, `<PROXY_USER>`, `<DB>` | Included on connection records (including disconnect) from 8.4.8-8 onward (documented behavior). |
162+
| `<SERVER_ID>` | On `Startup`, `Shutdown`, and similar audit records. |
163+
| `<DB>`, `<TABLE>` | Database and table name on table-access records (`TableRead`, `TableInsert`, …). |
164+
| `<VARIABLE_NAME>`, `<VARIABLE_VALUE>` | Global variable audit events. |
165+
| `<STORED_PROGRAM>` | Stored program events (`DB` also appears). |
166+
| `<FLAGS>`, `<REWRITTEN_QUERY>` | Parse events (`SQLTEXT` may appear). |
167+
| `<COMPONENT>`, `<PRODUCER>`, `<MESSAGE>`, `<MESSAGE_ATTRIBUTES>` | Message events (attributes use the same `ATTRIBUTE` / `NAME` / `VALUE` pattern as connection attributes). |
168+
169+
Characters such as `<`, `>`, `&`, and `"` in element text are XML-escaped by
170+
the component. Very long values may be truncated according to server-side
171+
limits.

‎docs/audit-log-filter-overview.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ The Audit Log Filter component allows you to monitor, log, and block a connectio
44

55
Enabling the component produces a log file that contains a record of server activity. The log file has information on connections and databases accessed by that connection.
66

7+
Set [`audit_log_filter.format`](audit-log-filter-variables.md#audit_log_filterformat) at startup to choose NEW (default), OLD, or JSON output. For new-style XML, the description in this documentation is aligned to the server from Percona Server for MySQL 8.4.8-8 onward on the {{vers}} line (this docs build: {{release}})—for example audit logging on/off uses `Startup` / `Shutdown`, disconnect events use `Disconnect`, and the NEW formatter does not emit `STATUS_CODE` or fields such as `VERSION`, `STARTUP_OPTIONS`, `MYSQL_VERSION`, and `OS_VERSION` on the audit lifecycle records. See [Audit Log Filter file format overview](audit-log-filter-formats.md) and [XML (new style)](audit-log-filter-new.md).
8+
79
The component uses the `mysql` system database to store filter and user account data. Set the [`audit_log_filter.database`](audit-log-filter-variables.md#audit_log_filterdatabase) variable at server startup to select a different database.
810

911
The `AUDIT_ADMIN` privilege is required to enable users to manage the Audit Log Filter component.

‎docs/filter-audit-log-filter-files.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ The audit filter log filtering is based on rules. The filter rule definition has
77
* Audit event subclass
88
* Audit event fields (for example, `COMMAND_CLASS` or `STATUS`)
99

10+
From Percona Server for MySQL 8.4.8-8: The NEW XML formatter (`audit_log_filter.format=NEW`) behavior described here—including which `<NAME>` strings and child elements appear in each `<AUDIT_RECORD>`—was verified for that release on the {{vers}} line. This documentation build is {{release}}. If your server predates 8.4.8-8, or you use a build whose audit code differs, compare against a real log from your server.
11+
12+
When you inspect new-style XML logs for that line of releases, expect values such as `Startup`, `Shutdown`, `Disconnect`, `Query Start`, `Query Status End`, and `Connection` as the `COMMAND_CLASS` on connect and disconnect. For the full field list per event type, see [XML (new style)](audit-log-filter-new.md).
13+
1014
You can define multiple filters and assign any filter to multiple accounts. You can also create a default filter for specific user accounts. The filters are defined using function calls. After the filter is defined, the filter is stored in `mysql` system tables.
1115

1216
## Audit Log Filter functions

0 commit comments

Comments
 (0)