Skip to content

Commit d3ddba4

Browse files
inikeppatrickbirch
inikep
authored and
patrickbirch
committed
Improve and fix misc issues
1 parent a8c5ec3 commit d3ddba4

23 files changed

Lines changed: 555 additions & 239 deletions

‎docs/_static/audit-data-flow.png‎

7.45 MB
Loading

‎docs/_static/audit-filter-hierarchy.png‎

5.37 MB
Loading

‎docs/_static/audit-log-filter-arch.png‎

6.39 MB
Loading

‎docs/_static/audit-log-filter-modification-lifecycle.png‎

6.32 MB
Loading

‎docs/_static/audit-logging-strategy.png‎

5.79 MB
Loading

‎docs/audit-log-filter-definition-fields.md‎

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
# Audit Log Filter definition fields
2+
3+
This page lists the class, event, and field names accepted when you validate filter definitions with [`audit_log_filter_set_filter()`](audit-log-filter-variables.md#audit_log_filter_set_filterfilter_name-definition). It mirrors the reference in the Percona Server source tree: [filter_definition_fields.md](https://github.com/percona/percona-server/blob/8.4/components/audit_log_filter/filter_definition_fields.md).
4+
5+
[`audit_log_filter.event_mode`](audit-log-filter-variables.md#audit_log_filterevent_mode) determines which of the sections below participate in validation; the Notes spell out `REDUCED` versus `FULL`.
6+
7+
## Notes
8+
9+
* The names below are filter-definition names. They are not necessarily the same as keys used by the JSON log formatter.
10+
* Field type is the type accepted by the current validator in the server’s `get_event_field_value_type()` implementation.
11+
* Some numeric-looking fields are validated as `string` because they are not explicitly typed in `get_event_field_value_type()`.
12+
* Filter-definition validation only accepts the class names documented in the following sections.
13+
* When [`audit_log_filter.event_mode`](audit-log-filter-variables.md#audit_log_filterevent_mode) is `REDUCED` (the default from 8.4.9-9 onward), only these events are tracked and accepted by filter-definition validation:
14+
* `general`: `status`
15+
* `connection`: `connect`, `disconnect`, `change_user`
16+
* `table_access`: `read`, `insert`, `update`, `delete`
17+
* `message`: `internal`, `user`
18+
19+
In `REDUCED` mode, class names that only exist for extended auditing (`global_variable`, `command`, `query`, `stored_program`, `authentication`, `parse`) are rejected entirely. Subclass names that are not in the list above (for example `general/log`, `connection/pre_authenticate`) are also rejected during filter validation. At runtime, events outside the `REDUCED` set are silently skipped. With `FULL`, those six classes and their subclasses are valid filter targets.
20+
* Lifecycle-related records with class names `audit`, `server_startup`, and `server_shutdown` are not valid filter-definition targets. Startup and shutdown lifecycle events are ignored by the audit log filter if they are received.
21+
* For `connection.connection_type`, the validator accepts numeric values `0`–`5` and the pseudo-constants `::undefined`, `::tcp/ip`, `::socket`, `::named_pipe`, `::ssl`, and `::shared_memory`.
22+
23+
## `general`
24+
25+
Supported events: `log`, `error`, `result`, `status`
26+
REDUCED mode: only `status`
27+
28+
| Field name | Field type | Description |
29+
| --- | --- | --- |
30+
| `general_error_code` | integer | Event error code. |
31+
| `general_thread_id` | unsigned integer | Event thread ID. Currently an alias of `general_connection_id`. |
32+
| `general_connection_id` | unsigned integer | Event connection ID. |
33+
| `general_user.str` | string | User name recorded for the general event. |
34+
| `general_user.length` | unsigned integer | User name length. |
35+
| `general_command.str` | string | General command text, for example `Query`. |
36+
| `general_command.length` | unsigned integer | General command text length. |
37+
| `general_query.str` | string | SQL statement text associated with the event. |
38+
| `general_query.length` | unsigned integer | SQL statement text length. |
39+
| `general_host.str` | string | Client host name. |
40+
| `general_host.length` | unsigned integer | Client host name length. |
41+
| `general_sql_command.str` | string | SQL command name associated with the statement, for example `select`. |
42+
| `general_sql_command.length` | unsigned integer | SQL command name length. |
43+
| `general_external_user.str` | string | External user or OS login associated with the event. |
44+
| `general_external_user.length` | unsigned integer | External user or OS login length. |
45+
| `general_ip.str` | string | Client IP address. |
46+
| `general_ip.length` | unsigned integer | Client IP address length. |
47+
48+
## `connection`
49+
50+
Supported events: `connect`, `disconnect`, `change_user`, `pre_authenticate`
51+
REDUCED mode: `connect`, `disconnect`, `change_user`
52+
53+
| Field name | Field type | Description |
54+
| --- | --- | --- |
55+
| `status` | integer | Current connection event status. |
56+
| `connection_id` | unsigned integer | Connection ID. |
57+
| `user.str` | string | User name of this connection. |
58+
| `user.length` | unsigned integer | User name length. |
59+
| `priv_user.str` | string | Privileged user name. |
60+
| `priv_user.length` | unsigned integer | Privileged user name length. |
61+
| `external_user.str` | string | External user name or OS login. |
62+
| `external_user.length` | unsigned integer | External user name length. |
63+
| `proxy_user.str` | string | Proxy user used for the connection. |
64+
| `proxy_user.length` | unsigned integer | Proxy user name length. |
65+
| `host.str` | string | Connection host name. |
66+
| `host.length` | unsigned integer | Connection host name length. |
67+
| `ip.str` | string | Connection IP address. |
68+
| `ip.length` | unsigned integer | Connection IP address length. |
69+
| `database.str` | string | Default database specified at connection time. |
70+
| `database.length` | unsigned integer | Default database name length. |
71+
| `connection_type` | integer | Connection type code (numeric `0`–`5` or pseudo-constant; see below). |
72+
73+
`connection_type` values:
74+
75+
| Value | Meaning |
76+
| --- | --- |
77+
| `0` or `::undefined` | Undefined |
78+
| `1` or `::tcp/ip` | TCP/IP |
79+
| `2` or `::socket` | Socket |
80+
| `3` or `::named_pipe` | Named pipe |
81+
| `4` or `::ssl` | TCP/IP with encryption |
82+
| `5` or `::shared_memory` | Shared memory |
83+
84+
## `table_access`
85+
86+
Supported events: `read`, `insert`, `update`, `delete`
87+
REDUCED mode: all events
88+
89+
| Field name | Field type | Description |
90+
| --- | --- | --- |
91+
| `connection_id` | unsigned integer | Event connection ID. |
92+
| `sql_command_id` | integer | SQL command ID. |
93+
| `query.str` | string | SQL statement text. |
94+
| `query.length` | unsigned integer | SQL statement text length. |
95+
| `table_database.str` | string | Database name associated with event. |
96+
| `table_database.length` | unsigned integer | Database name length. |
97+
| `table_name.str` | string | Table name associated with event. |
98+
| `table_name.length` | unsigned integer | Table name length. |
99+
100+
## `global_variable` *(FULL mode only)*
101+
102+
Supported events: `get`, `set`
103+
104+
| Field name | Field type | Description |
105+
| --- | --- | --- |
106+
| `connection_id` | string | Event connection ID. |
107+
| `variable_name.str` | string | Variable name. |
108+
| `variable_name.length` | string | Variable name length. |
109+
| `variable_value.str` | string | Variable value. |
110+
| `variable_value.length` | string | Variable value length. |
111+
112+
## `command` *(FULL mode only)*
113+
114+
Supported events: `start`, `end`
115+
116+
| Field name | Field type | Description |
117+
| --- | --- | --- |
118+
| `status` | string | Command event status code. |
119+
| `connection_id` | string | Event connection ID. |
120+
| `command.str` | string | Command text. |
121+
| `command.length` | string | Command text length. |
122+
123+
## `query` *(FULL mode only)*
124+
125+
Supported events: `start`, `nested_start`, `status_end`, `nested_status_end`
126+
127+
| Field name | Field type | Description |
128+
| --- | --- | --- |
129+
| `status` | string | Query event status code. |
130+
| `connection_id` | string | Event connection ID. |
131+
| `sql_command_id` | string | SQL command string associated with the query event. The field name is retained as `sql_command_id` for compatibility. |
132+
| `query.str` | string | SQL query text. |
133+
| `query.length` | string | SQL query text length. |
134+
| `query_charset` | string | SQL query character set name. |
135+
136+
## `stored_program` *(FULL mode only)*
137+
138+
Supported events: `execute`
139+
140+
| Field name | Field type | Description |
141+
| --- | --- | --- |
142+
| `connection_id` | string | Event connection ID. |
143+
| `database.str` | string | Database where the stored program is defined. |
144+
| `database.length` | string | Database name length. |
145+
| `name.str` | string | Stored program name. |
146+
| `name.length` | string | Stored program name length. |
147+
148+
## `authentication` *(FULL mode only)*
149+
150+
Supported events: `flush`, `authid_create`, `credential_change`, `authid_rename`, `authid_drop`
151+
152+
| Field name | Field type | Description |
153+
| --- | --- | --- |
154+
| `status` | string | Authentication event status. |
155+
| `connection_id` | string | Event connection ID. |
156+
| `user.str` | string | User name. |
157+
| `user.length` | string | User name length. |
158+
| `host.str` | string | Host name. |
159+
| `host.length` | string | Host name length. |
160+
161+
## `message`
162+
163+
Supported events: `internal`, `user`
164+
REDUCED mode: all events
165+
166+
| Field name | Field type | Description |
167+
| --- | --- | --- |
168+
| `connection_id` | string | Event connection ID. |
169+
| `component.str` | string | Component name. |
170+
| `component.length` | string | Component name length. |
171+
| `producer.str` | string | Message producer name. |
172+
| `producer.length` | string | Message producer name length. |
173+
| `message.str` | string | Message text. |
174+
| `message.length` | string | Message text length. |
175+
176+
## `parse` *(FULL mode only)*
177+
178+
Supported events: `preparse`, `postparse`
179+
180+
| Field name | Field type | Description |
181+
| --- | --- | --- |
182+
| `connection_id` | string | Event connection ID. |
183+
| `flags` | string | Parse rewrite flags value. |
184+
| `query.str` | string | Original SQL query text. |
185+
| `query.length` | string | Original SQL query text length. |
186+
| `rewritten_query.str` | string | Rewritten SQL query text. |
187+
| `rewritten_query.length` | string | Rewritten SQL query text length. |
188+
189+
## See also
190+
191+
* [Write audit_log_filter definitions](write-filter-definitions.md)
192+
* [Audit Log Filter restrictions](audit-log-filter-restrictions.md)
193+
* [Audit log filter functions, options, and variables](audit-log-filter-variables.md)

‎docs/audit-log-filter-formats.md‎

Lines changed: 12 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,24 @@
11
# Audit Log Filter file format overview
22

3-
When an auditable event occurs, the component writes a record to the log file.
3+
When an auditable event occurs, the component writes a record to the log file. After the component starts, the first record describes the server and the options at startup. After the first record, auditable events include connections, disconnections, executed SQL statements, and so on.
44

5-
After the component starts, the first record lists the description of the server and the options at startup. After the first record, the auditable events are connections, disconnections, SQL statements executed, and so on. Statements within stored procedures or triggers are not logged, only the top-level statements.
5+
Which server events produce audit records depends on [`audit_log_filter.event_mode`](audit-log-filter-variables.md#audit_log_filterevent_mode), not on the log file format. See that variable for `REDUCED` versus `FULL` and for releases before the variable existed.
66

7-
If files are referenced by `LOAD_DATA`, the contents are not logged.
7+
If `LOAD_DATA` references files, the component does not log the file contents.
88

9-
Set with the `audit_log_filter.format` system variable at startup. The available format types are the following;
9+
Set the format with the `audit_log_filter.format` system variable at startup. The available format types are the following:
1010

1111
| Format Type | Command | Description |
1212
|---|---|---|
1313
| [XML (new style)](audit-log-filter-new.md) | `audit_log_filter.format=NEW` | The default format |
1414
| [XML (old style)](audit-log-filter-old.md) | `audit_log_filter.format=OLD` | The original version of the XML format |
15-
| [JSON](audit-log-filter-json.md) | `audit_log_filter.format=JSON` | Files written as a JSON array |
16-
| [JSONL](audit-log-filter-json.md) | `audit_log_filter.format=JSONL` | Introduced in Percona Server for MySQL 8.4.9-9. Each event is a single compact JSON object on its own line inside a wrapping JSON array (the file is still valid JSON). Easy to process with `grep`, `jq`, `wc -l`, streaming pipelines, and log aggregation systems. |
15+
| [JSON](audit-log-filter-json.md) | `audit_log_filter.format=JSON` | The JSON format writes the audit log as a JSON array |
16+
| [JSONL](audit-log-filter-json.md) | `audit_log_filter.format=JSONL` | Added in 8.4.9-9. One compact JSON object per line inside a wrapping array (details in the JSON/JSONL topic). |
1717

18-
By default, the file contents in the new-style XML format are not compressed or encrypted.
19-
20-
Changing the `audit_log_filter.format`, you should also change
21-
the `audit_log_filter.file` name. For example, changing the `audit_log_filter.format`
22-
to JSON or JSONL, change the `audit_log_filter.file` to `audit.json` or `audit.jsonl` respectively. If you don't change
23-
the `audit_log_filter.file` name, then all audit log filter files have the same
24-
base name and you won't be able to easily find when the format changed.
18+
By default, the component does not compress or encrypt new-style XML log files.
2519

20+
When you change the `audit_log_filter.format`, you should also change
21+
the `audit_log_filter.file` name. For example, when you change the `audit_log_filter.format`
22+
to JSON or JSONL, change the `audit_log_filter.file` to `audit.json` or `audit.jsonl` respectively. If you don't change
23+
the `audit_log_filter.file` name, then all audit log filter files share the same
24+
base name and you won't easily see when the format changed.

0 commit comments

Comments
 (0)