fix: [db] Change attributes.comment column charset to utf8mb4

[brainstorm]

Fix: attributes.comment column cannot store emoji characters

Bug Description

The attributes.comment column uses utf8mb3_unicode_ci collation, which only supports 3-byte UTF-8 characters. When users try to add attribute comments containing emoji characters (especially flag emojis like 🇮 🇪 🇪 🇸 🇫 🇷 ), a PDO exception is thrown.

Error:

SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value:
'\xF0\x9F\x87\xAE\xF0\x9F...' for column `misp`.`attributes`.`comment` at row 1

Note: The SQLSTATE 22007 label "Invalid datetime format" is misleading. This is a character encoding issue, not datetime-related. MySQL reuses this SQLSTATE code for various conversion failures beyond its ANSI SQL datetime meaning.

Full prod backtrace:

2026-03-16 07:10:57 Error: [PDOException] SQLSTATE[22007]: Invalid
datetime format: 1366 Incorrect string value:
'\xF0\x9F\x87\xAE\xF0\x9F...' for column `misp`.`attributes`.`comment` at row 1
Request URL: /attributes/add/19
User somebody@mispsalot.se (User #12) @ 10.0.0.40
Error in:
/var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php,
Line: 502
Stack Trace:
#0 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(502): PDOStatement->execute()
#1 /var/www/MISP/app/Model/Datasource/Database/MysqlObserverExtended.php(71): DboSource->_execute()
#2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(1132): MysqlObserverExtended->execute()
#3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1943): DboSource->create()
#4 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1761): Model->_doSave()
#5 /var/www/MISP/app/Model/MispAttribute.php(575): Model->save()
#6 /var/www/MISP/app/Model/MispAttribute.php(2906): MispAttribute->save()
#7 /var/www/MISP/app/Controller/AttributesController.php(325): MispAttribute->captureAttribute()
#8 [internal function]: AttributesController->add()
#9 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(500): ReflectionMethod->invokeArgs()
#10 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(193): Controller->invokeAction()
#11 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke()
#12 /var/www/MISP/app/webroot/index.php(107): Dispatcher->dispatch()
#13 {main}

Root Cause

1. MySQL's utf8mb3 character set only supports 1-3 byte UTF-8 characters
2. Emoji characters (especially flag emojis) are 4-byte UTF-8 sequences
3. The database connection was also using utf8 (maps to utf8mb3) instead of utf8mb4

Steps to Reproduce

# Get auth key
AUTH_KEY=$(./app/Console/cake User change_authkey 1 | awk '{print $NF}')

# Create event
EVENT_ID=$(curl -sk https://localhost/events/add \
  -H "Authorization: $AUTH_KEY" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -d '{"Event":{"info":"Test","distribution":0,"threat_level_id":4,"analysis":0}}' \
  | jq -r '.Event.id')

# Trigger bug - add attribute with emoji comment
curl -sk "https://localhost/attributes/add/$EVENT_ID" \
  -H "Authorization: $AUTH_KEY" \
  -H "Content-Type: application/json" \
  -d '{"Attribute":{"type":"ip-src","value":"10.0.0.1","comment":"Testing flag: 🇮 🇪 "}}'

# Before fix: PDO exception in error.log
# After fix: Success with emoji stored correctly

Fix

1. Add migration case 147 to convert attributes.comment to utf8mb4_unicode_ci
2. Update database connection encoding from utf8 to utf8mb4

Potential Similar Issues

Other columns that may have the same problem:

Table	Column	Current Charset
shadow_attributes	comment	utf8_bin
event_blacklists	comment	utf8_unicode_ci
org_blacklists	comment	utf8_unicode_ci

These are NOT fixed by this PR but should be considered for future updates.

Testing

After applying the fix, the same curl command succeeds:

{
    "Attribute": {
        "id": "1",
        "comment": "Testing emoji: 🇮 🇪  🇪 🇸  🇫 🇷  and regular emojis 😀 🎉"
    }
}

Questions

• Does it require a DB change?
• Are you using it in production?
• Does it require a change in the API (PyMISP for example)?

[brainstorm]

Hey! @iglocska First pull request here so let me know if I made some obvious mistake: for instance, I'm not too sure if you do database migrations through that case: if other code is needed elsewhere, etc... happy to fix it! ;)