eucjp_2004_encoding.phpt fails on macOS in release builds

[iluuu1994]

Description

https://github.com/php/php-src/actions/runs/20119044115/job/57735104285

Given this succeeds in debug builds, this might be:

• Undefined behavior in php-src
• A miscompilation in (Apple) Clang

UBSan did not report any issues, though that doesn't mean much.

Reduced excerpt

006+ mb_check_encoding failed on good EUC-JP-2004 string: (8ff4df8ffed5afe0c7e6cfc5edd8e8cdcbbfdef0d6ee)
007+ mb_convert_encoding not working on EUC-JP-2004 input: (8ff4df8ffed5afe0c7e6cfc5edd8e8cdcbbfdef0d6ee)
008+ Expected UTF-32BE: (00007e9800009e850002146e0000966a00008cc400008f4200008422000067d000006ea500005e57)
009+ Actually got:      (00007e9800009e85000000250000966a00008cc400008f4200008422000067d000006ea500005e57)
010+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (8ff4df8ffed5afe0c7e6cfc5edd8e8cdcbbfdef0d6ee) when converting to UTF-32BE
011+ mb_check_encoding failed on good EUC-JP-2004 string: (d0e8adb4e0fbabc0bcddf6a9afa2)
012+ mb_convert_encoding not working on EUC-JP-2004 input: (d0e8adb4e0fbabc0bcddf6a9afa2)
013+ Expected UTF-32BE: (00004ff600002473000074ca00000291000067530002373f000054c6)
014+ Actually got:      (00004ff600002473000074ca000002910000675300000025000054c6)
015+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (d0e8adb4e0fbabc0bcddf6a9afa2) when converting to UTF-32BE
016+ mb_check_encoding failed on good EUC-JP-2004 string: (8fa8d9cfeed6d0d3e9f7e68ff8aaefeed6e4)
017+ mb_convert_encoding not working on EUC-JP-2004 input: (8fa8d9cfeed6d0d3e9f7e68ff8aaefeed6e4)
018+ Expected UTF-32BE: (000037f400021e3400005d840000569400003e3f00027614000095d400005e1a)
019+ Actually got:      (000037f40000002500005d840000569400003e3f00027614000095d400005e1a)
020+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (8fa8d9cfeed6d0d3e9f7e68ff8aaefeed6e4) when converting to UTF-32BE
021+ mb_check_encoding failed on good EUC-JP-2004 string: (fdb98ff2f08fafec8fadb2e7a4e7c7faddf2ad)
022+ mb_convert_encoding not working on EUC-JP-2004 input: (fdb98ff2f08fafec8fadb2e7a4e7c7faddf2ad)
023+ Expected UTF-32BE: (00009348000079c700006af100022b4600008153000081df00026aff00009b2e)
024+ Actually got:      (00009348000079c700006af100022b4600008153000081df0000002500009b2e)
025+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (fdb98ff2f08fafec8fadb2e7a4e7c7faddf2ad) when converting to UTF-32BE
026+ mb_check_encoding failed on good EUC-JP-2004 string: (bde6c7c98fa5ace3cd8ff0eeb9d2fbb3a6d4)
027+ mb_convert_encoding not working on EUC-JP-2004 input: (bde6c7c98fa5ace3cd8ff0eeb9d2fbb3a6d4)
028+ Expected UTF-32BE: (0000918700006d3e0000588f00007a37000074240000822a00026e40000003c5)
029+ Actually got:      (0000918700006d3e0000588f00007a37000074240000822a00000025000003c5)
030+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (bde6c7c98fa5ace3cd8ff0eeb9d2fbb3a6d4) when converting to UTF-32BE
031+ mb_check_encoding failed on good EUC-JP-2004 string: (c6edf9a98ff8bcb9f0a1aed3f98ff9f3f9b7a4a8fcb8)
032+ mb_convert_encoding not working on EUC-JP-2004 input: (c6edf9a98ff8bcb9f0a1aed3f98ff9f3f9b7a4a8fcb8)
033+ Expected UTF-32BE: (000075770002550e0002770e0000544a0000ff40000056ee00008fe00000fa4e000030480000fa64)
034+ Actually got:      (00007577000000250002770e0000544a0000ff40000056ee00008fe00000fa4e000030480000fa64)
035+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (c6edf9a98ff8bcb9f0a1aed3f98ff9f3f9b7a4a8fcb8) when converting to UTF-32BE
036+ mb_check_encoding failed on good EUC-JP-2004 string: (f8c8f5f08feeefddaed3a8f6e08fa1c0dfd8c6fe)
037+ mb_convert_encoding not working on EUC-JP-2004 input: (f8c8f5f08feeefddaed3a8f6e08fa1c0dfd8c6fe)
038+ Expected UTF-32BE: (000075170000690a00006e2200006aea000054e600023cfe00004f16000070f100005165)
039+ Actually got:      (000075170000690a00006e2200006aea000054e60000002500004f16000070f100005165)
040+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (f8c8f5f08feeefddaed3a8f6e08fa1c0dfd8c6fe) when converting to UTF-32BE
041+ mb_check_encoding failed on good EUC-JP-2004 string: (8ff5e2a5fa8ffbd7e2e1e3f58ffaf2afc2b1f3abb3abeb)
042+ mb_convert_encoding not working on EUC-JP-2004 input: (8ff5e2a5fa8ffbd7e2e1e3f58ffaf2afc2b1f3abb3abeb)
043+ Expected UTF-32BE: (00008222000030b10000309a00028d10000077da00007b0a000289bb0002123d00009060000002500000031f)
044+ Actually got:      (00008222000030b10000309a00028d10000077da00007b0a000289bb0000002500009060000002500000031f)
045+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (8ff5e2a5fa8ffbd7e2e1e3f58ffaf2afc2b1f3abb3abeb) when converting to UTF-32BE
046+ mb_check_encoding failed on good EUC-JP-2004 string: (8ffca3dade8fafbc8ffcb1fca2dcaee1f1fbc9bac8)
047+ mb_convert_encoding not working on EUC-JP-2004 input: (8ffca3dade8fafbc8ffcb1fca2dcaee1f1fbc9bac8)
048+ Expected UTF-32BE: (000292b100006772000069c0000097f100008a370000692a0000763b000270f4000054c9)
049+ Actually got:      (000292b100006772000069c0000097f100008a370000692a0000763b00000025000054c9)
050+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (8ffca3dade8fafbc8ffcb1fca2dcaee1f1fbc9bac8) when converting to UTF-32BE
051+ mb_check_encoding failed on good EUC-JP-2004 string: (a6e5aea2f8e3baa1f6c5)
052+ mb_convert_encoding not working on EUC-JP-2004 input: (a6e5aea2f8e3baa1f6c5)
053+ Expected UTF-32BE: (000026170002000b0000769e00006b6400006b77)
054+ Actually got:      (00002617000000250000769e00006b6400006b77)
055+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (a6e5aea2f8e3baa1f6c5) when converting to UTF-32BE
056+ mb_check_encoding failed on good EUC-JP-2004 string: (bceecfe3f6cb8facd0f7c3)
057+ mb_convert_encoding not working on EUC-JP-2004 input: (bceecfe3f6cb8facd0f7c3)
058+ Expected UTF-32BE: (000073e0000216b400006bd6000226f300007064)
059+ Actually got:      (000073e00000002500006bd6000226f300007064)
060+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (bceecfe3f6cb8facd0f7c3) when converting to UTF-32BE
061+ mb_check_encoding failed on good EUC-JP-2004 string: (c3f28ff5a6f8da8ff4a1f8fea6e8a7aa8ff3d0)
062+ mb_convert_encoding not working on EUC-JP-2004 input: (c3f28ff5a6f8da8ff4a1f8fea6e8a7aa8ff3d0)
063+ Expected UTF-32BE: (000092f3000080110000764b00007ccd0002548e000026000000041800007b69)
064+ Actually got:      (000092f3000080110000764b00007ccd00000025000026000000041800007b69)
065+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (c3f28ff5a6f8da8ff4a1f8fea6e8a7aa8ff3d0) when converting to UTF-32BE
066+ mb_check_encoding failed on good EUC-JP-2004 string: (8ffaad2b8fa4cadbf4b9e9afcc8fa8c9cdc1a2e3b8a2)
067+ mb_convert_encoding not working on EUC-JP-2004 input: (8ffaad2b8fa4cadbf4b9e9afcc8fa8c9cdc1a2e3b8a2)
068+ Expected UTF-32BE: (000090dc0000002b000036050000688d000062f70002131b00005cc900008f3f0000226a00006a29)
069+ Actually got:      (000090dc0000002b000036050000688d000062f70000002500005cc900008f3f0000226a00006a29)
070+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (8ffaad2b8fa4cadbf4b9e9afcc8fa8c9cdc1a2e3b8a2) when converting to UTF-32BE
071+ mb_check_encoding failed on good EUC-JP-2004 string: (b9e5e5b2cdc3e9a1e9febaa7f9ee8fefaf)
072+ mb_convert_encoding not working on EUC-JP-2004 input: (b9e5e5b2cdc3e9a1e9febaa7f9ee8fefaf)
073+ Expected UTF-32BE: (000052ab00007d4f000050ad000085410000875900005a5a00025da100006f25)
074+ Actually got:      (000052ab00007d4f000050ad000085410000875900005a5a0000002500006f25)
075+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (b9e5e5b2cdc3e9a1e9febaa7f9ee8fefaf) when converting to UTF-32BE
076+ mb_check_encoding failed on good EUC-JP-2004 string: (8faffee6c9f1af8ff1a5ddd6d5d3f5f2)
077+ mb_convert_encoding not working on EUC-JP-2004 input: (8faffee6c9f1af8ff1a5ddd6d5d3f5f2)
078+ Expected UTF-32BE: (00006b8100007fe900009912000074ab00006bb700005b51000235c4)
079+ Actually got:      (00006b8100007fe900009912000074ab00006bb700005b5100000025)
080+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (8faffee6c9f1af8ff1a5ddd6d5d3f5f2) when converting to UTF-32BE
081+ mb_check_encoding failed on good EUC-JP-2004 string: (bfeff7ecc8acbab9edc58ff7add5eaa4a2)
082+ mb_convert_encoding not working on EUC-JP-2004 input: (bfeff7ecc8acbab9edc58ff7add5eaa4a2)
083+ Expected UTF-32BE: (0000968f000247f10000516b00005dee00008efc0000860000005be200003042)
084+ Actually got:      (0000968f000000250000516b00005dee00008efc0000860000005be200003042)
085+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (bfeff7ecc8acbab9edc58ff7add5eaa4a2) when converting to UTF-32BE
086+ mb_check_encoding failed on good EUC-JP-2004 string: (8ff5fa8ff9e9e5cef7c2cfd4feadedf3e0f0)
087+ mb_convert_encoding not working on EUC-JP-2004 input: (8ff5fa8ff9e9e5cef7c2cfd4feadedf3e0f0)
088+ Expected UTF-32BE: (000082830000fa6600007de10000704e00020b9f00009a030000901100007441)
089+ Actually got:      (000082830000fa6600007de10000704e0000002500009a030000901100007441)
090+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (8ff5fa8ff9e9e5cef7c2cfd4feadedf3e0f0) when converting to UTF-32BE
091+ mb_check_encoding failed on good EUC-JP-2004 string: (f9d4e7f6debeaafcc9dbbdb5baf5)
092+ mb_convert_encoding not working on EUC-JP-2004 input: (f9d4e7f6debeaafcc9dbbdb5baf5)
093+ Expected UTF-32BE: (000259c40000839300006db50000028100005e030000903100007a84)
094+ Actually got:      (000000250000839300006db50000028100005e030000903100007a84)
095+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (f9d4e7f6debeaafcc9dbbdb5baf5) when converting to UTF-32BE
096+ mb_check_encoding failed on good EUC-JP-2004 string: (f7dceef7dfeabdb1fbecceb1abd9)
097+ mb_convert_encoding not working on EUC-JP-2004 input: (f7dceef7dfeabdb1fbecceb1abd9)
098+ Expected UTF-32BE: (00007187000092cf0000714c0000897200027684000075590000030b)
099+ Actually got:      (00007187000092cf0000714c0000897200000025000075590000030b)
100+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (f7dceef7dfeabdb1fbecceb1abd9) when converting to UTF-32BE
101+ mb_check_encoding failed on good EUC-JP-2004 string: (d9ddcdf3fcd1c2f5fbd1e6ac)
102+ mb_convert_encoding not working on EUC-JP-2004 input: (d9ddcdf3fcd1c2f5fbd1e6ac)
103+ Expected UTF-32BE: (0000639f00006b04000283cd00006fef0000459d00007f68)
104+ Actually got:      (0000639f00006b040000002500006fef0000459d00007f68)
105+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (d9ddcdf3fcd1c2f5fbd1e6ac) when converting to UTF-32BE
106+ mb_check_encoding failed on good EUC-JP-2004 string: (eca6b8c78ff2b8c2b3dedad7bcbdd7b5c7ebd5fee6)
107+ mb_convert_encoding not working on EUC-JP-2004 input: (eca6b8c78ff2b8c2b3dedad7bcbdd7b5c7ebd5fee6)
108+ Expected UTF-32BE: (00008b80000056fa000077ec00007d9a00006e4d00005f4800007ae3000097a000008aa50002a190)
109+ Actually got:      (00008b80000056fa000077ec00007d9a00006e4d00005f4800007ae3000097a000008aa500000025)
110+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (eca6b8c78ff2b8c2b3dedad7bcbdd7b5c7ebd5fee6) when converting to UTF-32BE
111+ mb_check_encoding failed on good EUC-JP-2004 string: (dcd7f3ceada98ff9def6b2b1bffaeee7da)
112+ mb_convert_encoding not working on EUC-JP-2004 input: (dcd7f3ceada98ff9def6b2b1bffaeee7da)
113+ Expected UTF-32BE: (0000699500009ea500002468000282f3000237630000904b0000832200008259)
114+ Actually got:      (0000699500009ea500002468000282f3000000250000904b0000832200008259)
115+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (dcd7f3ceada98ff9def6b2b1bffaeee7da) when converting to UTF-32BE
116+ mb_check_encoding failed on good EUC-JP-2004 string: (e3a3b8d9a6bfcde1d7bbf5baccb8)
117+ mb_convert_encoding not working on EUC-JP-2004 input: (e3a3b8d9a6bfcde1d7bbf5baccb8)
118+ Expected UTF-32BE: (000078fd00008de80000266700006d7400005f41000231c400009727)
119+ Actually got:      (000078fd00008de80000266700006d7400005f410000002500009727)
120+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (e3a3b8d9a6bfcde1d7bbf5baccb8) when converting to UTF-32BE
121+ mb_check_encoding failed on good EUC-JP-2004 string: (bda5affbc7b8ebfeaecbeec58ffabb)
122+ mb_convert_encoding not working on EUC-JP-2004 input: (bda5affbc7b8ebfeaecbeec58ffabb)
123+ Expected UTF-32BE: (00006101000218bd000057dc00008b6b000051030000915600009137)
124+ Actually got:      (0000610100000025000057dc00008b6b000051030000915600009137)
125+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (bda5affbc7b8ebfeaecbeec58ffabb) when converting to UTF-32BE
126+ mb_check_encoding failed on good EUC-JP-2004 string: (c2ed8ffee8f9c7afddfed8d2af)
127+ mb_convert_encoding not working on EUC-JP-2004 input: (c2ed8ffee8f9c7afddfed8d2af)
128+ Expected UTF-32BE: (0000702700009f41000257710000589e00009d1d000052f5)
129+ Actually got:      (0000702700009f41000000250000589e00009d1d000052f5)
130+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (c2ed8ffee8f9c7afddfed8d2af) when converting to UTF-32BE
131+ mb_check_encoding failed on good EUC-JP-2004 string: (f4c6bdd2bbf0e0e3bdb4fcc9ddb48ff0a1f9f0)
132+ mb_convert_encoding not working on EUC-JP-2004 input: (f4c6bdd2bbf0e0e3bdb4fcc9ddb48ff0a1f9f0)
133+ Expected UTF-32BE: (00005fc900008ff000008aee000073ee00008f2f0002827700006b16000071b300007c6d)
134+ Actually got:      (00005fc900008ff000008aee000073ee00008f2f0000002500006b16000071b300007c6d)
135+ mb_convert_encoding incremented illegal_chars on valid EUC-JP-2004 string: (f4c6bdd2bbf0e0e3bdb4fcc9ddb48ff0a1f9f0) when converting to UTF-32BE
     EUC-JP-2004 verification and conversion works for all valid characters
137+ mb_convert_encoding not working on EUC-JP-2004 input: (fc83f6a9)
138+ Expected UTF-32BE: "%�7?" (000000250002373f)
139+ Actually got:      "%%" (0000002500000025)
140+ mb_convert_encoding not working on EUC-JP-2004 input: (f22efcd1)
141+ Expected UTF-32BE: (00000025000283cd)
142+ Actually got:      "%%" (0000002500000025)
143+ mb_convert_encoding not working on EUC-JP-2004 input: (f21ef9c7)
144+ Expected UTF-32BE: "%�Wq" (0000002500025771)
145+ Actually got:      "%%" (0000002500000025)
146+ mb_convert_encoding not working on EUC-JP-2004 input: (eefff9ee)
147+ Expected UTF-32BE: (0000002500025da1)
148+ Actually got:      "%%" (0000002500000025)
149+ mb_convert_encoding not working on EUC-JP-2004 input: (ec81f6e0)
150+ Expected UTF-32BE: (0000002500023cfe)
151+ Actually got:      "%%" (0000002500000025)
152+ mb_convert_encoding not working on EUC-JP-2004 input: (ea3ef8fe)
153+ Expected UTF-32BE: (000000250002548e)
154+ Actually got:      "%%" (0000002500000025)
155+ mb_convert_encoding not working on EUC-JP-2004 input: (e817fadd)
156+ Expected UTF-32BE: (0000002500026aff)
157+ Actually got:      "%%" (0000002500000025)
158+ mb_convert_encoding not working on EUC-JP-2004 input: (e676fbb3)
159+ Expected UTF-32BE: "%�n@" (0000002500026e40)
160+ Actually got:      "%%" (0000002500000025)
161+ mb_convert_encoding not working on EUC-JP-2004 input: (e293f9a9)
162+ Expected UTF-32BE: "%�U�" (000000250002550e)
163+ Actually got:      "%%" (0000002500000025)
164+ mb_convert_encoding not working on EUC-JP-2004 input: (e068cfe3)
165+ Expected UTF-32BE: (00000025000216b4)
166+ Actually got:      "%%" (0000002500000025)
167+ mb_convert_encoding not working on EUC-JP-2004 input: (dd23f5f2)
168+ Expected UTF-32BE: (00000025000235c4)
169+ Actually got:      "%%" (0000002500000025)
170+ mb_convert_encoding not working on EUC-JP-2004 input: (dc77cfd4)
171+ Expected UTF-32BE: (0000002500020b9f)
172+ Actually got:      "%%" (0000002500000025)
173+ mb_convert_encoding not working on EUC-JP-2004 input: (db89fcc9)
174+ Expected UTF-32BE: (0000002500028277)
175+ Actually got:      "%%" (0000002500000025)
176+ mb_convert_encoding not working on EUC-JP-2004 input: (db16f9d4)
177+ Expected UTF-32BE: (00000025000259c4)
178+ Actually got:      "%%" (0000002500000025)
179+ mb_convert_encoding not working on EUC-JP-2004 input: (d662f7ec)
180+ Expected UTF-32BE: (00000025000247f1)
181+ Actually got:      "%%" (0000002500000025)
182+ mb_convert_encoding not working on EUC-JP-2004 input: (d65dfbc9)
183+ Expected UTF-32BE: (00000025000270f4)
184+ Actually got:      "%%" (0000002500000025)
185+ mb_convert_encoding not working on EUC-JP-2004 input: (d309fbec)
186+ Expected UTF-32BE: (0000002500027684)
187+ Actually got:      "%%" (0000002500000025)
188+ mb_convert_encoding not working on EUC-JP-2004 input: (cd8baffb)
189+ Expected UTF-32BE: (00000025000218bd)
190+ Actually got:      "%%" (0000002500000025)
191+ mb_convert_encoding not working on EUC-JP-2004 input: (cc4caea2)
192+ Expected UTF-32BE: "%��" (000000250002000b)
193+ Actually got:      "%%" (0000002500000025)
194+ mb_convert_encoding not working on EUC-JP-2004 input: (ca0dafcc)
195+ Expected UTF-32BE: "%��
196+ Actually got:      "%%" (0000002500000025)
197+ mb_convert_encoding not working on EUC-JP-2004 input: (c77bfee6)
198+ Expected UTF-32BE: (000000250002a190)
199+ Actually got:      "%%" (0000002500000025)
200+ mb_convert_encoding not working on EUC-JP-2004 input: (c57bafc2)
201+ Expected UTF-32BE: "%��=" (000000250002123d)
202+ Actually got:      "%%" (0000002500000025)
203+ mb_convert_encoding not working on EUC-JP-2004 input: (c303afe0)
204+ Expected UTF-32BE: "%��n" (000000250002146e)
205+ Actually got:      "%%" (0000002500000025)
206+ mb_convert_encoding not working on EUC-JP-2004 input: (c009f6b2)
207+ Expected UTF-32BE: "%�7c" (0000002500023763)
208+ Actually got:      "%%" (0000002500000025)
209+ mb_convert_encoding not working on EUC-JP-2004 input: (be49cfee)
210+ Expected UTF-32BE: "%��4" (0000002500021e34)
211+ Actually got:      "%%" (0000002500000025)
212+ mb_convert_encoding not working on EUC-JP-2004 input: (bc36f5ba)
213+ Expected UTF-32BE: (00000025000231c4)
214+ Actually got:      "%%" (0000002500000025)
215+ mb_convert_encoding not working on EUC-JP-2004 input: (ad0aafc2)
216+ Expected UTF-32BE: "%��=" (000000250002123d)
217+ Actually got:      "%%" (0000002500000025)
218+ mb_convert_encoding not working on EUC-JP-2004 input: (ac1bfadd)
219+ Expected UTF-32BE: (0000002500026aff)
220+ Actually got:      "%%" (0000002500000025)
221+ mb_convert_encoding not working on EUC-JP-2004 input: (a569f5ba)
222+ Expected UTF-32BE: (00000025000231c4)
223+ Actually got:      "%%" (0000002500000025)
224+ mb_convert_encoding not working on EUC-JP-2004 input: (a44bafe0)
225+ Expected UTF-32BE: "%��n" (000000250002146e)
226+ Actually got:      "%%" (0000002500000025)
227+ mb_convert_encoding not working on EUC-JP-2004 input: (a218cfee)
228+ Expected UTF-32BE: "%��4" (0000002500021e34)
229+ Actually got:      "%%" (0000002500000025)
230+ mb_convert_encoding not working on EUC-JP-2004 input: (8ff97ffcc9)
231+ Expected UTF-32BE: (0000002500028277)
232+ Actually got:      "%%" (0000002500000025)
233+ mb_convert_encoding not working on EUC-JP-2004 input: (8ff66af5f2)
234+ Expected UTF-32BE: (00000025000235c4)
235+ Actually got:      "%%" (0000002500000025)
236+ mb_convert_encoding not working on EUC-JP-2004 input: (8ff22df9a9)
237+ Expected UTF-32BE: "%�U�" (000000250002550e)
238+ Actually got:      "%%" (0000002500000025)
239+ mb_convert_encoding not working on EUC-JP-2004 input: (8ff217affb)
240+ Expected UTF-32BE: (00000025000218bd)
241+ Actually got:      "%%" (0000002500000025)
242+ mb_convert_encoding not working on EUC-JP-2004 input: (8fef0cfbb3)
243+ Expected UTF-32BE: "%�n@" (0000002500026e40)
244+ Actually got:      "%%" (0000002500000025)
245+ mb_convert_encoding not working on EUC-JP-2004 input: (8fee58f8fe)
246+ Expected UTF-32BE: (000000250002548e)
247+ Actually got:      "%%" (0000002500000025)
248+ mb_convert_encoding not working on EUC-JP-2004 input: (8faf43f7ec)
249+ Expected UTF-32BE: (00000025000247f1)
250+ Actually got:      "%%" (0000002500000025)
251+ mb_convert_encoding not working on EUC-JP-2004 input: (8fa879f9c7)
252+ Expected UTF-32BE: "%�Wq" (0000002500025771)
253+ Actually got:      "%%" (0000002500000025)
254+ mb_convert_encoding not working on EUC-JP-2004 input: (8fa32efbc9)
255+ Expected UTF-32BE: (00000025000270f4)
256+ Actually got:      "%%" (0000002500000025)
257+ mb_convert_encoding not working on EUC-JP-2004 input: (8e71fbec)
258+ Expected UTF-32BE: (0000002500027684)
259+ Actually got:      "%%" (0000002500000025)
260+ mb_convert_encoding not working on EUC-JP-2004 input: (8e6bcfe3)
261+ Expected UTF-32BE: (00000025000216b4)
262+ Actually got:      "%%" (0000002500000025)

PHP Version

PHP 8.4+

Operating System

macOS 15.7.2, Apple clang version 16.0.0 (clang-1600.0.26.3)

[iluuu1994]

/cc @alexdowad, @youkidearitai. I don't know if either of you have a mac to reproduce this issue? I unfortunately don't. I can try setting up a Hackintosh VM (though I failed at this last time I tried).

[alexdowad]

I have a Mac and can have a look.

[alexdowad]

@iluuu1994 I built php-src on my Mac (Apple M4, running Sequoia 15.6.1), ran the tests with make test, and the failure which occurred in CI did not happen.
(Yes, I made sure that the mbstring tests are running.)
🤔

[alexdowad]

Looking at the test failure excerpt which @iluuu1994 kindly provided, it seems that every failure is essentially:
A EUC-JP-2004 input which should have been converted to a codepoint outside the BMP (>U+20000), is converted to an ASCII percent sign (%) instead.
Let me think about that and see if we can get any clue where things are going wrong...

[youkidearitai]

I try reproduce, but I can't also reproduced.
macOS is 15.7.2（24G325）.

[alexdowad]

Randomly picking out one failed case for further analysis:

245+ mb_convert_encoding not working on EUC-JP-2004 input: (8fee58f8fe)
246+ Expected UTF-32BE: (000000250002548e)
247+ Actually got:      "%%" (0000002500000025)

0x8F EE 58 is junk. 0x8F is valid, the following 0xEE is valid, but 0x58 is not a legal EUC-JP-2004 byte coming after 0x8F EE, so mbstring will output an error marker (in this case, "%") and restart a new character from the next byte.

Then, 0xF8 FE is valid; it maps to U+2548E, which is the Kanji 𥒎.

If mbstring had read 0x8F EE, then (wrongly) immediately output an error marker and restarted, 0x58 would be accepted as a valid ASCII byte and would be output as U+0058. Obviously, that didn't happen. So I believe that mbstring did read 0x8F EE 58 as expected, output an error marker as expected, then something went wrong here:

/* Conversion for CJK Unified Ideographs ext.B (U+2XXXX) */
if (!w) {
	int k = mbfl_bisec_srch2(w1, jisx0213_jis_u5_key, jisx0213_u5_tbl_len);
	if (k >= 0) {
		w = jisx0213_jis_u5_tbl[k] + 0x20000;
	}
}

*out++ = w ? w : MBFL_BAD_INPUT;

(Lines 9236-9244, mbfilter_cjk.c)

That does a binary search in a table to find the conversion for codepoints >U+20000. If it doesn't find the desired character, it outputs an error marker (which is what we see).

This is the table where the binary search should occur:

static const unsigned short jisx0213_jis_u5_key[] = {
0x04C7,0x0545,0x054F,0x0563,0x057E,0x1117,0x1126,0x1131,
0x1EF1,0x1F29,0x1F3E,0x1F47,0x1F75,0x1FDF,0x204F,0x2058,
0x2076,0x2083,0x209D,0x20EA,0x211E,0x2134,0x2157,0x2192,
0x219A,0x226B,0x2284,0x228E,0x2291,0x2299,0x22A9,0x22D3,
0x22DA,0x22DC,0x22F1,0x22F8,0x22F9,0x22FB,0x22FC,0x2300,
0x2301,0x2306,0x2309,0x230B,0x230C,0x231C,0x2327,0x232D,
0x2341,0x2344,0x2346,0x2350,0x2351,0x2357,0x235E,0x2360,
0x2369,0x2371,0x2372,0x2378,0x237B,0x2396,0x23A7,0x23AE,
0x23AF,0x23B7,0x23BA,0x23D6,0x23D9,0x23DB,0x23E0,0x23E7,
0x23E8,0x23EF,0x23F1,0x23F2,0x2400,0x240D,0x2419,0x241F,
0x2422,0x2430,0x2431,0x2459,0x245B,0x2464,0x2469,0x2489,
0x249E,0x24A6,0x24AB,0x24BB,0x24C0,0x24C1,0x24C9,0x24CB,
0x24CC,0x24D0,0x24ED,0x2514,0x2518,0x2519,0x252F,0x2531,
0x2532,0x2537,0x2538,0x2539,0x253C,0x253E,0x254A,0x254B,
0x254C,0x2550,0x256C,0x256D,0x257D,0x2592,0x2593,0x2595,
0x2596,0x25A1,0x25AC,0x25B4,0x25BC,0x25BD,0x25C3,0x25C8,
0x25D4,0x25E5,0x25FA,0x260D,0x260F,0x2610,0x2611,0x2641,
0x2656,0x265C,0x2670,0x2673,0x268F,0x26A0,0x26A6,0x26C0,
0x26E8,0x26F9,0x26FB,0x2700,0x270F,0x2728,0x272C,0x2731,
0x2734,0x2740,0x2742,0x2745,0x274A,0x274C,0x274D,0x2751,
0x2755,0x2766,0x2771,0x2784,0x279E,0x279F,0x27B9,0x27C4,
0x27C5,0x27C7,0x27D9,0x27E4,0x27E5,0x27FA,0x27FB,0x27FC,
0x27FE,0x2802,0x2804,0x2807,0x2809,0x280C,0x2813,0x2814,
0x2819,0x281A,0x2822,0x2827,0x2834,0x284E,0x2850,0x2857,
0x285A,0x285E,0x2878,0x287D,0x2889,0x2899,0x289B,0x289D,
0x28A0,0x28A2,0x28A6,0x28AD,0x28B3,0x28B6,0x28E5,0x28EF,
0x28FE,0x2916,0x291F,0x2920,0x2921,0x2932,0x2935,0x2963,
0x2964,0x296A,0x296D,0x2972,0x2986,0x2987,0x2989,0x2991,
0x2999,0x299B,0x299F,0x29B3,0x29C0,0x29D4,0x29D6,0x29D7,
0x29E0,0x29EA,0x29ED,0x29F4,0x29FD,0x2A03,0x2A14,0x2A19,
0x2A21,0x2A26,0x2A35,0x2A39,0x2A3A,0x2A45,0x2A48,0x2A68,
0x2A69,0x2A70,0x2A7E,0x2A7F,0x2A8A,0x2A8B,0x2A97,0x2A98,
0x2AA3,0x2AA4,0x2AAD,0x2AAE,0x2AB4,0x2AB5,0x2AC5,0x2AC6,
0x2ACE,0x2AD1,0x2AD3,0x2AD4,0x2AD8,0x2ADC,0x2ADE,0x2AE0,
0x2AE8,0x2AF7,0x2AF8,0x2B0D,0x2B17,0x2B21,0x2B2B,0x2B2E,
0x2B32,0x2B4B,0x2B5F,0x2B7E,0x2B7F,0x2B8C,0x2B8E,0x2B90,
0x2B9A,0x2BA0,0x2BA3,0x2BB6,0x2BBA,0x2BBC,0x2BC3,0x2BC6,
0x2BE4,0x2BE9,0x2BEB,0x2BFF,0x2C01,0x2C03,0x2C07
};

In the failing CI run which @iluuu1994 kindly shared, many EUC-JP-2004 conversion operations involving codepoints >U+20000 all failed. That implies that the problem was not caused by something like a corrupted bit or byte somewhere in this table. Otherwise, we would expect that only a couple of test cases would fail.

The function mbfl_bisec_srch2 is used for many Japanese and Chinese text encodings. But we are only seeing test failures for EUC-JP-2004. That implies that the problem was not caused by a miscompilation of mbfl_bisec_srch2.

Further, the table jisx0213_jis_u5_key is not just used for EUC-JP-2004, but also for SJIS-2004 and ISO-2022-JP-2004. But we don't see any test failures for those encodings. So again, it doesn't look like the problem is caused by corruption of jisx0213_jis_u5_key.

If anything, what I am seeing seems consistent with a miscompilation or corruption of this little snippet of code itself:

if (!w) {
	int k = mbfl_bisec_srch2(w1, jisx0213_jis_u5_key, jisx0213_u5_tbl_len);
	if (k >= 0) {
		w = jisx0213_jis_u5_tbl[k] + 0x20000;
	}
}

Any thoughts from others? @iluuu1994 @youkidearitai @cmb69 @ndossche

[iluuu1994]

Thank you for your analysis! I'll try to pull the quoted code into an unoptimized function to see if it changes anything.

[iluuu1994]

And voilà:

iluuu1994#168

This indeed seems to be the culprit. Very nice deduction by @alexdowad.

@alexdowad @youkidearitai Could you tell me which version of Clang you've tried this with? It would be optimal if we could pin-point the issue and even share a diff of the assembly to report an issue, but this is very hard (or rather extremely tedious) to do via CI.

[alexdowad]

7:02 ~/Programming/php-src % clang -v
Apple clang version 17.0.0 (clang-1700.0.13.5)
Target: arm64-apple-darwin24.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

[alexdowad]

In the failing CI test run, I see this:

Apple clang version 16.0.0 (clang-1600.0.26.3)
Target: x86_64-apple-darwin24.6.0
Thread model: posix
InstalledDir: /Applications/Xcode_16.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

I'll try installing Clang 16 on my Mac and see if I can replicate the problem that way.

[iluuu1994]

@alexdowad Thank you. Good to know. CI runs with clang-1600.0.26.3, so possibly this issue is already fixed.

I'll try installing Clang 16 on my Mac and see if I can replicate the problem that way.

Thank you!