There might be problems with upgrading: https://buildkite.com/materialize/nightly/builds/16176

15:1: executing query failed: db error: ERROR: MYSQL source validation: The MySQL system variable 'binlog_row_metadata' is set to an unsupported value: MINIMAL. Materialize requires this variable to be set to 'FULL' to use the "CREATE TABLE FROM SOURCE" syntax for MySQL sources.
     |
   2 | $ postgres-execute c ... [rest of line truncated for security]
   5 | $ mysql-connect name ... [rest of line truncated for security]
  14 |   FROM MYSQL CONNECTION mysql2_true;
  15 | > CREATE TABLE mysql_source_tableB_true FROM SOURCE mysql_source2_true (REFERENCE public.mysql_source_table_true);
     | ^

Or maybe the tests just need to be updated?

There might be problems with upgrading: https://buildkite.com/materialize/nightly/builds/16176

15:1: executing query failed: db error: ERROR: MYSQL source validation: The MySQL system variable 'binlog_row_metadata' is set to an unsupported value: MINIMAL. Materialize requires this variable to be set to 'FULL' to use the "CREATE TABLE FROM SOURCE" syntax for MySQL sources.
     |
   2 | $ postgres-execute c ... [rest of line truncated for security]
   5 | $ mysql-connect name ... [rest of line truncated for security]
  14 |   FROM MYSQL CONNECTION mysql2_true;
  15 | > CREATE TABLE mysql_source_tableB_true FROM SOURCE mysql_source2_true (REFERENCE public.mysql_source_table_true);
     | ^

Or maybe the tests just need to be updated?

I reverted the code that throws that error - that error type has literally been deleted. I think this is a case where the python code running the test is updated, but the rust isn't?

Yes, that's how the upgrade test works, the new Python code runs against old Materialize version when we upgrade it. You can use conditional versions to run different code if it's required: https://github.com/MaterializeInc/materialize/blob/main/doc/developer/testdrive.md#run-an-actionquery-conditionally-on-version
Tell me if you want me to take a closer look and fix up the tests!

Pushed a potential fix for the test, verifying: https://buildkite.com/materialize/nightly/builds/16178

It's a bit unfortunate that we now always have FULL set in MySQL for testing. @patrickwwbutler How bad is that?

I'm hoping it's a temporary bandaid and we'll reapply the PR soon anyway.

It's a bit unfortunate that we now always have FULL set in MySQL for testing. @patrickwwbutler How bad is that?

I'm hoping it's a temporary bandaid and we'll reapply the PR soon anyway.

Well having FULL always set is actually how I missed the bug that caused the incident in the first place, so I guess it's not great, but it shouldn't matter at all for now besides the upgrade tests, and when we reapply I will set up proper testing of both FULL and MINIMAL to avoid this

Thank you for staying late to help!

What we could do is use MINIMAL in all tests except feature-benchmark, workload-replay and platform-checks with upgrades, which stay on FULL