Run tests against statically-linked binaries

[szokeasaurusrex]

Run our test suite against statically-linked binaries, since these are what we distribute to users

[linear]

CLI-147 Run tests against statically-linked binaries

[szokeasaurusrex]

Fixed in #2653