Skip to content

Bump QCheck to 0.26 (and fix float comparisons)#565

Merged
jmid merged 2 commits into
mainfrom
bump-qcheck
Jul 10, 2025
Merged

Bump QCheck to 0.26 (and fix float comparisons)#565
jmid merged 2 commits into
mainfrom
bump-qcheck

Conversation

@jmid

@jmid jmid commented Jul 10, 2025

Copy link
Copy Markdown
Collaborator

The QCheck 0.26 release https://github.com/c-cube/qcheck/releases/tag/v0.26 changes the distribution of the float generator to avoid blind spots: c-cube/qcheck#350.

As we have a couple of tests with floats this PR bumps the QCheck version to see if we've missed anything float-related.

@jmid

jmid commented Jul 10, 2025

Copy link
Copy Markdown
Collaborator Author

Indeed, this reveals comparisons of float results using =.
I've also been able to reproduce locally (may require bumping count as nan is produced relatively rarely with Gen.float).
The most interesting part is that the result is sometimes printed as -nan:

### OCaml runtime: debug mode ###   
### set OCAMLRUNPARAM=v=0 to silence this message
random seed: 534962093
generated error  fail  pass / total     time test name
[✗]   982     0     1   981 / 10000     0.0s STM Float Array test sequential
[✓]     1     0     1     0 / 10000    11.0s STM Float Array test parallel

--- Failure --------------------------------------------------------------------

Test STM Float Array test sequential failed (7 shrink steps):

   Set (6, nan)
   To_seq


+++ Messages ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Messages for test STM Float Array test sequential:

  Results incompatible with model

   Set (6, nan) : Ok (())
   To_seq : [1.; 1.; 1.; 1.; 1.; 1.; -nan; 1.; 1.; 1.; 1.; 1.; 1.; 1.; 1.; 1.]

Looking more into this, it seems the cmd printer is using Util.Pp.float which uses fprintf fmt "%F"
whereas the result printer is using the Float combinator printer relying on QCheck.Print.float which uses string_of_float. We should probably use the same printer in both arguments and results to avoid false alarms.

@jmid

jmid commented Jul 10, 2025

Copy link
Copy Markdown
Collaborator Author

CI summary: 16 workflows failed all due to nan float comparisons using =:
32bit 5.3
32bit 5.4
Bytecode 5.4
FP 5.4
Linux 5.3
Linux 5.3 debug
Linux trunk debug <--- CI log contains 223k lines of updating memprof. and is then cut off since ocaml/ocaml#14057
Linux-ARM64 5.3
Linux-ARM64 5.4
macOS-ARM64 trunk
MinGW bytecode 5.3
MinGW bytecode trunk
MSVC 5.4
MSVC bytecode 5.4 (still going)
musl 5.3
musl 5.4

Out of 50 workflows 16 failed all with a genuine error

@jmid

jmid commented Jul 10, 2025

Copy link
Copy Markdown
Collaborator Author

CI summary: all 50 workflows passed - merging

@jmid jmid changed the title Bump QCheck to 0.26 Bump QCheck to 0.26 (and fix float comparisons) Jul 10, 2025
@jmid jmid merged commit a7bfabb into main Jul 10, 2025
50 checks passed
@jmid jmid deleted the bump-qcheck branch July 10, 2025 15:45
@jmid

jmid commented Jul 10, 2025

Copy link
Copy Markdown
Collaborator Author

CI summary for merge to main: 51 workflows passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant