Improve test run output

[cgay]

I'd like to see several improvements to Testworks output:

• An option to display no progress output except for failing tests. --progress failures I suppose. i.e., don't display pass, not implemented, expected to fail.
• It should be rare that tests do output to the console. When they do it's generally for debugging. (threads-test-suite stands out as doing a lot of essentially meaningless output.) All such debug output should be via test-output:testworks and should by default go to test-temp-directory()/test-name.log and there can be an option to also send it to the console. --test-output=all, for example...not sure of name.
• Display test run time as a number of microseconds, or heuristically ms, usec, ns. My time library already does this so perhaps wait and use that. Seconds as the unit is just too big these days.
• Always display the final summary output. Specifically, with the options that are passed to make check (--report json --report-file foo.json) I still want the final count of passing/failing tests to be shown on the terminal.
• No need to display passing/failing suites. I've never found this remotely useful.
• In the summary output, display the counts for tests last, after checks, because that last line should have the info I really care about.
• Don't display "0 benchmarks". (This might might already be the case?) Our plan is to run benchmarks as separate libraries.
• Possibly: note which tests are new after each --load so they can be associated with their .so file and we can execute them in that same order and display the .so filename before any progress output for that set of tests. (The output from make check is currently pretty random and hard to know what tests belong to what library.)
• Can probably elide "Running test " before each test in the progress output. Just print the test name there.
• Can greatly improve output for test-equal failures:

  test-encode/decode-varint failed
      encode-varint-bytes(i) = bytes: [{stretchy vector 128, 128, 128, 128, 248, 255, 255, 255, 255, 127} and {stretchy vector 3} are not =.  sizes differ (10 and 1), element 0 is the first non-matching element]
        FAILED in 0.002164s and 58KiB
  

  should become something like this

  test-encode/decode-varint: values differ starting at element 0
      encode-varint-bytes(i): size=10 {stretchy vector 128, 128, 128, 128, 248, 255, 255, 255, 255, 127}
      bytes:                  size=1 {stretchy vector 3}
  

Those are my thoughts. We'll see how it develops.

[housel]

Don't display "0 benchmarks", and if there are no tests (because it's a benchmark suite) then don't display "0 tests". The two cases can be symmetric.

[cgay]

None of this makes sense:

$ ../_build/bin/time-test-suite --test test-load-tzif-version-1
Running test test-load-tzif-version-1: NOT IMPLEMENTED in 0.000822s and 68KiB

test-load-tzif-version-1 not implemented

Ran 0 checks: PASSED
Ran 1 test: PASSED (1 not implemented)
NOT IMPLEMENTED in 0.000822 seconds

• Huge amounts of duplication. I don't need to be told 4 times that the test is not implemented.
• "Ran 0 checks: PASSED" should not say PASSED. Either say NOT IMPLEMENTED or just leave it blank.
• "Ran 1 test: PASSED (1 not implemented)" -- which is it? passed or not implemented?
• "NOT IMPLEMENTED in 0.000822 seconds" -- Strange wording. Avoid the issue by saying "Result: NOT IMPLEMENTED, Total run time: 822µs"

[cgay]

I still find this output for failed checks to be pretty bad:

FAILED: $expected-failure = test-results.result-status [#"expected-failure" and #"unexpected-success" are not =.]

I think better would be

FAILED: #"expected-failure" ~= #"unexpected-success" for assert-equal($expected-failure, test-results.result-status)

Or even better, a multi-line display:

FAILED: assert-equal($expected-failure, test-results.result-status)
    Expected: #"expected-failure"
    Got: #"unexpected-success"

[cgay]

Current assert-equal output now looks like this:

FAILED: lox-test-suite
  FAILED: test-eval-precedence
    FAILED: 7 = %eval("2*(3+4)/3;")
      want: 7
      got:  4.6666667d0
  FAILED: test-eval-program
    FAILED: #[1, 2, 3, 4] = #[1, 2, 5, 4]
      want: #[1, 2, 3, 4]
      got:  #[1, 2, 5, 4]
      detail: element 2 is the first mismatch

[cgay]

deft test http-server-test-suite -- --run test-repeated-start-stop-with-connection --debug crashes

I get this output:

Ran 25 tests: 24 skipped

It should say Ran 1 test, skipped 24.