Integrate the busybox tests suite results

[sylvestre]

Just like we are doing with GNU, we should store and publish the results.

make prepare-busytest
        bindir=$(pwd)/target/debug
        cd tmp/busybox-*/testsuite
        output=$(bindir=$bindir ./runtest 2>&1 || true)
        printf "%s\n" "${output}"
        n_fails=$(echo "$output" | grep "^FAIL:\s" | wc --lines)
        if [ $n_fails -gt 0 ] ; then echo "::warning ::${n_fails}+ test failures" ; fi

is that is executed in the CI

[sylvestre]

GNU tests:
https://github.com/uutils/coreutils/blob/main/.github/workflows/GnuTests.yml
Used by
https://github.com/uutils/coreutils-tracking

[kralo]

Do you still find this interesting, should I look into it?

[sylvestre]

We did a bunch of the work. It is published here:
https://github.com/uutils/coreutils-tracking/blob/main/busybox-result.json
but i would be happy to get a comment like GNU when we are fixing a test or regressing one

See
https://github.com/uutils/coreutils/blob/main/.github/workflows/GnuComment.yml

[westurner]

GH also supports file:line_number-based PR comments: https://github.com/ammaraskar/sphinx-action/blob/c61ac11d9ee097caf8983c10c8b5af5861b32b54/sphinx_action/status_check.py#L30 :

 "::{command} file={file},line={line}::{message}".format(

edit:

• https://github.com/amitds1997/pytest-annotate-pr
• https://vitaliimelnychuk.medium.com/code-coverage-reports-in-your-github-pull-requests-f0d164541376
• https://www.kenmuse.com/blog/creating-github-checks/ > "Is it Check or Status?"

[ChrisDryden]

The busybox config was missing a few flags that allowed it to run the whole test suite: #9850

I am seeing that there's a bunch of useful tests in there that are useful to us for edge cases for GNU, but there's also a few tests there the expectation is fully different from the GNU implementation and was wondering if I could do something similar to the sed fixes for the GNU test to indicate that the test is an expected failure versus a regression.