Option to keep going on error

[pbreheny]

It would be nice if tinytest offered an option to continue testing a file or directory even if an error is encountered. For example, if we could run run_test_file('test.r') and get a message like:

20 tests  2 fails  1 error

if one of the tests was something like expect_equal(f(x), 3) and f(x) produced an error.

Just FYI, my specific use case here, which I admit is rather different from typical package testing, is that I'm using tinytest to grade student programming assignments. Sometimes their code crashes for various edge cases, but I still want to know how they did on the other cases. I realize that I can wrap the assertions in try()...this isn't bad, but kind of tedious / cluttered.

[markvanderloo]

Hi there,

In general we can not guarantee that code lines after an error are even possible to run, because that may depend on the previous lines having been executed correctly.

[pbreheny]

Well, I certainly agree that stopping execution when an unexpected error is encountered must be the default. But similar to base R (catch.script.errors) and knitr (error=TRUE), I would argue that the option to keep going on errors is occasionally useful.