Make errors in question runtime, and testcase input generation more prominent. #1703.

Author: sangwinc

doc/en/STACK_question_admin/Testing.md

@@ -153,6 +153,11 @@ This test will throw an error when `n` is zero.
 
 In many situations this kind of test creation will be simpler than mapping onto student inputs.
 
+1. If you place `s_assert` in the question variables then any run-time errors will stop the question being used by students.  These errors will appear when you bulk-test the question variants.
+2. If you place `s_assert` in the feedback variables of a PRT then any run-time errors will be ignored.  
+
+Note, the purpose of `s_assert` is unit testing of question variables within question testing.
+
 ## STACK-Maxima sandbox ##
 
 It is very useful to be able to use the desktop Maxima application to test questions.  To do this it is very helpful to load all the STACK libraries.  Details on how to do this are in the [STACK-Maxima sandbox](../CAS/STACK-Maxima_sandbox.md) page.

stack/questiontest.php

@@ -160,13 +160,17 @@ public function process_results($question, $response) {
                 if ($inputresponse != '') {
                     $emptytestcase = false;
                 }
+                $inputerrors = $inputstate->errors;
+                if (!empty($question->runtimeerrors)) {
+                    $inputerrors = implode('; ', array_keys($question->runtimeerrors));
+                }
                 $results->set_input_state(
                     $inputname,
                     $inputresponse,
                     $inputstate->contentsmodified,
                     $inputstate->contentsdisplayed,
                     $inputstate->status,
-                    $inputstate->errors
+                    $inputerrors
                 );
             }
         }

stack/questiontestresult.php

@@ -344,6 +344,11 @@ public function passed() {
         if ($this->emptytestcase) {
             return false;
         }
+        foreach ($this->get_input_states() as $state) {
+            if ($state->errors !== "") {
+                return false;
+            }
+        }
         foreach ($this->get_prt_states() as $state) {
             if (!$state->testoutcome) {
                 return false;