@@ -17,29 +17,39 @@ python3 setup.py bdist_wheel
1717
1818## Provided functionality
1919
20+ ### Test Registration
21+ * ` @environment_test(task_name="default") ` - register a function as an environment check (must return ` bool ` )
22+ * ` @submission_test(task_name="default") ` - register a function as a submission test (must return ` bool ` or ` TestResult ` )
23+ * ` @extended_submission_test(task_name="default") ` - register an extended submission test
24+ * ` TestResult(success: bool, score: Optional[float]) ` - return type for scored submission tests
25+ * ` run_tests() ` - execute all registered tests (deprecated; called automatically by ` task.py ` )
26+ * ` test_result_will_be_submitted() ` - check whether the current run is a final submission or just a check
27+ * ` get_instance_info() ` - retrieve instance metadata (` InstanceInfo ` dataclass)
28+ * ` get_user_environment() ` - retrieve environment variables set by the user
29+
2030### Drop & Execute
21- * ` drop_privileges ` and ` drop_privileges_to(uid, gid) ` decorates allow to execute function in unprivileged context
22- * TODO: ctxt mgr?
31+ * ` drop_privileges ` and ` drop_privileges_to(uid, gid) ` decorators allow executing a function in unprivileged context
2332
2433### Asserts
25- Return Boolean value ( False) instead of AssertionError in case of failure
26- * ` assert_is_file ` - check whether given path points to file
27- * ` assert_is_executable ` - check whether given path points to executable file
28- * ` assert_is_dir ` - check whether given path points to directory
34+ Return ` False ` instead of raising ` AssertionError ` on failure
35+ * ` assert_is_file ` - check whether given path points to a file
36+ * ` assert_is_exec ` - check whether given path points to an executable file
37+ * ` assert_is_dir ` - check whether given path points to a directory
2938
3039### Utils
3140Colored output
3241* ` print_ok ` - prints text green
3342* ` print_warn ` - prints text yellow
3443* ` print_err ` - prints text red
3544
36- Run (shell) command after dropping privileges. Wraps subprocess.run
45+ Run (shell) command after dropping privileges. Wraps ` subprocess.run `
3746* ` run ` - subprocess.run
38- * ` run_shell ` - subprocess.run with ` shell=True ` .
47+ * ` run_shell ` - subprocess.run with ` shell=True `
48+ * ` run_capture_output ` - subprocess.run capturing stdout/stderr
49+ * ` run_with_payload ` - run a binary with specific stdin payload
3950
4051### Checks
4152Various checks to run on instances
42- * ` run_pylint ` - pylint linter - disable by setting environment variable NO_LINT="1"
43- * ` run_mypy ` - mypy type checker - disable by setting environment variable NO_LINT="1"
44- * ` contains_flag ` - execute given python script with python3 and check whether output contains a given flag value
45- * TODO: stuff like: ` exec_trace = exec(/submitted_file); assert_executes_syscalls(exec_trace, 'write /flag' ) `
53+ * ` run_pylint ` - pylint linter (disable with ` NO_LINT="1" ` )
54+ * ` run_mypy ` - mypy type checker (disable with ` NO_LINT="1" ` )
55+ * ` contains_flag ` - execute given python script and check whether output contains a given flag value
0 commit comments