Skip to content

Commit 74738a1

Browse files
author
Nils Bars
committed
Update README with current API surface
Add test registration decorators, TestResult, instance info helpers, and run_with_payload. Fix outdated function names and descriptions.
1 parent 784841b commit 74738a1

1 file changed

Lines changed: 22 additions & 12 deletions

File tree

README.md

Lines changed: 22 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -17,29 +17,39 @@ python3 setup.py bdist_wheel
1717

1818
## Provided functionality
1919

20+
### Test Registration
21+
* `@environment_test(task_name="default")` - register a function as an environment check (must return `bool`)
22+
* `@submission_test(task_name="default")` - register a function as a submission test (must return `bool` or `TestResult`)
23+
* `@extended_submission_test(task_name="default")` - register an extended submission test
24+
* `TestResult(success: bool, score: Optional[float])` - return type for scored submission tests
25+
* `run_tests()` - execute all registered tests (deprecated; called automatically by `task.py`)
26+
* `test_result_will_be_submitted()` - check whether the current run is a final submission or just a check
27+
* `get_instance_info()` - retrieve instance metadata (`InstanceInfo` dataclass)
28+
* `get_user_environment()` - retrieve environment variables set by the user
29+
2030
### Drop & Execute
21-
* `drop_privileges` and `drop_privileges_to(uid, gid)` decorates allow to execute function in unprivileged context
22-
* TODO: ctxt mgr?
31+
* `drop_privileges` and `drop_privileges_to(uid, gid)` decorators allow executing a function in unprivileged context
2332

2433
### Asserts
25-
Return Boolean value (False) instead of AssertionError in case of failure
26-
* `assert_is_file` - check whether given path points to file
27-
* `assert_is_executable` - check whether given path points to executable file
28-
* `assert_is_dir` - check whether given path points to directory
34+
Return `False` instead of raising `AssertionError` on failure
35+
* `assert_is_file` - check whether given path points to a file
36+
* `assert_is_exec` - check whether given path points to an executable file
37+
* `assert_is_dir` - check whether given path points to a directory
2938

3039
### Utils
3140
Colored output
3241
* `print_ok` - prints text green
3342
* `print_warn` - prints text yellow
3443
* `print_err` - prints text red
3544

36-
Run (shell) command after dropping privileges. Wraps subprocess.run
45+
Run (shell) command after dropping privileges. Wraps `subprocess.run`
3746
* `run` - subprocess.run
38-
* `run_shell` - subprocess.run with `shell=True`.
47+
* `run_shell` - subprocess.run with `shell=True`
48+
* `run_capture_output` - subprocess.run capturing stdout/stderr
49+
* `run_with_payload` - run a binary with specific stdin payload
3950

4051
### Checks
4152
Various checks to run on instances
42-
* `run_pylint` - pylint linter - disable by setting environment variable NO_LINT="1"
43-
* `run_mypy` - mypy type checker - disable by setting environment variable NO_LINT="1"
44-
* `contains_flag` - execute given python script with python3 and check whether output contains a given flag value
45-
* TODO: stuff like: `exec_trace = exec(/submitted_file); assert_executes_syscalls(exec_trace, 'write /flag' )`
53+
* `run_pylint` - pylint linter (disable with `NO_LINT="1"`)
54+
* `run_mypy` - mypy type checker (disable with `NO_LINT="1"`)
55+
* `contains_flag` - execute given python script and check whether output contains a given flag value

0 commit comments

Comments
 (0)