Stability fixes and a tweak for a Python running bug found

Author: jmgasper

ECS_RUNNER_LIFETIME.md

@@ -98,7 +98,7 @@ Inside the task, the trusted Java parent runner:
 1. Fetches the challenge config, tester JAR, and tester metadata from `marathon-match-api-v6`.
 2. Downloads the submission from the configured Submission API URL.
 3. Posts an initial progress callback for `PROVISIONAL` and `SYSTEM` runs.
-4. Starts the tester in an isolated child JVM. Submitted solution commands run as the separate `scorer` user with scrubbed environment, restricted filesystem access, and no outbound INET/INET6 socket creation.
+4. Starts the tester in an isolated child JVM. Submitted solution commands run as the separate `scorer` user with scrubbed environment, restricted filesystem access, and no outbound INET/INET6 socket creation. For standard generic-runner seeds, the child invokes `mm-scorer-isolate --cleanup-scorer-state` before and after each test case so scorer-owned files in fixed writable roots are reset between seeds.
 5. Uploads public/private artifacts back through Submission API.
 6. Posts the final scoring callback to `POST /v6/marathon-match/internal/scoring-results`.
 7. Exits with code `0` only after the final callback succeeds.

README.md

@@ -157,6 +157,7 @@ The ECS task still needs trusted outbound access to fetch challenge config, down
 - The trusted parent runner holds `ACCESS_TOKEN`, performs network calls, and never loads untrusted submission code directly.
 - The parent launches a separate child JVM through `mm-runner-isolate` with a scrubbed environment, so submission processes do not inherit the bearer token or other runner env vars.
 - Generic submitted solution commands run through `mm-scorer-isolate` as the separate non-root `scorer` user. Downloaded tester JARs and serialized scorer config are kept runner-owned mode `0400`, so submitted code cannot read or modify them from `/tmp`.
+- Standard generic-runner seed execution asks `mm-scorer-isolate` to reset scorer-owned writable state before and after each test case. The cleanup helper only scans fixed writable roots such as `/tmp`, `/var/tmp`, `/dev/shm`, and the scorer home, and only removes entries owned by the scorer UID.
 - Generic submitted solution commands also run under a filesystem allowlist that permits runtime/toolchain reads and scorer temp writes but does not permit reading infrastructure paths such as `/etc/hostname`, `/etc/resolv.conf`, `/proc/self/cgroup`, `/proc/self/mounts`, or proc network tables.
 - Native wrappers block `io_uring` and creation of non-`AF_UNIX` sockets for submitted solution processes and their fork/exec children, so submissions cannot open live outbound network connections.
 - Standard Topcoder Marathon testers run through the generic runner flow, which creates the callback score payload from trusted runner code. Custom tester `runTester(...)` result maps remain supported for advanced cases.

ecs-runner/README.md

@@ -21,7 +21,7 @@ This image is the runtime container for marathon match scoring tasks launched by
 - The tester executes in a separate child JVM launched through `mm-runner-isolate` as uid/gid `10001` (`runner`) with a scrubbed environment, so `ACCESS_TOKEN` and other runner env vars are not inherited by untrusted code.
 - Generic submitted solution commands execute through the setuid-root `mm-scorer-isolate` bridge as uid/gid `10002` (`scorer`). The bridge drops its supervisor back to the invoking `runner` uid after it forks the solution child, retaining only `CAP_KILL` so tester timeouts can still terminate the lower-privilege solution process group.
 - Downloaded tester JARs are mode `0400` runner-owned files. The serialized scorer config starts as a mode `0400` runner-owned handoff file and the child JVM deletes it immediately after loading it, before tester or submitted solution code executes.
-- Generic Marathon seed execution resets scorer-owned writable state, including `/tmp`, before and after every test case, so files written by one seed are not visible to later seeds in the same submission run.
+- Generic Marathon seed execution resets scorer-owned writable state before and after every test case, so files written by one seed are not visible to later seeds in the same submission run. The Java child invokes the setuid `mm-scorer-isolate --cleanup-scorer-state` mode, which scans only fixed writable roots such as `/tmp`, `/var/tmp`, `/dev/shm`, and the scorer home, and removes only entries owned by uid `10002`.
 - Generic submitted solution commands run under a Landlock filesystem allowlist. Runtime and toolchain files are readable, `/tmp` and the scorer home are writable, and infrastructure-revealing paths such as `/etc/hostname`, `/etc/resolv.conf`, `/proc/self/cgroup`, `/proc/self/mounts`, and proc network tables are not readable by submitted code.
 - Artifact previews and artifact zip uploads include only non-symlink regular files from the runner artifact directories. Submitted symlinks are ignored instead of being dereferenced by the trusted parent runner.
 - Submitted solution processes and their fork/exec children cannot use `io_uring` and can create only `AF_UNIX` sockets. These restrictions are kernel seccomp filters inherited across fork/exec, so clearing `LD_PRELOAD` in a spawned child process does not restore INET or INET6 socket access. Outbound network access from the submission itself is therefore blocked even though the parent runner still has the trusted egress it needs.

ecs-runner/scripts/mm-net-isolate.c

@@ -177,6 +177,221 @@ static void close_extra_fds(void) {
     }
 }
 
+/**
+ * Checks whether a directory entry is "." or "..".
+ *
+ * name is the entry name returned by readdir. Returns 1 for the current or
+ * parent directory pseudo-entry and 0 for all other names.
+ */
+static int is_current_or_parent_directory(const char *name) {
+    return strcmp(name, ".") == 0 || strcmp(name, "..") == 0;
+}
+
+/**
+ * Builds a child path under a parent directory.
+ *
+ * buffer receives the joined path, buffer_size is its capacity, parent is the
+ * directory being scanned, and name is a direct child entry. Returns 0 when the
+ * joined path fits and 1 when it would be truncated.
+ */
+static int join_child_path(
+    char *buffer,
+    size_t buffer_size,
+    const char *parent,
+    const char *name
+) {
+    int written = snprintf(buffer, buffer_size, "%s/%s", parent, name);
+    if (written < 0 || (size_t) written >= buffer_size) {
+        fprintf(stderr, "cleanup path too long under %s\n", parent);
+        return 1;
+    }
+    return 0;
+}
+
+/**
+ * Removes one untrusted path tree without following symlinks.
+ *
+ * The cleanup entry point is used by the Java runner between test cases. It
+ * runs with the scorer helper's setuid-root privilege, but it only removes
+ * entries that were already selected as owned by the scorer UID. path is the
+ * selected top-level entry. Returns 0 when the tree was removed or already
+ * disappeared and 1 when any entry could not be removed.
+ */
+static int remove_tree_no_follow(const char *path) {
+    struct stat stat_buffer;
+    if (lstat(path, &stat_buffer) != 0) {
+        if (errno == ENOENT || errno == ENOTDIR) {
+            return 0;
+        }
+        fprintf(stderr, "lstat(%s): %s\n", path, strerror(errno));
+        return 1;
+    }
+
+    if (S_ISDIR(stat_buffer.st_mode)) {
+        DIR *dir = opendir(path);
+        struct dirent *entry;
+        int failed = 0;
+
+        if (dir == NULL) {
+            if (errno == ENOENT || errno == ENOTDIR) {
+                return 0;
+            }
+            fprintf(stderr, "opendir(%s): %s\n", path, strerror(errno));
+            return 1;
+        }
+
+        while ((entry = readdir(dir)) != NULL) {
+            char child_path[PATH_MAX];
+            if (is_current_or_parent_directory(entry->d_name)) {
+                continue;
+            }
+            if (
+                join_child_path(
+                    child_path,
+                    sizeof(child_path),
+                    path,
+                    entry->d_name
+                ) != 0
+            ) {
+                failed = 1;
+                continue;
+            }
+            if (remove_tree_no_follow(child_path) != 0) {
+                failed = 1;
+            }
+        }
+
+        if (closedir(dir) != 0) {
+            fprintf(stderr, "closedir(%s): %s\n", path, strerror(errno));
+            failed = 1;
+        }
+        if (rmdir(path) != 0 && errno != ENOENT && errno != ENOTDIR) {
+            fprintf(stderr, "rmdir(%s): %s\n", path, strerror(errno));
+            failed = 1;
+        }
+        return failed;
+    }
+
+    if (unlink(path) != 0 && errno != ENOENT && errno != ENOTDIR) {
+        fprintf(stderr, "unlink(%s): %s\n", path, strerror(errno));
+        return 1;
+    }
+    return 0;
+}
+
+/**
+ * Deletes scorer-owned top-level entries under one writable directory.
+ *
+ * directory is a fixed cleanup root compiled into the helper. deleted_entries
+ * receives the number of selected top-level entries successfully removed.
+ * Returns 0 when the directory was scanned cleanly and 1 when any selected
+ * entry could not be inspected or removed. Missing directories are ignored.
+ */
+static int cleanup_scorer_owned_entries_under(
+    const char *directory,
+    int *deleted_entries
+) {
+    DIR *dir = opendir(directory);
+    struct dirent *entry;
+    int failed = 0;
+
+    if (dir == NULL) {
+        if (errno == ENOENT || errno == ENOTDIR) {
+            return 0;
+        }
+        fprintf(stderr, "opendir(%s): %s\n", directory, strerror(errno));
+        return 1;
+    }
+
+    while ((entry = readdir(dir)) != NULL) {
+        char entry_path[PATH_MAX];
+        struct stat stat_buffer;
+
+        if (is_current_or_parent_directory(entry->d_name)) {
+            continue;
+        }
+        if (
+            join_child_path(
+                entry_path,
+                sizeof(entry_path),
+                directory,
+                entry->d_name
+            ) != 0
+        ) {
+            failed = 1;
+            continue;
+        }
+
+        if (lstat(entry_path, &stat_buffer) != 0) {
+            if (errno == ENOENT || errno == ENOTDIR) {
+                continue;
+            }
+            fprintf(stderr, "lstat(%s): %s\n", entry_path, strerror(errno));
+            failed = 1;
+            continue;
+        }
+
+        if (stat_buffer.st_uid != (uid_t) MM_ISOLATED_UID) {
+            continue;
+        }
+
+        if (remove_tree_no_follow(entry_path) == 0) {
+            *deleted_entries += 1;
+        } else {
+            failed = 1;
+        }
+    }
+
+    if (closedir(dir) != 0) {
+        fprintf(stderr, "closedir(%s): %s\n", directory, strerror(errno));
+        failed = 1;
+    }
+    return failed;
+}
+
+/**
+ * Deletes scorer-owned writable state from fixed, compiled-in directories.
+ *
+ * This mode intentionally accepts no path arguments. It is called by the Java
+ * runner before and after each seed to prevent scorer-owned filesystem state
+ * from carrying across tests. Returns 0 when cleanup completed and 1 when any
+ * cleanup root or selected scorer-owned entry could not be processed.
+ */
+static int cleanup_scorer_writable_state(void) {
+    static const char *const cleanup_directories[] = {
+        "/tmp",
+        "/var/tmp",
+        "/dev/shm",
+        MM_ISOLATED_HOME
+    };
+    size_t index;
+    int deleted_entries = 0;
+    int failed = 0;
+
+    if (geteuid() != 0) {
+        fprintf(stderr, "cleanup requires setuid-root scorer helper\n");
+        return 1;
+    }
+
+    for (
+        index = 0;
+        index < sizeof(cleanup_directories) / sizeof(cleanup_directories[0]);
+        index++
+    ) {
+        if (
+            cleanup_scorer_owned_entries_under(
+                cleanup_directories[index],
+                &deleted_entries
+            ) != 0
+        ) {
+            failed = 1;
+        }
+    }
+
+    printf("%d\n", deleted_entries);
+    return failed;
+}
+
 #if MM_INSTALL_SOCKET_FILTER
 static int install_socket_filter(void) {
     struct sock_filter filter[] = {
@@ -849,6 +1064,10 @@ int main(int argc, char **argv) {
     }
 
     close_extra_fds();
+    if (argc == 2 && strcmp(argv[1], "--cleanup-scorer-state") == 0) {
+        return cleanup_scorer_writable_state();
+    }
+
     sanitize_environment();
 
 #if MM_SUPERVISE_CHILD