Updates for ECS runner for feedback items

Author: jmgasper

docs/full-marathon-match-test.md

@@ -1,6 +1,6 @@
 # Full Marathon Match Test Script
 
-`scripts/run-full-marathon-match-test.mjs` drives a full Marathon Match smoke run against deployed Topcoder APIs. It creates a Marathon Match challenge through challenge-api-v6, creates a custom tester from `tester.java`, configures the Marathon Match challenge, registers submitters, submits fixture submissions during the Submission phase, waits for provisional scoring and artifacts, opens Review, waits for SYSTEM scoring on each member's latest submission, and waits for autopilot-v6 to close Review and complete the challenge.
+`scripts/run-full-marathon-match-test.mjs` drives a full Marathon Match smoke run against deployed Topcoder APIs. It creates a Marathon Match challenge through challenge-api-v6, creates a custom tester from `tester.java`, configures the Marathon Match challenge, registers submitters, submits fixture submissions during the Submission phase, waits for provisional scoring and the example member artifact, opens Review, waits for SYSTEM scoring on each member's latest submission, and waits for autopilot-v6 to close Review and complete the challenge.
 
 ## Fixture Directory
 
@@ -72,7 +72,7 @@ If `submissions` is omitted, submission files can be placed directly in the fixt
 
 Registration and Submission durations are seconds. They default to `3600` each and can be set with `--registration-duration-seconds`, `--submission-duration-seconds`, `REGISTRATION_DURATION_SECONDS`, `SUBMISSION_DURATION_SECONDS`, or `challenge.phaseDurations`.
 
-The tester source can be a standard Topcoder Marathon tester with a `main(...)` method. The ECS runner handles submission source discovery, compilation, seed execution, aggregate scoring, and `metadata.testScores` output with member-visible testcase ordinals. Custom tester-level `runTester(String, com.topcoder.scorer.models.ScorerConfig)` methods are still supported for special cases.
+The tester source can be a standard Topcoder Marathon tester with a `main(...)` method. The ECS runner handles submission source discovery, compilation, seed execution, aggregate scoring, `metadata.testScores` output with member-visible testcase ordinals, and EXAMPLE `output.txt` entries with actual seed and seed-score details. Custom tester-level `runTester(String, com.topcoder.scorer.models.ScorerConfig)` methods are still supported for special cases.
 
 For advanced create payload fields, add `challenge.createPayload` with any valid `POST /v6/challenges` body fields. The script still applies the configured Registration and Submission duration overrides.
 
@@ -139,7 +139,7 @@ Pass `--challenge-id "<challenge_uuid>"` to reuse an existing challenge instead
 
 ## Validation
 
-During Submission, every created submission must receive a provisional review summation with a numeric `aggregateScore`. By default, each submission must also have at least one artifact attached through `GET /v6/submissions/:submissionId/artifacts`; override with `--min-artifacts 0` if a tester does not emit artifacts.
+During Submission, every created submission must receive a provisional review summation with a numeric `aggregateScore`. By default, each submission must also have at least one artifact attached through `GET /v6/submissions/:submissionId/artifacts`; in the standard config this is the EXAMPLE member-visible artifact, because PROVISIONAL scoring does not publish competitor-visible artifacts. Override with `--min-artifacts 0` if a tester does not emit artifacts.
 
 During Review, the script waits for a SYSTEM review summation on the latest submission for each member. If `expectedScores.provisional` or `expectedScores.system` is present in the manifest, the script checks the scored value using `--score-tolerance` (default `0.000001`).
 

docs/marathon-processor-specification-and-scoring-terminology.md

@@ -80,10 +80,11 @@ file with `rustc 1.96.0` stable as last verified:
 
 ```bash
 rustc --edition=2024 -O Solution.rs -o Solution
-./Solution
+RUST_BACKTRACE=1 ./Solution
 ```
 
 The ECS runner installs Rust through `rustup` with `RUSTUP_TOOLCHAIN=stable`, so the exact compiler patch version advances when the runner image is rebuilt. As of the verification date above, the stable channel resolves to `rustc 1.96.0`.
+Rust submissions are executed with `RUST_BACKTRACE=1` inside the ECS runner so panic diagnostics include a backtrace in captured stderr.
 
 ### C# with Mono
 
@@ -242,10 +243,10 @@ For Rust submissions, compile first and pass the compiled binary:
 
 ```bash
 rustc --edition=2024 -O Solution.rs -o Solution
-java -jar Tester.jar -exec "./Solution" -seed 1
+java -jar Tester.jar -exec "env RUST_BACKTRACE=1 ./Solution" -seed 1
 ```
 
-Members may print debug information to standard error. The tester captures solution output and error text and the processor includes relevant output in the scoring artifacts.
+Members may print debug information to standard error. The tester captures solution output and error text. Internal artifacts include per-seed `stdout/{seed}.txt` and `stderr/{seed}.txt` files for operator diagnostics, while member-visible EXAMPLE artifacts expose only the approved output summary for the scoring phase, including each seed and the score received for that seed.
 
 ## Download and Access Submissions
 
@@ -280,10 +281,11 @@ Example and Provisional scoring can both be mapped to the same open Submission p
 
 System scoring is dispatched during Review for the pending review and uses the configured System seed range.
 
-For each executed test, the generic runner records member-visible results with
-1-based testcase ordinals rather than configured seed values:
+For each executed EXAMPLE test, the generic runner records member-visible
+results with 1-based testcase ordinals and the actual configured seed values:
 
 - testcase ordinal
+- actual seed value
 - raw tester score
 - run time in milliseconds
 - tester or solution error text

docs/submission-phase-scoring.md

@@ -101,7 +101,7 @@ Runner task logs are written to CloudWatch using the submission runner log strea
 
 For `PROVISIONAL` and `SYSTEM` scoring, the runner writes progress to the phase review summation metadata through `POST /v6/marathon-match/internal/scoring-progress`. Review API exposes this as `reviewSummation.metadata.testProcess` (`provisional` or `system`), `reviewSummation.metadata.testProgress` (`0` to `1`), and `reviewSummation.metadata.testStatus` (`IN PROGRESS`, `SUCCESS`, or `FAILED`) when metadata is included in the response. In-progress summations keep a neutral placeholder score and must be rendered as unavailable based on `testStatus`; only `FAILED` progress uses the failed-score sentinel. SYSTEM timeout failures also include `reviewSummation.metadata.timed_out = true`.
 
-For generic runner artifacts, the public provisional/system `output.txt` includes testcase ordinal, runtime, runner/tester errors, and submitted-solution stderr, but excludes submitted-solution stdout and raw per-test scores. Per-test score details remain in the private internal `reviews.json` artifact and callback metadata.
+For generic runner artifacts, only EXAMPLE scoring uploads the member-visible public artifact. Its public `output.txt` includes each testcase ordinal, actual seed, seed score, runtime, runner/tester errors, and submitted-solution stderr, while submitted-solution stdout stays private. Every generic scoring phase uploads private internal artifacts: per-test score details remain in `reviews.json` and callback metadata, while captured submitted-solution output is split into `stdout/{seed}.txt` and `stderr/{seed}.txt` files for each actual seed. PROVISIONAL and SYSTEM scoring do not publish competitor-visible artifacts.
 
 When more than one phase config matches the currently open challenge phases, the handler launches one scorer task per match. This is the supported way to run both `EXAMPLE` and `PROVISIONAL` scoring from the same Submission phase.
 

ecs-runner/README.md

@@ -23,6 +23,7 @@ This image is the runtime container for marathon match scoring tasks launched by
 - Downloaded tester JARs are mode `0400` runner-owned files. The serialized scorer config starts as a mode `0400` runner-owned handoff file and the child JVM deletes it immediately after loading it, before tester or submitted solution code executes.
 - Generic Marathon seed execution resets scorer-owned writable state before and after every test case, so files written by one seed are not visible to later seeds in the same submission run. The Java child invokes the setuid `mm-scorer-isolate --cleanup-scorer-state` mode, which scans only fixed writable roots such as `/tmp`, `/var/tmp`, `/dev/shm`, and the scorer home, and removes only entries owned by uid `10002`.
 - Generic submitted solution commands run under a Landlock filesystem allowlist. Runtime and toolchain files are readable, `/proc/self/maps` is readable for glibc/Mono stack introspection, `/tmp` and the scorer home are writable, and infrastructure-revealing paths such as `/etc/hostname`, `/etc/resolv.conf`, `/proc/self/cgroup`, `/proc/self/mounts`, and proc network tables are not readable by submitted code.
+- Rust submissions run with `RUST_BACKTRACE=1` so panic stderr captured in `output.txt` includes a backtrace.
 - Artifact previews and artifact zip uploads include only non-symlink regular files from the runner artifact directories. Submitted symlinks are ignored instead of being dereferenced by the trusted parent runner.
 - Submitted solution processes and their fork/exec children cannot use `io_uring` and can create only `AF_UNIX` sockets. These restrictions are kernel seccomp filters inherited across fork/exec, so clearing `LD_PRELOAD` in a spawned child process does not restore INET or INET6 socket access. Outbound network access from the submission itself is therefore blocked even though the parent runner still has the trusted egress it needs.
 - The child JVM runs standard Topcoder Marathon testers through the generic runner flow. Custom tester `runTester(...)` result maps remain supported for advanced cases, but standard testers do not need ECS-specific code.
@@ -141,7 +142,7 @@ Once this is saved and the config is active, new scoring launches use the new EC
 
 The runner rejects output once generated public `output.txt` content or a public/private artifact archive would exceed `MM_RUNNER_MAX_OUTPUT_BYTES`. The default is `10000000` bytes. The trusted parent runner passes the resolved limit into the isolated tester child, so generated generic-runner output and uploaded artifact zips use the same cap.
 
-Generic runner public `output.txt` entries show testcase ordinal, runtime, runner/tester errors, and submitted-solution stderr. They intentionally omit submitted-solution stdout and raw per-test scores. The private internal artifact (`reviews.json` in the `*-internal` zip) carries the per-test `testScores` used for diagnostics and score verification.
+Generic runner member-visible artifacts are uploaded only for EXAMPLE scoring. Their public `output.txt` entries show testcase ordinal, actual seed, seed score, runtime, runner/tester errors, and submitted-solution stderr, while intentionally omitting submitted-solution stdout. Every generic scoring phase uploads private internal artifacts in the `*-internal` zip: `reviews.json` carries per-test `testScores`, and `stdout/{seed}.txt` plus `stderr/{seed}.txt` carry captured submitted-solution output for each actual seed. PROVISIONAL and SYSTEM scoring do not publish competitor-visible artifacts.
 
 ## Local smoke test
 

ecs-runner/boilerplate/checker/solution_rust.sh

@@ -4,4 +4,4 @@ TIMEOUT=$2
 COMPILE_TIMEOUT=$3
 
 timeout ${COMPILE_TIMEOUT}s rustc --edition=2024 -O /workdir/${NAME}.rs -o /workdir/${NAME}
-echo "timeout ${TIMEOUT}s /workdir/${NAME}" > /workdir/command.txt
+echo "timeout ${TIMEOUT}s env RUST_BACKTRACE=1 /workdir/${NAME}" > /workdir/command.txt