Better fix for re-running scores after tester change on active challenge

Author: jmgasper

docs/Marathon Match Setup and Management.md

@@ -27,7 +27,7 @@ Before starting:
 6. Save the scorer config separately with `Save Scorer Config`.
 7. Launch the challenge after the scorer config is saved and operationally ready.
 8. Monitor submissions from the Work app Submissions tab, including score links, artifacts, and ECS runner logs.
-9. If the tester changes during the challenge, publish a new version, update the scorer config, and rerun the latest submissions.
+9. If the tester changes during the challenge, publish a new version and update the scorer config; active configs automatically rerun the latest submissions for the open scoring phase.
 
 ## Challenge Creation
 
@@ -284,8 +284,8 @@ Use this flow when the tester source changes after the challenge is active.
 2. In the scorer section, select the current scorer and click `New Version`.
 3. Create the new version with the updated Java source.
 4. Wait until compilation reaches `SUCCESS`.
-5. Save the scorer config so the challenge points at the new tester ID.
-6. Trigger a rerun of latest submissions for the open Submission/Provisional phase using the Work app rerun control or `POST /v6/marathon-match/challenge/:challengeId/rerun`.
+5. Save the scorer config so the challenge points at the new tester ID. If the config is active, this automatically reruns the latest submission from each submitter for the open scoring phase.
+6. Use the Work app rerun control or `POST /v6/marathon-match/challenge/:challengeId/rerun` only when you need to manually repeat the same latest-submission rescore.
 7. If Review/System scores also need to be regenerated, call `POST /v6/marathon-match/challenge/:challengeId/rerun/system`.
 8. Monitor the rerun ECS tasks, runner logs, artifacts, and review scores.
 
@@ -296,12 +296,12 @@ Submission/Provisional rerun behavior:
 - Requires at least one open challenge phase.
 - Requires the scorer config to be active.
 - Requires the selected tester to be compiled successfully.
-- Requires a phase config that matches the currently open challenge phase.
+- Requires at least one phase config that matches the currently open challenge phase.
 - Fetches `isLatest=true` submissions from Submission API.
-- Queues one ECS task for the latest submission per member.
-- Uses the current open phase's scorer seeds and test count. During Submission this is normally the `PROVISIONAL` config.
+- Queues one ECS task for the latest submission per member per matching phase config.
+- Uses each matching phase config's scorer seeds and test count. During Submission this normally includes both `EXAMPLE` and `PROVISIONAL` when both are mapped to the Submission phase.
 
-The response includes the queued submission IDs and ECS task IDs or per-submission errors.
+The response includes the queued submission IDs, phase config type, and ECS task IDs or per-submission errors.
 
 System rerun behavior:
 
@@ -399,7 +399,7 @@ During the challenge:
 - Watch Submission tab scores and timestamps.
 - Spot-check artifacts for early submissions.
 - Open runner logs for failures, timeouts, compile errors, and missing output.
-- If tester source changes, publish a new version and rerun latest submissions.
+- If tester source changes, publish a new version and update the scorer config. Active configs automatically rerun latest submissions; use the manual rerun control if you need to repeat that rescore.
 
 After submission closes:
 

docs/marathon-match-setup.md

@@ -158,9 +158,9 @@ To switch an active challenge to a newer tester:
 1. Publish the new tester version with `PUT /v6/marathon-match/testers/:id`
 2. Wait for `GET /v6/marathon-match/testers/:newTesterId` to return `compilationStatus = SUCCESS`
 3. Update the challenge config with `PUT /v6/marathon-match/challenge/:challengeId` and the new `testerId`
-4. Trigger a rescore of current competitors with `POST /v6/marathon-match/challenge/:challengeId/rerun`
+4. The update automatically reruns the latest submission from each submitter when the config is active. You can also trigger the same rescore manually with `POST /v6/marathon-match/challenge/:challengeId/rerun`.
 
-The standard rerun endpoint selects `isLatest` submissions for the challenge in received order and launches ECS scorer tasks for them in parallel using the scorer config for the currently open challenge phase. This does not happen automatically when `testerId` changes, so the rerun call is the operational step that recalculates Submission/Provisional scores.
+The standard rerun endpoint selects `isLatest` submissions for the challenge in received order and launches ECS scorer tasks for every scorer config that matches the currently open challenge phase. During Submission, this normally reruns both `EXAMPLE` and `PROVISIONAL` scoring when both configs are mapped to the Submission phase, so the latest displayed scores are recalculated with the current tester.
 
 To restart Review/System tests, call:
 

docs/marathon-processor-specification-and-scoring-terminology.md

@@ -26,19 +26,19 @@ Supported source extensions are:
 
 | Extension   | Language / runtime                                                           |
 | ----------- | ---------------------------------------------------------------------------- |
-| `.cpp`      | C++                                                                          |
-| `.java`     | Java                                                                         |
+| `.cpp`      | C++23 / GNU++23 using G++ 14.2.0                                             |
+| `.java`     | Java 11 using Temurin OpenJDK 11.0.31+11 and `javac --release 11`            |
 | `.py`       | Python 3.12                                                                  |
 | `.cs`       | C# using Mono                                                                |
 | `.cs_net10` | C# using .NET 10 / C# 14                                                     |
 | `.cs_net7`  | C# using .NET 7 / C# 11                                                      |
-| `.rs`       | Rust latest stable                                                           |
+| `.rs`       | Rust 2024 edition using `rustc 1.96.0` stable as last verified               |
 
 The runner normalizes the selected source file into a temporary compile workspace before building or executing it.
 
 ### C++
 
-C++ submissions are compiled with `g++` using GNU++23 and a fixed
+C++ submissions are compiled with G++ 14.2.0 using GNU++23 and a fixed
 `x86-64` architecture target with generic tuning. The runner does not use
 `-march=native`, so binaries do not depend on the specific Fargate host that
 compiled the submission.
@@ -50,7 +50,8 @@ g++ -std=gnu++23 -O3 -march=x86-64 -mtune=generic Solution.cpp -o Solution
 
 ### Java
 
-Java submissions are compiled and executed with:
+Java submissions are compiled with Temurin OpenJDK `javac 11.0.31` and
+executed on the Java 11 runtime:
 
 ```bash
 javac --release 11 Solution.java
@@ -74,7 +75,8 @@ python3 Solution.py
 
 ### Rust
 
-Rust submissions use the `.rs` extension and are compiled as a single source file with the latest stable Rust compiler:
+Rust submissions use the `.rs` extension and are compiled as a single source
+file with `rustc 1.96.0` stable as last verified:
 
 ```bash
 rustc --edition=2024 -O Solution.rs -o Solution

docs/submission-phase-scoring.md

@@ -120,7 +120,7 @@ Use:
 
 `POST /v6/marathon-match/challenge/:challengeId/rerun`
 
-This endpoint selects the latest submission for each member in received order and launches ECS scorer tasks in parallel using the scorer config that matches the currently open challenge phase. During Submission this is normally the challenge's `PROVISIONAL` phase config. Use it after changing the tester or when current latest submissions need to be rescored without waiting for new submission events.
+This endpoint selects the latest submission for each member in received order and launches ECS scorer tasks in parallel for every scorer config that matches the currently open challenge phase. During Submission this normally includes both `EXAMPLE` and `PROVISIONAL` when they are mapped to the Submission phase. It is also triggered automatically after an active challenge's `testerId` changes, and can be called manually when current latest submissions need to be rescored without waiting for new submission events.
 
 For Review/System scoring reruns, use:
 

src/api/marathon-match-config/marathon-match-config.controller.ts

@@ -142,7 +142,7 @@ export class MarathonMatchConfigController {
   @ApiResponse({
     status: 400,
     description:
-      'Challenge/config inactive, challenge has no open phase, tester is not compiled successfully, or PROVISIONAL phase config is missing.',
+      'Challenge/config inactive, challenge has no open phase, tester is not compiled successfully, or no phase config matches the open phase.',
   })
   @ApiResponse({ status: 403, description: 'Forbidden.' })
   @ApiResponse({ status: 404, description: 'Marathon match config not found.' })

src/api/marathon-match-config/marathon-match-config.service.spec.ts

@@ -254,6 +254,7 @@ describe('MarathonMatchConfigService', () => {
         results: [
           {
             submissionId: 'submission-1',
+            configType: PhaseConfigType.PROVISIONAL,
             taskId: 'task-1',
           },
         ],
@@ -430,6 +431,182 @@ describe('MarathonMatchConfigService', () => {
     );
   });
 
+  it('reruns all phase configs mapped to the currently open submission phase', async () => {
+    const { service, ecsService, httpService, m2mService, prisma } =
+      createService();
+    const user = {
+      isMachine: false,
+      userId: '40051399',
+    } as never;
+
+    prisma.marathonMatchConfig.findUnique.mockResolvedValue({
+      id: 'config-1',
+      challengeId: '30000123',
+      active: true,
+      submissionApiUrl: 'https://submissions.example.com/v6',
+      reviewScorecardId: 'scorecard-1',
+      testerId: 'tester-1',
+      testTimeout: 90000,
+      compileTimeout: 120000,
+      taskDefinitionName: 'mm-runner',
+      taskDefinitionVersion: '7',
+      tester: {
+        compilationStatus: CompilationStatus.SUCCESS,
+      },
+      phaseConfigs: [
+        {
+          id: 'phase-provisional',
+          configType: PhaseConfigType.PROVISIONAL,
+          phaseId: 'submission-phase',
+          startSeed: BigInt(100),
+          numberOfTests: 50,
+        },
+        {
+          id: 'phase-example',
+          configType: PhaseConfigType.EXAMPLE,
+          phaseId: 'submission-phase',
+          startSeed: BigInt(1),
+          numberOfTests: 10,
+        },
+        {
+          id: 'phase-system',
+          configType: PhaseConfigType.SYSTEM,
+          phaseId: 'system-phase',
+          startSeed: BigInt(1000),
+          numberOfTests: 5000,
+        },
+      ],
+    });
+    m2mService.getM2MToken.mockResolvedValue('m2m-token');
+    httpService.get
+      .mockReturnValueOnce(
+        of({
+          data: {
+            status: 'ACTIVE',
+            phases: [
+              {
+                phaseId: 'registration-phase',
+                isOpen: true,
+                actualStartDate: '2026-06-01T00:00:00.000Z',
+              },
+              {
+                phaseId: 'submission-phase',
+                isOpen: true,
+                actualStartDate: '2026-06-01T00:00:00.000Z',
+              },
+            ],
+          },
+        }),
+      )
+      .mockReturnValueOnce(
+        of({
+          data: {
+            result: {
+              content: [
+                {
+                  id: 'submission-1',
+                  memberId: '40051399',
+                  submittedDate: '2026-06-02T00:00:00.000Z',
+                  isLatest: true,
+                  virusScan: true,
+                },
+                {
+                  id: 'submission-2',
+                  memberId: '40051400',
+                  submittedDate: '2026-06-03T00:00:00.000Z',
+                  isLatest: true,
+                  virusScan: true,
+                },
+              ],
+            },
+          },
+          headers: {
+            'x-total-pages': '1',
+          },
+        }),
+      );
+    ecsService.launchScorerTask.mockImplementation((...args: unknown[]) => {
+      const submissionId = String(args[1]);
+      const scoringPhase = args[3] as { configType: PhaseConfigType };
+      return Promise.resolve({
+        taskArn: `arn:aws:ecs:task/${scoringPhase.configType}-${submissionId}`,
+        taskId: `${scoringPhase.configType}-${submissionId}`,
+      });
+    });
+
+    const result = await service.rerunLatestSubmissions('30000123', user);
+
+    expect(result).toEqual({
+      challengeId: '30000123',
+      submissionsQueued: 2,
+      results: [
+        {
+          submissionId: 'submission-1',
+          configType: PhaseConfigType.EXAMPLE,
+          taskArn: 'arn:aws:ecs:task/EXAMPLE-submission-1',
+          taskId: 'EXAMPLE-submission-1',
+        },
+        {
+          submissionId: 'submission-2',
+          configType: PhaseConfigType.EXAMPLE,
+          taskArn: 'arn:aws:ecs:task/EXAMPLE-submission-2',
+          taskId: 'EXAMPLE-submission-2',
+        },
+        {
+          submissionId: 'submission-1',
+          configType: PhaseConfigType.PROVISIONAL,
+          taskArn: 'arn:aws:ecs:task/PROVISIONAL-submission-1',
+          taskId: 'PROVISIONAL-submission-1',
+        },
+        {
+          submissionId: 'submission-2',
+          configType: PhaseConfigType.PROVISIONAL,
+          taskArn: 'arn:aws:ecs:task/PROVISIONAL-submission-2',
+          taskId: 'PROVISIONAL-submission-2',
+        },
+      ],
+    });
+    expect(ecsService.launchScorerTask).toHaveBeenCalledTimes(4);
+    expect(ecsService.launchScorerTask).toHaveBeenNthCalledWith(
+      1,
+      '30000123',
+      'submission-1',
+      {
+        taskDefinitionName: 'mm-runner',
+        taskDefinitionVersion: '7',
+      },
+      {
+        configType: PhaseConfigType.EXAMPLE,
+        startSeed: BigInt(1),
+        numberOfTests: 10,
+        scorecardId: 'scorecard-1',
+      },
+      undefined,
+      {
+        memberId: '40051399',
+      },
+    );
+    expect(ecsService.launchScorerTask).toHaveBeenNthCalledWith(
+      3,
+      '30000123',
+      'submission-1',
+      {
+        taskDefinitionName: 'mm-runner',
+        taskDefinitionVersion: '7',
+      },
+      {
+        configType: PhaseConfigType.PROVISIONAL,
+        startSeed: BigInt(100),
+        numberOfTests: 50,
+        scorecardId: 'scorecard-1',
+      },
+      undefined,
+      {
+        memberId: '40051399',
+      },
+    );
+  });
+
   it('marks latest submissions failed without launching when virus scan has not passed', async () => {
     const {
       service,
@@ -530,6 +707,7 @@ describe('MarathonMatchConfigService', () => {
       results: [
         {
           submissionId: 'submission-1',
+          configType: PhaseConfigType.PROVISIONAL,
           error:
             'Marathon Match PROVISIONAL scoring skipped because the submission has not passed virus scanning.',
         },
@@ -878,6 +1056,7 @@ describe('MarathonMatchConfigService', () => {
         submissionsQueued: 10,
         results: submissionIds.map((submissionId) => ({
           submissionId,
+          configType: PhaseConfigType.PROVISIONAL,
           taskArn: `arn:aws:ecs:us-east-1:123456789012:task/${submissionId}`,
           taskId: `task-${submissionId}`,
         })),