Skip to content

rerunLatestSubmissions() always dispatches PROVISIONAL scoring regardless of the currently open phase #47

Description

@FedorPortnoi

Summary

rerunLatestSubmissions() in marathon-match-config.service.ts is called to re-score the latest submissions (e.g., after a configuration change). The function always dispatches a PROVISIONAL scoring event regardless of which phase (EXAMPLE, PROVISIONAL, or SYSTEM) is currently open. If called during the EXAMPLE or SYSTEM phase, submissions are re-scored against the wrong phase's seed range and test count, producing incorrect results.

Additionally, resolveOpenPhaseIds() falls back to accepting a closed currentPhase when no phase is actually open, allowing reruns to proceed even when the challenge has no active phase.

Affected File

src/api/marathon-match-config/marathon-match-config.service.ts

// rerunLatestSubmissions always hardcodes provisional:
await this.dispatchScoringEvent({
  // …
  testPhase: 'provisional',   // ← hardcoded, ignores current open phase
});

resolveOpenPhaseIds() fallback (lines 761–769) accepts a closed currentPhase:

// If no open phase found, falls back to currentPhase even if it is closed
return config.currentPhase ?? null;

Impact

  1. EXAMPLE phase rerun: Submissions re-scored with provisional seed count (50 seeds) instead of example seed count (10 seeds) — wrong results and 5× resource cost
  2. SYSTEM phase rerun: Submissions re-scored with provisional seed count instead of system seed count (5000 seeds) — wrong results and 1% of the intended coverage
  3. Post-close rerun: The resolveOpenPhaseIds fallback means reruns can be triggered against a closed challenge, potentially overwriting final SYSTEM scores with provisional-phase results

Expected

rerunLatestSubmissions() should determine the currently open phase dynamically and dispatch with the correct testPhase value. If no phase is open, the function should return an error rather than falling back to a default or closed phase.

const openPhase = await this.resolveOpenPhaseIds(config);
if (!openPhase) throw new Error('No open phase — cannot rerun');
await this.dispatchScoringEvent({ testPhase: openPhase,});

Severity: Medium

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions