Strengthen expert checkpoint smoke assertions

Author: lucasbellesi

languages/csharp/assessments/04-expert/README.md

@@ -23,18 +23,21 @@ Write a program that:
 dotnet run --project assessment-04-expert.csproj
 ```
 
-## Expected Output (shape)
+## Expected Output
 
 ```text
-Worker 0 partial sum: ...
-Worker 1 partial sum: ...
-...
+Worker 0 partial sum: 48
+Worker 1 partial sum: 97
+Worker 2 partial sum: 60
+
 Final summary:
-Total: ...
-Minimum: ...
-Maximum: ...
+Total: 205
+Minimum: 2
+Maximum: 45
 ```
 
+The worker lines may appear in a different order, but the three partial sums and the final totals should match these values.
+
 ## What To Check
 
 - Shared updates happen only inside the `lock`.

languages/csharp/projects/04-expert/README.md

@@ -17,15 +17,17 @@ dotnet run --project expert-project.csproj
 - Measure pipeline duration with `Stopwatch`.
 - Print per-step execution summary.
 
-## Sample Output
+## Expected Output
 
 ```text
 Running 3 jobs through 2 steps...
 Step load processed 3 jobs
 Step transform processed 3 jobs
-Elapsed (microseconds): 42
+Elapsed (microseconds): <positive integer>
 ```
 
+The first three lines should stay exact. Only the measured microsecond value should vary.
+
 ## Extension Ideas
 
 - Add optional logging levels.

languages/go/assessments/04-expert/README.md

@@ -23,18 +23,21 @@ Write a program that:
 go run main.go
 ```
 
-## Expected Output (shape)
+## Expected Output
 
 ```text
-Worker 0 partial sum: ...
-Worker 1 partial sum: ...
-...
+Worker 0 partial sum: 48
+Worker 1 partial sum: 97
+Worker 2 partial sum: 60
+
 Final summary:
-Total: ...
-Minimum: ...
-Maximum: ...
+Total: 205
+Minimum: 2
+Maximum: 45
 ```
 
+The worker lines may appear in a different order, but the three partial sums and the final totals should match these values.
+
 ## What To Check
 
 - Shared updates happen only while the mutex is held.

languages/go/projects/04-expert/README.md

@@ -17,15 +17,17 @@ go run main.go
 - Measure pipeline duration with `time`.
 - Print per-step execution summary.
 
-## Sample Output
+## Expected Output
 
 ```text
 Running 3 jobs through 2 steps...
 Step load processed 3 jobs
 Step transform processed 3 jobs
-Elapsed (microseconds): 42
+Elapsed (microseconds): <positive integer>
 ```
 
+The first three lines should stay exact. Only the measured microsecond value should vary.
+
 ## Extension Ideas
 
 - Add optional logging levels.

languages/python/assessments/04-expert/README.md

@@ -23,18 +23,21 @@ Write a program that:
 python main.py
 ```
 
-## Expected Output (shape)
+## Expected Output
 
 ```text
-Worker 0 partial sum: ...
-Worker 1 partial sum: ...
-...
+Worker 0 partial sum: 48
+Worker 1 partial sum: 97
+Worker 2 partial sum: 60
+
 Final summary:
-Total: ...
-Minimum: ...
-Maximum: ...
+Total: 205
+Minimum: 2
+Maximum: 45
 ```
 
+The worker lines may appear in a different order, but the three partial sums and the final totals should match these values.
+
 ## What To Check
 
 - Shared updates happen only while the lock is held.

languages/python/projects/04-expert/README.md

@@ -17,15 +17,17 @@ python main.py
 - Measure pipeline duration with `time.perf_counter`.
 - Print per-step execution summary.
 
-## Sample Output
+## Expected Output
 
 ```text
 Running 3 jobs through 2 steps...
 Step load processed 3 jobs
 Step transform processed 3 jobs
-Elapsed (microseconds): 42
+Elapsed (microseconds): <positive integer>
 ```
 
+The first three lines should stay exact. Only the measured microsecond value should vary.
+
 ## Extension Ideas
 
 - Add optional logging levels.

scripts/automation_core/ops.py

@@ -228,16 +228,23 @@ def run_command(
     cwd: Path | None = None,
     input_text: str | None = None,
     quiet_stdout: bool = False,
+    capture_stdout: bool = False,
     action: str | None = None,
     timeout_seconds: float | None = None,
 ) -> subprocess.CompletedProcess[str]:
     try:
+        stdout_target: int | None = None
+        if capture_stdout:
+            stdout_target = subprocess.PIPE
+        elif quiet_stdout:
+            stdout_target = subprocess.DEVNULL
+
         completed = subprocess.run(
             command,
             cwd=str(cwd) if cwd else None,
             input=input_text,
             text=True,
-            stdout=subprocess.DEVNULL if quiet_stdout else None,
+            stdout=stdout_target,
             stderr=None,
             check=False,
             timeout=timeout_seconds,
@@ -818,15 +825,34 @@ def smoke_runtime_job(
         elif "input_file" in job:
             input_text = str((working_dir / job["input_file"]).resolve()) + "\n"
 
-        run_command(
+        capture_stdout = bool(
+            job.get("required_stdout_contains") or job.get("required_stdout_patterns")
+        )
+
+        completed = run_command(
             command_builder(job, working_dir),
             cwd=working_dir,
             input_text=input_text,
-            quiet_stdout=True,
+            quiet_stdout=not capture_stdout,
+            capture_stdout=capture_stdout,
             action=label,
             timeout_seconds=timeout_seconds,
         )
 
+        if capture_stdout:
+            output = completed.stdout or ""
+            for expected in job.get("required_stdout_contains", []):
+                if expected not in output:
+                    raise AutomationError(
+                        f"{label} did not print expected text: {expected}\nActual output:\n{output}"
+                    )
+            for pattern in job.get("required_stdout_patterns", []):
+                if not re.search(pattern, output, re.MULTILINE):
+                    raise AutomationError(
+                        f"{label} did not match expected pattern: "
+                        f"{pattern}\nActual output:\n{output}"
+                    )
+
         for output_name in job.get("required_outputs", []):
             if not (working_dir / output_name).exists():
                 raise AutomationError(f"{label} did not create {output_name}")

scripts/automation_manifest.json

@@ -174,12 +174,31 @@
         },
         {
           "program": "languages/python/assessments/04-expert/main.py"
+          ,
+          "required_stdout_contains": [
+            "Worker 0 partial sum: 48",
+            "Worker 1 partial sum: 97",
+            "Worker 2 partial sum: 60",
+            "Final summary:",
+            "Total: 205",
+            "Minimum: 2",
+            "Maximum: 45"
+          ]
         },
         {
           "program": "languages/python/projects/03-advanced/main.py"
         },
         {
           "program": "languages/python/projects/04-expert/main.py"
+          ,
+          "required_stdout_contains": [
+            "Running 3 jobs through 2 steps...",
+            "Step load processed 3 jobs",
+            "Step transform processed 3 jobs"
+          ],
+          "required_stdout_patterns": [
+            "Elapsed \\(microseconds\\): \\d+"
+          ]
         },
         {
           "working_dir": "languages/python/projects/02-core",
@@ -237,12 +256,31 @@
         },
         {
           "program": "languages/go/assessments/04-expert/main.go"
+          ,
+          "required_stdout_contains": [
+            "Worker 0 partial sum: 48",
+            "Worker 1 partial sum: 97",
+            "Worker 2 partial sum: 60",
+            "Final summary:",
+            "Total: 205",
+            "Minimum: 2",
+            "Maximum: 45"
+          ]
         },
         {
           "program": "languages/go/projects/03-advanced/main.go"
         },
         {
           "program": "languages/go/projects/04-expert/main.go"
+          ,
+          "required_stdout_contains": [
+            "Running 3 jobs through 2 steps...",
+            "Step load processed 3 jobs",
+            "Step transform processed 3 jobs"
+          ],
+          "required_stdout_patterns": [
+            "Elapsed \\(microseconds\\): \\d+"
+          ]
         },
         {
           "working_dir": "languages/go/projects/02-core",
@@ -300,12 +338,31 @@
         },
         {
           "project": "languages/csharp/assessments/04-expert/assessment-04-expert.csproj"
+          ,
+          "required_stdout_contains": [
+            "Worker 0 partial sum: 48",
+            "Worker 1 partial sum: 97",
+            "Worker 2 partial sum: 60",
+            "Final summary:",
+            "Total: 205",
+            "Minimum: 2",
+            "Maximum: 45"
+          ]
         },
         {
           "project": "languages/csharp/projects/03-advanced/advanced-project.csproj"
         },
         {
           "project": "languages/csharp/projects/04-expert/expert-project.csproj"
+          ,
+          "required_stdout_contains": [
+            "Running 3 jobs through 2 steps...",
+            "Step load processed 3 jobs",
+            "Step transform processed 3 jobs"
+          ],
+          "required_stdout_patterns": [
+            "Elapsed \\(microseconds\\): \\d+"
+          ]
         },
         {
           "working_dir": "languages/csharp/projects/02-core",