Skip to content

Commit 3207a37

Browse files
committed
Merge branch 'main' of github.com:emagedoc/CodeClash
2 parents 68ded20 + aced4d4 commit 3207a37

18 files changed

Lines changed: 1320 additions & 98 deletions

File tree

.cursor/rules/batch_monitor.mdc

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
---
2+
description: Batch monitor as part of the viewer that shows aws batch jobs live
3+
alwaysApply: false
4+
---
5+
6+
You are building a batch monitor for AWS batch jobs that are running.
7+
This is integrated in the viewer, though you should keep the code in separate files.
8+
9+
You should put your python file in codeclash/viewer/app_aws.py
10+
and keep your other files in similar files from the rest of the viewer directory tree.
11+
12+
## Goal
13+
14+
You are keeping a list of all AWS batch jobs that are running or have been run in the last 24h.
15+
16+
## UI structure
17+
18+
This is essentially big table with the following columns:
19+
20+
- Time submitted (sortable)
21+
- Time running [HH:MM] (sortable)
22+
- Status (filterable by status)
23+
- Job name (sortable)
24+
- Job ID (sortable)
25+
- Links
26+
27+
I will now go into details about some of these columns.
28+
29+
### Status column
30+
31+
There are the following statuses:
32+
33+
- SUBMITTED (grey)
34+
- PENDING (grey)
35+
- RUNNABLE (grey)
36+
- STARTING (yellow)
37+
- RUNNING (faded green)
38+
- SUCCEEDED (green)
39+
- FAILED (red)
40+
41+
Use the colors that I just specified as background and format the statuses as "tags".
42+
43+
### Links
44+
45+
There is currently only one link
46+
47+
1. View job on AWS online. This is a link like so: https://039984708918-4ppzlrng.us-east-1.console.aws.amazon.com/batch/home?region=us-east-1#jobs/ec2/detail/ae71eab1-5da8-4167-92c9-9d02277947d1
48+
49+
2. Link on emagedoc, this is a link like so: https://emagedoc.xyz/?folder=batch%2FPvpTournament.HuskyBench.r15.s100.p2.claude-sonnet-4-20250514.o3.251017231148 where you can get the name of the folder based on what is described below in combining data
50+
51+
3. Link on s3: This is a link like so: https://039984708918-4ppzlrng.us-east-1.console.aws.amazon.com/s3/buckets/codeclash?region=us-east-1&bucketType=general&prefix=logs%2Fbatch%2FPvpTournament.HuskyBench.r15.s100.p2.claude-sonnet-4-20250514.claude-sonnet-4-5-20250929.251017231146%2F&showversions=false where again you get the name of the folder based on the combining data section
52+
53+
## Getting the data
54+
55+
You should get the data from AWS Batch.
56+
57+
You need the following parameters
58+
59+
```
60+
job_definition_name = "codeclash-default-job"
61+
job_queue = "codeclash-queue"
62+
region = "us-east-1"
63+
```
64+
65+
You should use the `aws/list_jobs.py` script to get the data.
66+
You should update the data every 10 seconds.
67+
68+
### Combining data
69+
70+
OK now something a bit more complicated. We want to associate the aws jobs with the games from the viewer.
71+
72+
FOr this we first need to map job ids to essentially the output folders/metadata files.
73+
74+
Do this as follows: do Path("logs/").rglob("metadata.json") to find all metadata file.
75+
For every metadata file, you can find the ["aws"]["AWS_BATCH_JOB_ID"] parameter to get the job id.
76+
77+
Now match together path to metadata with the job id in a dictionary.
78+
79+
## Adding this to the viewer
80+
81+
The url of this should be `/batch`.
82+
There should be a link on the picker page.

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,8 @@ logs/**
77
*.pdf
88

99
# Kilian's specific ignores
10-
batch_submit.sh
10+
batch_submit*.json
11+
batch_submit*.txt
1112
round_scores.json
1213
*.parquet
1314

aws/__init__.py

Whitespace-only changes.

aws/docker_and_sync.sh

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,4 +129,8 @@ echo "════════════════════════
129129
echo "✅ Wrapper script docker_and_sync.sh pologue finished, executing user command: $*"
130130
echo "═══════════════════════════════════════════════════════════════════════════════"
131131
# Execute the command passed to container
132+
# Temporarily disable set -e so we can capture the exit code
133+
set +e
132134
"$@"
135+
exit_code=$?
136+
exit $exit_code

aws/run_job.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -61,8 +61,8 @@ def get_latest_job_definition_arn(self) -> str:
6161
logger.debug(f"Latest job definition:\n{json.dumps(latest_job_def, indent=2, default=str)}")
6262
return latest_job_def["jobDefinitionArn"]
6363

64-
def submit_job(self, command: list[str], job_name: str | None = None) -> str:
65-
"""Submit a job to AWS Batch."""
64+
def submit_job(self, command: list[str], job_name: str | None = None) -> tuple[str, str]:
65+
"""Submit a job to AWS Batch. Returns (job_id, job_name)."""
6666
if job_name is None:
6767
human_name = command[0]
6868
# If have config take that instead
@@ -97,7 +97,7 @@ def submit_job(self, command: list[str], job_name: str | None = None) -> str:
9797
logger.info(f"Full Command (passed to container entrypoint/bootstrap.sh): {shlex.join(full_command)}")
9898
logger.info(f"To retrieve logs later, run: python get_job_log.py {job_id}")
9999

100-
return job_id
100+
return job_id, job_name
101101

102102
def get_job_status(self, job_id: str) -> dict[str, Any]:
103103
"""Get the current status of a job."""
@@ -186,7 +186,7 @@ def main():
186186
)
187187

188188
# Submit the job
189-
job_id = launcher.submit_job(command, args.job_name)
189+
job_id, job_name = launcher.submit_job(command, args.job_name)
190190

191191
# Wait for completion if requested
192192
if args.wait or args.show_logs:

aws/setup/batch/environment.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
"type": "EC2",
1414
"allocationStrategy": "BEST_FIT_PROGRESSIVE",
1515
"minvCpus": 0,
16-
"maxvCpus": 32,
16+
"maxvCpus": 400,
1717
"desiredvCpus": 0,
1818
"instanceTypes": [
1919
"m5.xlarge"

aws/setup/batch/job_definition.json

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,10 +33,6 @@
3333
{
3434
"name": "PORTKEY_API_KEY",
3535
"valueFrom": "arn:aws:secretsmanager:us-east-1:039984708918:secret:kilian-codeclash-keys-CTX9UJ:PORTKEY_API_KEY::"
36-
},
37-
{
38-
"name": "ANTHROPIC_API_KEY",
39-
"valueFrom": "arn:aws:secretsmanager:us-east-1:039984708918:secret:kilian-codeclash-keys-CTX9UJ:ANTHROPIC_API_KEY::"
4036
}
4137
]
4238
},

aws/utils/__init__.py

Whitespace-only changes.

aws/utils/job_monitor.py

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
"""Job monitoring utilities for AWS Batch jobs."""
2+
3+
import time
4+
from typing import Any
5+
6+
import boto3
7+
from rich.console import Console
8+
from rich.live import Live
9+
from rich.table import Table
10+
11+
from codeclash.utils.log import get_logger
12+
13+
logger = get_logger("job_monitor", emoji="👀")
14+
15+
16+
class JobMonitor:
17+
def __init__(self, region: str = "us-east-1"):
18+
self.batch_client = boto3.client("batch", region_name=region)
19+
self.console = Console()
20+
21+
def get_job_status(self, job_id: str) -> dict[str, Any]:
22+
"""Get the current status of a job."""
23+
response = self.batch_client.describe_jobs(jobs=[job_id])
24+
if response["jobs"]:
25+
return response["jobs"][0]
26+
else:
27+
raise ValueError(f"Job {job_id} not found")
28+
29+
@staticmethod
30+
def format_duration(seconds: float) -> str:
31+
"""Format duration in HH:MM:SS"""
32+
hours = int(seconds // 3600)
33+
minutes = int((seconds % 3600) // 60)
34+
secs = int(seconds % 60)
35+
return f"{hours:02d}:{minutes:02d}:{secs:02d}"
36+
37+
@staticmethod
38+
def get_status_color(status: str) -> str:
39+
"""Get the color for a job status."""
40+
status_lower = status.lower()
41+
if status_lower in ["submitted", "pending", "runnable"]:
42+
return "cyan"
43+
elif status_lower in ["starting", "running"]:
44+
return "yellow"
45+
elif status_lower == "succeeded":
46+
return "green"
47+
elif status_lower == "failed":
48+
return "red"
49+
else:
50+
return ""
51+
52+
def create_status_table(self, jobs_info: dict[str, dict]) -> Table:
53+
"""Create a rich table with job status information"""
54+
table = Table(title="AWS Batch Jobs Status")
55+
table.add_column("Status")
56+
table.add_column("Runtime", justify="right", style="magenta")
57+
table.add_column("Job ID", style="yellow")
58+
table.add_column("Job Name", style="blue")
59+
table.add_column("Config", style="green")
60+
61+
for job_id, info in jobs_info.items():
62+
status = info["status"]
63+
color = self.get_status_color(status)
64+
status_colored = f"[{color}]{status}[/{color}]" if color else status
65+
66+
table.add_row(
67+
status_colored,
68+
info["runtime"],
69+
job_id[:13] + "...",
70+
info["job_name"],
71+
info["config"],
72+
)
73+
74+
return table
75+
76+
def monitor(self, job_info_dict: dict[str, dict[str, str]]) -> None:
77+
"""Monitor submitted jobs and display status in real-time.
78+
79+
Args:
80+
job_info_dict: Dict mapping job_id -> {job_name, config}
81+
"""
82+
jobs_info: dict[str, dict] = {}
83+
start_times: dict[str, float] = {job_id: time.time() for job_id in job_info_dict}
84+
completion_times: dict[str, float] = {}
85+
86+
self.console.print("\n[bold green]Note:[/] You can ^C this script without your jobs dying\n")
87+
88+
try:
89+
with Live(self.create_status_table({}), refresh_per_second=1, console=self.console) as live:
90+
while True:
91+
all_done = True
92+
for job_id, info in job_info_dict.items():
93+
try:
94+
job_status = self.get_job_status(job_id)
95+
status = job_status["status"]
96+
97+
# Freeze runtime when job reaches terminal state
98+
if status in ["SUCCEEDED", "FAILED"] and job_id not in completion_times:
99+
completion_times[job_id] = time.time()
100+
101+
if job_id in completion_times:
102+
elapsed = completion_times[job_id] - start_times[job_id]
103+
else:
104+
elapsed = time.time() - start_times[job_id]
105+
106+
jobs_info[job_id] = {
107+
"status": status,
108+
"runtime": self.format_duration(elapsed),
109+
"job_name": info["job_name"],
110+
"config": info["config"],
111+
}
112+
113+
if status not in ["SUCCEEDED", "FAILED"]:
114+
all_done = False
115+
except Exception as e:
116+
logger.error(f"Error getting status for {job_id}: {e}", exc_info=True)
117+
if job_id not in completion_times:
118+
completion_times[job_id] = time.time()
119+
jobs_info[job_id] = {
120+
"status": "ERROR",
121+
"runtime": self.format_duration(completion_times[job_id] - start_times[job_id]),
122+
"job_name": info["job_name"],
123+
"config": info["config"],
124+
}
125+
126+
live.update(self.create_status_table(jobs_info))
127+
128+
if all_done:
129+
break
130+
131+
time.sleep(1)
132+
133+
self.console.print("\n[bold green]All jobs completed![/]")
134+
except KeyboardInterrupt:
135+
self.console.print("\n[bold yellow]Monitoring stopped. Jobs continue running on AWS.[/]")

batch_monitor.py

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
#!/usr/bin/env python3
2+
3+
import argparse
4+
import json
5+
from pathlib import Path
6+
7+
from aws.utils.job_monitor import JobMonitor
8+
from codeclash.utils.log import get_logger
9+
10+
logger = get_logger("batch_monitor", emoji="👁️")
11+
12+
13+
def load_job_files(job_files: list[Path]) -> dict[str, dict[str, str]]:
14+
"""Load job information from multiple JSON files.
15+
16+
Args:
17+
job_files: List of paths to batch_submit_{timestamp}.json files
18+
19+
Returns:
20+
Combined dict mapping job_id -> {job_name, config}
21+
"""
22+
all_jobs: dict[str, dict[str, str]] = {}
23+
24+
for job_file in job_files:
25+
logger.info(f"Loading jobs from {job_file}")
26+
jobs = json.loads(job_file.read_text())
27+
28+
# Handle both old format (job_id -> config) and new format (job_id -> {job_name, config})
29+
for job_id, value in jobs.items():
30+
if isinstance(value, dict):
31+
all_jobs[job_id] = value
32+
else:
33+
# Old format, convert to new format
34+
all_jobs[job_id] = {"job_name": "N/A", "config": value}
35+
36+
logger.info(f"Loaded {len(all_jobs)} jobs total")
37+
return all_jobs
38+
39+
40+
def main() -> None:
41+
parser = argparse.ArgumentParser(description="Monitor AWS Batch jobs from multiple batch_submit JSON files")
42+
parser.add_argument(
43+
"job_files",
44+
type=Path,
45+
nargs="+",
46+
help="One or more batch_submit_{timestamp}.json files to monitor",
47+
)
48+
parser.add_argument(
49+
"--region",
50+
default="us-east-1",
51+
help="AWS region (default: us-east-1)",
52+
)
53+
args = parser.parse_args()
54+
55+
all_jobs = load_job_files(args.job_files)
56+
57+
monitor = JobMonitor(region=args.region)
58+
monitor.monitor(all_jobs)
59+
60+
61+
if __name__ == "__main__":
62+
main()

0 commit comments

Comments
 (0)