Skip to content

Commit f4438e3

Browse files
kentwelcomeclaude
andauthored
fix(dab): restart dab-postgres on-failure so a mid-run crash doesn't lose the DB (#12)
The dab-postgres service had a pg_isready healthcheck and a service_healthy startup gate but no restart policy (unlike dab-mongo). When postgres:17 dies mid-run (crash/OOM) it is never restarted, the container leaves dab-net, and `dab-postgres` stops resolving for the rest of the trial ('could not translate host name dab-postgres'). For hybrid pg+duckdb datasets like PANCANCER_ATLAS the clinical data then becomes unreachable and the agent can only abstain — observed on PANCANCER_ATLAS q2/q3 (both 'UNABLE TO DETERMINE', reward 0), which alone account for ~0.056 of stratified pass@1. Add 'restart: on-failure' to dab-postgres, mirroring the dab-mongo fix, so the populated data dir comes straight back up and main's healthcheck recovers. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent cb298b0 commit f4438e3

2 files changed

Lines changed: 20 additions & 0 deletions

File tree

packages/razorback-plugin-dab/src/razorback_plugin_dab/generate/compose.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,12 @@ def generate_compose(
116116
"POSTGRES_PASSWORD": POSTGRES_PASSWORD,
117117
"POSTGRES_DB": pg_dbs[0],
118118
},
119+
# postgres:17 can die mid-run (crash/OOM). Without a restart policy the
120+
# container leaves dab-net and `dab-postgres` stops resolving for the rest
121+
# of the trial — clinical data (e.g. PANCANCER_ATLAS) becomes unreachable
122+
# and the agent can only abstain. Restart on failure so the populated data
123+
# dir comes straight back up and the healthcheck recovers (mirrors dab-mongo).
124+
"restart": "on-failure",
119125
"healthcheck": {
120126
"test": ["CMD-SHELL", f"pg_isready -U {POSTGRES_USER} -d {pg_dbs[0]}"],
121127
"interval": "5s",

packages/razorback-plugin-dab/tests/unit/test_compose_postgres.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,20 @@ def test_postgres_service_emitted(tmp_path: Path):
4040
assert "dab-net" in pg["networks"]
4141

4242

43+
def test_postgres_has_restart_policy(tmp_path: Path):
44+
# postgres:17 can die mid-run (crash/OOM). Without restart-on-failure the
45+
# container leaves dab-net and `dab-postgres` stops resolving for the rest of
46+
# the trial, so clinical data becomes unreachable and the agent can only
47+
# abstain (observed on PANCANCER_ATLAS q2/q3). Mirror the dab-mongo fix.
48+
compose_text = generate_compose(
49+
db_config=_BOOKREVIEW_LIKE,
50+
dataset_name="bookreview",
51+
data_root=tmp_path,
52+
)
53+
pg = yaml.safe_load(compose_text)["services"]["dab-postgres"]
54+
assert pg["restart"] == "on-failure"
55+
56+
4357
def test_main_service_depends_on_postgres(tmp_path: Path):
4458
compose_text = generate_compose(
4559
db_config=_BOOKREVIEW_LIKE,

0 commit comments

Comments
 (0)