Skip to content

Commit ff2c93a

Browse files
LEANDERANTONYclaude
andcommitted
fix(resume-builder): two regressions caught by transcript replay
QA replay of the user's original 10-turn session against the post-1A+1B agent exposed two real failure modes: 1. **max_iterations=5 too tight for serial tool calls.** When the model chose to fetch 6 GitHub READMEs SEQUENTIALLY (one fetch_github_readme per loop iteration) instead of in parallel, the agentic loop exhausted at iteration 5, raised AgentExecutionError, and the regex step-machine fallback dumped the URL list into experience_notes — producing none of the captured projects/skills the user expected. Bumped cap to 12 (10 URLs of headroom + 2 for the final answer). Above ~12 is a runaway-loop signal; below ~10 leaves no room for the ~6-URL-in-one-turn case the user actually exercised. 2. **projects_notes captured as Python list repr.** The LLM occasionally emits prose fields as LISTS of per-item descriptions after a multi-tool round-trip ([{"name": "Proj A", ...}, ...] or ["Proj A — …", "Proj B — …"]). _apply_llm_draft_updates was wrapping in str(), which captured the Python repr "['…', '…']" into projects_notes. The structuring LLM then choked on the malformed shape and the regex fallback rendered the raw list literal into the markdown. Added _coerce_prose_field(value) that joins list/tuple inputs with \n\n so the structuring pass sees clean newline-separated entries instead. Applied to all three prose fields (experience_notes / education_notes / projects_notes) in both _apply_draft_updates and _apply_llm_draft_updates for symmetry. Verification: 61 tests across the affected suites green. Slice-1B follow-up only — the actual remaining quality bugs visible in the rendered PDF (projects-not-bulleted, Link field grabs first tech word, first-person voice leak in education/publications) are structuring-prompt issues for the next slice. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 78184fe commit ff2c93a

1 file changed

Lines changed: 61 additions & 21 deletions

File tree

backend/services/resume_builder_service.py

Lines changed: 61 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -406,13 +406,13 @@ def _apply_draft_updates(session: ResumeBuilderSession, updates: dict):
406406
updates.get("professional_summary", "") or ""
407407
).strip()
408408
if "experience_notes" in updates:
409-
session.draft.experience_notes = str(
410-
updates.get("experience_notes", "") or ""
411-
).strip()
409+
session.draft.experience_notes = _coerce_prose_field(
410+
updates.get("experience_notes")
411+
)
412412
if "education_notes" in updates:
413-
session.draft.education_notes = str(
414-
updates.get("education_notes", "") or ""
415-
).strip()
413+
session.draft.education_notes = _coerce_prose_field(
414+
updates.get("education_notes")
415+
)
416416
if "skills" in updates:
417417
skills = updates.get("skills", [])
418418
if not isinstance(skills, list):
@@ -428,7 +428,9 @@ def _apply_draft_updates(session: ResumeBuilderSession, updates: dict):
428428
[str(item).strip() for item in certifications if str(item).strip()]
429429
)
430430
if "projects_notes" in updates:
431-
session.draft.projects_notes = str(updates.get("projects_notes", "") or "").strip()
431+
session.draft.projects_notes = _coerce_prose_field(
432+
updates.get("projects_notes")
433+
)
432434
if "publications" in updates:
433435
publications = updates.get("publications", [])
434436
if not isinstance(publications, list):
@@ -1839,29 +1841,59 @@ def _apply_llm_draft_updates(session: ResumeBuilderSession, updates: dict):
18391841
updates.get("professional_summary") or ""
18401842
).strip()
18411843
if "experience_notes" in updates:
1842-
session.draft.experience_notes = str(
1843-
updates.get("experience_notes") or ""
1844-
).strip()
1844+
session.draft.experience_notes = _coerce_prose_field(
1845+
updates.get("experience_notes")
1846+
)
18451847
if "education_notes" in updates:
1846-
session.draft.education_notes = str(
1847-
updates.get("education_notes") or ""
1848-
).strip()
1848+
session.draft.education_notes = _coerce_prose_field(
1849+
updates.get("education_notes")
1850+
)
18491851
if "skills" in updates:
18501852
session.draft.skills = dedupe_strings(_coerce_string_list(updates.get("skills")))
18511853
if "certifications" in updates:
18521854
session.draft.certifications = dedupe_strings(
18531855
_coerce_string_list(updates.get("certifications"))
18541856
)
18551857
if "projects_notes" in updates:
1856-
session.draft.projects_notes = str(
1857-
updates.get("projects_notes") or ""
1858-
).strip()
1858+
session.draft.projects_notes = _coerce_prose_field(
1859+
updates.get("projects_notes")
1860+
)
18591861
if "publications" in updates:
18601862
session.draft.publications = dedupe_strings(
18611863
_coerce_string_list(updates.get("publications"))
18621864
)
18631865

18641866

1867+
def _coerce_prose_field(value) -> str:
1868+
"""Coerce an LLM-emitted prose field to a clean string.
1869+
1870+
The LLM contract specifies that ``experience_notes`` /
1871+
``education_notes`` / ``projects_notes`` are STRINGS — verbatim
1872+
user prose, one block per field. In practice the model
1873+
occasionally emits them as LISTS of entries, especially right
1874+
after a multi-URL tool round-trip where each fetched README
1875+
becomes a logical project. Naively wrapping that in ``str(...)``
1876+
captured the Python ``repr()`` of the list, including the square
1877+
brackets and quoted entries — and the downstream structuring
1878+
prompt choked on the malformed shape, then the regex fallback
1879+
rendered the raw list literal into the markdown.
1880+
1881+
This helper joins list/tuple inputs with ``\n\n`` so the
1882+
structuring pass sees newline-separated entries instead of a
1883+
stringified list. Single-string inputs pass through unchanged.
1884+
"""
1885+
if value is None:
1886+
return ""
1887+
if isinstance(value, (list, tuple)):
1888+
cleaned_parts = [
1889+
str(item).strip()
1890+
for item in value
1891+
if str(item or "").strip()
1892+
]
1893+
return "\n\n".join(cleaned_parts).strip()
1894+
return str(value).strip()
1895+
1896+
18651897
def _summarize_tool_event(event: dict) -> dict:
18661898
"""Compress one tool-loop trace entry for `conversation_history`.
18671899
@@ -1943,11 +1975,19 @@ def _run_llm_turn(
19431975
tools=RESUME_BUILDER_TOOL_SPECS,
19441976
tool_executor=execute_resume_builder_tool,
19451977
expected_keys=prompt["expected_keys"],
1946-
# Five iterations is enough for the realistic case (one
1947-
# URL → one fetch → one JSON answer) with headroom for the
1948-
# model to retry on transient tool errors. A higher cap
1949-
# invites runaway loops on malformed responses.
1950-
max_iterations=5,
1978+
# Twelve iterations: covers the realistic worst case of a
1979+
# user pasting ~10 GitHub URLs in one turn and the model
1980+
# choosing to fetch them SERIALLY instead of in parallel
1981+
# (one fetch_github_readme per iteration + the final
1982+
# answer = N+1 iterations). The original cap of 5 hit a
1983+
# regression on QA replay when the model serialized 6
1984+
# fetches — the loop exhausted at iteration 5, the
1985+
# service caught AgentExecutionError, and the regex
1986+
# fallback dumped the URL list into experience_notes.
1987+
# Above ~12 is a runaway-loop signal (prompt drift or
1988+
# tool bug); below ~10 leaves no headroom for the
1989+
# ~6-URL-in-one-turn case the user actually exercised.
1990+
max_iterations=12,
19511991
temperature=None,
19521992
max_completion_tokens=get_openai_max_completion_tokens_for_task("resume_builder"),
19531993
task_name="resume_builder",

0 commit comments

Comments
 (0)