TaskMaster Integration: Story-Based Task Management

[elasticdotventures]

Summary

Integrates TaskMaster-AI for story-based task management with proper separation of concerns.

Key Innovation: Stories > Tasks for AI performance

• Each "task" is written as a rich user story with narrative context
• Produces better autonomous coding results than bare instructions
• Aligns with insight: "taskmaster-ai would perform better as storymaster-ai"

Architecture

Separation of Concerns:

• TaskMaster-AI: Owns all task CRUD operations (via MCP or CLI)
• Ralph: Consumes task data, NEVER accesses tasks.json directly

Changes

1. TaskMaster Integration (ralph/taskmaster_adapter.py)

• Created TaskMasterClient Protocol for abstraction
• Task dataclass with full TaskMaster schema support
• CLITaskMasterClient - shells out to taskmaster CLI commands
• MCPTaskMasterClient stub for future MCP integration
• Hybrid mode: tries MCP, falls back to CLI

2. Runner Integration (ralph/runner.py)

• Displays task progress: "X done, Y in progress, Z pending"
• TaskMasterClient integration for status tracking
• Maintains <promise>COMPLETE</promise> detection

3. Agent Instructions (CLAUDE.md)

• Emphasizes story format over bare tasks
• Added "Understanding Stories vs Tasks" section
• Updated all references from prd.json to tasks.json
• Clarifies: Stories provide cognitive context for better AI results

4. Preflight Safety (ralphython.py)

• Verifies .taskmaster/ directory exists in git root
• Verifies .taskmaster is in .gitignore (prevents accidental commits)
• All MCP tools use taskmaster CLI instead of direct file access
• Removed all obsolete prd.json handling

5. Story-Based Format (tasks.json)

• 8 stories defined with full narrative context
• Each story includes: who, what, why, how
• Acceptance criteria for validation
• Example story demonstrates proper format

6. Configuration (ralph/config.py)

• Added use_mcp: bool field
• Added taskmaster_url: Optional[str] field
• Added opencode_model and opencode_extra_args fields
• OpenCode executor support prepared

7. Schema Definition (schemas/taskmaster-schema.json)

• Complete JSON Schema for TaskMaster format
• Validates task structure, status values, dependencies

Files Changed

• ✅ ralph/taskmaster_adapter.py (NEW)
• ✅ ralph/runner.py
• ✅ ralph/config.py
• ✅ CLAUDE.md
• ✅ ralphython.py
• ✅ tasks.json (NEW)
• ✅ schemas/taskmaster-schema.json (NEW)
• ✅ .gitignore
• ❌ prd.json (DELETED)
• ✅ ralph.sh (simplified)

Testing

Requires taskmaster-ai CLI to be installed:

# Install taskmaster-ai
pip install taskmaster-ai  # (when available)

# Run Ralph with TaskMaster
uv run ralph --tool amp 10

Breaking Changes

• ⚠️ Removes prd.json - replaced by tasks.json (TaskMaster format)
• ⚠️ Requires .taskmaster/ directory in git repo root
• ⚠️ Requires taskmaster CLI or MCP server

Migration Path

For existing prd.json users:

1. Install taskmaster-ai
2. Initialize: taskmaster init
3. Convert stories to TaskMaster format with full narrative context
4. Ensure .taskmaster/ is in .gitignore

Why This Matters

Cognitive Context = Better AI:

• Bare tasks: "Add OpenCode executor" ❌
• Rich stories: "As a developer... I need... so that... implementation approach..." ✅

Stories provide the narrative context that autonomous AI agents need to make better implementation decisions.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Integrates TaskMaster-AI into Ralph’s Python runner to surface task/story progress, and introduces a TaskMaster-style tasks.json + schema alongside packaging, CI, tests, and docs updates.

Changes:

• Added TaskMaster adapter + runner integration to show “done / in-progress / pending” summaries.
• Added Python project scaffolding (pyproject/uv lock), CI workflow, and an expanded unit/integration test suite with tool stubs.
• Added TaskMaster-format tasks.json and JSON Schema; updated docs/instructions to reference the new story-based format.

Reviewed changes

Copilot reviewed 33 out of 38 changed files in this pull request and generated 16 comments.

Show a summary per file

File	Description
uv.lock	Adds locked dev/runtime dependencies for the Python rewrite.
pyproject.toml	Defines package metadata, CLI entrypoint, and tool configuration (ruff/mypy/pytest).
.github/workflows/python-ci.yml	Adds CI job to run ruff, mypy, and pytest with coverage.
.gitignore	Adds TaskMaster/Python ignore patterns (but currently also lists tasks.json).
tasks.json	Adds story-based TaskMaster task list content.
schemas/taskmaster-schema.json	Adds JSON Schema for TaskMaster-format tasks.
ralph/taskmaster_adapter.py	Introduces CLI/MCP abstraction layer for task operations.
ralph/runner.py	Integrates TaskMaster client + progress summaries into the loop.
ralph/config.py	Adds TaskMaster + OpenCode-related config fields.
ralph/executors.py	Implements tool executors and subprocess output capture.
ralph/file_manager.py	File utilities for PRD/progress (still PRD-based).
ralph/logging_utils.py	Adds emoji-prefixed logging helpers.
ralph/ralph_cli.py	Adds Python CLI entry point for running the loop.
ralph/archiver.py	Adds branch-change detection + archiving (still PRD-based).
ralph/init.py	Adds package version export.
ralph/README.md	Adds Python Ralph documentation (currently still PRD-centric).
ralphython.py	Adds TaskMaster preflight + MCP tools; removes PRD ingestion in this script.
ralph.sh	Simplifies wrapper to run uv run ralph.
README.md	Updates top-level docs for Python rewrite + CI badge.
CLAUDE.md	Updates agent instructions to use tasks.json story format.
AGENTS.md	Adds guidance for running tools via uv run python -m ....
progress.txt	Updates progress log content with additional iteration notes.
justfile	Adds convenience commands for running/checking/testing Ralph.
tests/conftest.py	Adds pytest fixture to reset logging between tests.
tests/test_runner.py	Adds unit tests for runner loop behavior.
tests/test_ralph_integration.py	Adds subprocess-based integration tests + stub tool wiring.
tests/test_ralph_cli.py	Adds CLI argument parsing tests.
tests/test_logging_utils.py	Adds tests for logging helpers.
tests/test_file_manager.py	Adds tests for progress/PRD file helpers.
tests/test_executors.py	Adds executor + subprocess wrapper tests.
tests/test_config.py	Adds config-from-env tests.
tests/test_archiver.py	Adds archiver behavior tests.
tests/tool_stubs/amp	Adds stub amp CLI for integration tests.
tests/tool_stubs/claude	Adds stub claude CLI for integration tests.
tests/tool_stubs/codex	Adds stub codex CLI for integration tests.
tests/pycache/conftest.cpython-311-pytest-9.0.2.pyc	Compiled artifact added (should not be committed).
tests/pycache/test_ralphython.cpython-311-pytest-9.0.2.pyc	Compiled artifact added (should not be committed).
ralph/pycache/archiver.cpython-311.pyc	Compiled artifact added (should not be committed).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

This subprocess piping approach can deadlock: the child process writes to the pipe FD (via fileno()), but the parent only reads from the pipe after subprocess.run returns. If the tool outputs more than the OS pipe buffer, the child will block and hang the run. Consider switching to subprocess.Popen with stdout=PIPE and streaming reads (teeing to stderr) or using communicate() with capture_output (if real-time output isn't required).

Suggested change

	stdout_stream: IO[str] = cast(IO[str], tee_stream)
	try:
	completed = subprocess.run(
	command,
	input=input_text,
	stdout=stdout_stream,
	stderr=subprocess.STDOUT,
	text=True,
	cwd=cwd_value,
	env=env_value,
	check=False,
	)
	# Close the write end so we can read remaining data
	with contextlib.suppress(OSError):
	os.close(tee_stream._pipe_write)
	# Read any remaining data from the read end
	with contextlib.suppress(OSError):
	remaining = os.read(tee_stream._pipe_read, 1024 * 1024).decode("utf-8")
	if remaining:
	tee_stream._parts.append(remaining)
	sys.stderr.write(remaining)
	sys.stderr.flush()
	except OSError as exc:
	log_error(configure_logging(), f"Failed to execute {' '.join(command)}", exc)
	detail = f"Failed to execute {' '.join(command)}"
	return Failure(ExecutorError(detail=detail, command=tuple(command), output=str(exc)))
	finally:
	tee_stream.close()
	output = tee_stream.value
	if completed.returncode == 0:
	return Success(output)
	detail = f"Command {' '.join(command)} exited with {completed.returncode}"
	return Failure(
	ExecutorError(
	detail=detail,
	command=tuple(command),
	returncode=completed.returncode,
	try:
	process = subprocess.Popen(
	command,
	stdin=subprocess.PIPE if input_text is not None else None,
	stdout=subprocess.PIPE,
	stderr=subprocess.STDOUT,
	text=True,
	cwd=cwd_value,
	env=env_value,
	)
	# Send any provided input and close stdin so the child sees EOF.
	if input_text is not None and process.stdin is not None:
	try:
	process.stdin.write(input_text)
	process.stdin.close()
	except OSError:
	# Ignore errors writing to stdin (e.g., if the process exits early).
	with contextlib.suppress(OSError):
	process.stdin.close()
	# Stream output from the child, teeing to stderr and accumulating it.
	if process.stdout is not None:
	for chunk in process.stdout:
	if not chunk:
	break
	tee_stream.write(chunk)
	sys.stderr.write(chunk)
	sys.stderr.flush()
	returncode = process.wait()
	except OSError as exc:
	log_error(configure_logging(), f"Failed to execute {' '.join(command)}", exc)
	detail = f"Failed to execute {' '.join(command)}"
	return Failure(ExecutorError(detail=detail, command=tuple(command), output=str(exc)))
	finally:
	# Ensure the tee stream is closed even if an error occurs.
	tee_stream.close()
	output = tee_stream.value
	if returncode == 0:
	return Success(output)
	detail = f"Command {' '.join(command)} exited with {returncode}"
	return Failure(
	ExecutorError(
	detail=detail,
	command=tuple(command),
	returncode=returncode,

[Copilot]

create_client() documents that tasks_file is deprecated and unused, but run_ralph still passes WORKING_DIR / "tasks.json". This is misleading (suggests a file-based fallback exists when it doesn’t). Consider removing the argument here, or (better) removing tasks_file from the factory signature if it’s not needed.

Suggested change

tasks_file=WORKING_DIR / "tasks.json",

[Copilot]

The helper docstring says Ralph expects a PRD, but the current Python CLI path doesn’t read prd.json anymore (runner uses TaskMaster + prompt files). Consider updating this docstring to reflect the actual required workspace files.

[Copilot]

The integration-test workspace setup writes a prd.json, but the current CLI/integration path doesn’t appear to consume it. Consider removing this file creation (or switching to tasks.json if TaskMaster-based setup is required) so the test fixture reflects real prerequisites.

[Copilot]

This section claims Ralph reads prd.json and does branch detection/archival. In the current Python implementation, run_ralph() doesn’t use PRD/archiver; it uses TaskMaster for task status and only requires prompt files + progress file. Please update this doc to match the current runtime behavior (and/or wire archiver into the CLI if that’s still intended).

Suggested change

	1. **Initialization**: Ralph reads `prd.json` and `progress.txt` from the project root
	2. **Branch Detection**: Checks if git branch has changed since last run
	3. **Archival**: If branch changed, archives previous run to `archive/{date}-{branch-name}/`
	4. **Iteration Loop**: Runs the selected tool (amp/claude/codex) repeatedly
	5. **Completion Detection**: Monitors tool output for `<promise>COMPLETE</promise>` signal
	6. **Progress Tracking**: Updates `progress.txt` after each iteration
	1. **Initialization**: Ralph loads the prompt file(s) and a progress file from the project root
	2. **Task Management**: A TaskMaster instance tracks the current task state and iteration count
	3. **Iteration Loop**: Runs the selected tool (amp/claude/codex) repeatedly under TaskMaster control
	4. **Completion Detection**: Monitors tool output (e.g. for `<promise>COMPLETE</promise>`) to determine when the task is done
	5. **Progress Tracking**: Updates the progress file after each iteration

[Copilot]

Import of 'MagicMock' is not used.

[Copilot]

Import of 'textwrap' is not used.

[Copilot]

Import of 'Path' is not used.

[Copilot]

Import of 'MagicMock' is not used.

[Copilot]

Instance of context-manager class _TeeToStderr is closed in a finally block. Consider using 'with' statement.

Suggested change

	tee_stream = _TeeToStderr()
	stdout_stream: IO[str] = cast(IO[str], tee_stream)
	try:
	completed = subprocess.run(
	command,
	input=input_text,
	stdout=stdout_stream,
	stderr=subprocess.STDOUT,
	text=True,
	cwd=cwd_value,
	env=env_value,
	check=False,
	)
	# Close the write end so we can read remaining data
	with contextlib.suppress(OSError):
	os.close(tee_stream._pipe_write)
	# Read any remaining data from the read end
	with contextlib.suppress(OSError):
	remaining = os.read(tee_stream._pipe_read, 1024 * 1024).decode("utf-8")
	if remaining:
	tee_stream._parts.append(remaining)
	sys.stderr.write(remaining)
	sys.stderr.flush()
	except OSError as exc:
	log_error(configure_logging(), f"Failed to execute {' '.join(command)}", exc)
	detail = f"Failed to execute {' '.join(command)}"
	return Failure(ExecutorError(detail=detail, command=tuple(command), output=str(exc)))
	finally:
	tee_stream.close()
	output = tee_stream.value
	completed = None
	output = ""
	with _TeeToStderr() as tee_stream:
	stdout_stream: IO[str] = cast(IO[str], tee_stream)
	try:
	completed = subprocess.run(
	command,
	input=input_text,
	stdout=stdout_stream,
	stderr=subprocess.STDOUT,
	text=True,
	cwd=cwd_value,
	env=env_value,
	check=False,
	)
	# Close the write end so we can read remaining data
	with contextlib.suppress(OSError):
	os.close(tee_stream._pipe_write)
	# Read any remaining data from the read end
	with contextlib.suppress(OSError):
	remaining = os.read(tee_stream._pipe_read, 1024 * 1024).decode("utf-8")
	if remaining:
	tee_stream._parts.append(remaining)
	sys.stderr.write(remaining)
	sys.stderr.flush()
	except OSError as exc:
	log_error(configure_logging(), f"Failed to execute {' '.join(command)}", exc)
	detail = f"Failed to execute {' '.join(command)}"
	return Failure(ExecutorError(detail=detail, command=tuple(command), output=str(exc)))
	output = tee_stream.value
	assert completed is not None