Move to global ~/.mfbt config, integrate Ralph into TUI, add auto re-auth

Major changes:
- Migrate config from per-project .mfbt/ to global ~/.mfbt/ (schema v1→v2)
- Add shared OAuth flow (auth_flow.py) with auto token refresh and
  interactive re-auth on 401 via on_auth_required callback
- Integrate Ralph orchestration into main TUI via RalphPanel widget
  (r key), replacing standalone tui_app.py
- Add coding agent pre-flight checks (coding_agents.py) with
  PreflightModal for verifying agent installation and MCP config
- Extend TUI to 4-level navigation: Projects > Phases > Modules > Features
- Simplify config/auth functions to operate on ~/.mfbt/ without
  requiring project_root parameter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: shuveb

CLAUDE.md

@@ -7,7 +7,7 @@
 - **Language:** Python 3.10+ (uses modern features: pattern matching, improved type hints)
 - **Backend:** Integrates with mfbt REST API (`{BASE_URL}/api/v1/`)
 - **Auth:** OAuth 2.1 with PKCE (1hr access tokens, 30-day refresh tokens), plus API key management (`mfbtsk-{uuid}`, passed as `Authorization: Bearer {key}`)
-- **Config:** `.mfbt` directory stores auth tokens and project configuration
+- **Config:** Global `~/.mfbt/` directory stores auth tokens and configuration (no per-project config)
 
 ## API Specification
 
@@ -17,11 +17,12 @@ The file `openapi.json` in the project root contains the full OpenAPI spec for t
 
 ### Core Systems
 
-1. **Authentication & Configuration** — Browser-based OAuth flow, token refresh/session management, API key support, project selection post-auth
-2. **Interactive TUI Mode** — Launched when CLI runs without subcommands; K9S-style keyboard-driven navigation with real-time status updates
-3. **Subcommands** — `status`, `next`, `modules`, `features`, plus commands for brainstorming phases, implementations, and jobs. Consistent output formatting (table, JSON, etc.)
-4. **API Integration Layer** — REST client for all mfbt endpoints, paginated responses (`{items, total, page, page_size, total_pages}`), error handling (404 for unauthorized, 402 for token limits), job polling for async operations, WebSocket support for real-time job status
+1. **Authentication & Configuration** — Browser-based OAuth flow with PKCE, auto token refresh + interactive re-auth on 401, API key support. Shared auth flow in `auth_flow.py`. Config/auth in `~/.mfbt/` (schema v2, no per-project `project_id`).
+2. **Interactive TUI Mode** — Launched when CLI runs without subcommands; K9S-style keyboard-driven navigation: Projects > Phases > Modules > Features. Ralph orchestration integrated via `r` key.
+3. **Subcommands** — `auth`, `projects` (list, show, create, archive, delete), `ralph`, `tui`. Consistent output formatting (table, JSON, etc.)
+4. **API Integration Layer** — REST client with `on_auth_required` callback for auto re-auth on 401. Paginated responses (`{items, total, page, page_size, total_pages}`), error handling (402 for token limits), job polling, WebSocket support.
 5. **API Key Management** — CRUD via `/api/v1/users/me/api-keys`
+6. **Coding Agent Checks** — `coding_agents.py` provides pre-flight checks (agent installed, MCP configured) before Ralph orchestration.
 
 ### Key API Endpoints
 
@@ -97,8 +98,12 @@ This project uses the **mfbt MCP server**, which exposes a virtual filesystem (V
 ## Development
 
 - **Venv:** `.venv/bin/python`, `.venv/bin/pytest`
-- **Run tests:** `.venv/bin/pytest tests/unit/ralph/ -v`
+- **Run tests:** `.venv/bin/pytest tests/unit/ -v`
+- **Config functions no longer take `project_root`** — `load_config()`, `save_config()`, `load_auth()`, `save_auth()`, `init_mfbt_dir()`, `resolve_config()` all operate on `~/.mfbt/` via `get_mfbt_dir()`. `TokenManager.__init__` takes only `base_url`.
 - **API response formats:** List endpoints (`/features`, `/implementations`) return **paginated** `{"items": [...], "total": N, ...}` — always extract via `body["items"]`. Some endpoints (`/brainstorming-phases`) return plain lists. See `src/mfbt/tui/data_provider.py` for the canonical parsing pattern.
-- **Ralph subcommand:** `src/mfbt/commands/ralph/` — orchestrator, display (console), tui_display (Textual), agent runner, progress (API), prompt builder, types
-- **Ralph TUI:** Standalone Textual app (`tui_app.py`), separate from main mfbt TUI. Uses `RalphTUIDisplay` adapter (same 8-method interface as `RalphDisplay`) with `call_from_thread()` to marshal UI updates from the orchestrator's worker thread.
+- **TUI navigation:** Projects > Phases > Modules > Features (4 levels). `NavigationState` stack depth: 0=projects, 1=phases, 2=modules, 3=features. Phase list shows brainstorming phases + virtual "Orphan modules" entry.
+- **Ralph in TUI:** Integrated into main TUI (`r` key), not a standalone app. Uses `RalphPanel` widget in `#main-content` with `PreflightModal` for agent checks. Standalone `tui_app.py` deleted.
+- **Ralph subcommand:** `src/mfbt/commands/ralph/` — orchestrator, display (console), tui_display (Textual adapter), ralph_widgets (TUI widgets), agent runner, progress (API), prompt builder, types.
 - **Display duck typing:** `RalphOrchestrator.display` is typed as `Any` — both `RalphDisplay` and `RalphTUIDisplay` are structurally compatible (same 8 methods).
+- **Key new files:** `auth_flow.py` (shared OAuth), `coding_agents.py` (agent pre-flight checks), `tui/screens/phase_list.py`, `tui/screens/preflight_modal.py`, `tui/screens/ralph_panel.py`, `commands/ralph/ralph_widgets.py`.
+- **TUI shortcuts:** `r` = Ralph, `ctrl+r` = Refresh, `d` = Describe, `enter` = Open/Detail, `esc` = Back, `?` = Help, `q` = Quit.

src/mfbt/__main__.py

@@ -22,7 +22,7 @@ def build_parser() -> argparse.ArgumentParser:
 commands:
   (none)              Launch interactive TUI (default)
   auth                Authentication commands (login, status, refresh, logout)
-  projects            Manage projects (list, show, create, switch, archive, delete)
+  projects            Manage projects (list, show, create, archive, delete)
   ralph               Auto-invoke coding agents to implement pending features
   tui                 Launch interactive TUI
 
@@ -78,23 +78,24 @@ def main(args: list[str] | None = None) -> None:
 
     setup_logging(verbosity=parsed.verbose)
 
-    from mfbt.config import find_project_root, init_mfbt_dir, resolve_config
+    from mfbt.config import init_mfbt_dir, is_authenticated, resolve_config
 
     console = Console()
-    project_root = find_project_root()
-    if project_root is None:
-        console.print("[red]Error:[/red] No project root found.")
+
+    init_mfbt_dir()
+
+    if not is_authenticated():
+        console.print("[red]Error:[/red] Not authenticated.")
         console.print(
-            "Run from within a project directory, or use [bold]mfbt --help[/bold]."
+            "Run [bold]mfbt auth login[/bold] to authenticate first."
         )
         return
 
-    init_mfbt_dir(project_root)
-    config = resolve_config(project_root)
+    config = resolve_config()
 
     from mfbt.tui import launch_tui
 
-    launch_tui(project_root, config)
+    launch_tui(config)
 
 
 if __name__ == "__main__":

src/mfbt/api_client.py

@@ -12,7 +12,7 @@
 import random
 import threading
 import time
-from collections.abc import Iterator
+from collections.abc import Callable, Iterator
 from dataclasses import dataclass
 from typing import TYPE_CHECKING, Any
 
@@ -272,11 +272,13 @@ def __init__(
         timeout: float = DEFAULT_TIMEOUT,
         max_retries: int = DEFAULT_MAX_RETRIES,
         cache: ResponseCache | None = None,
+        on_auth_required: Callable[[], bool] | None = None,
     ) -> None:
         self.base_url = base_url.rstrip("/")
         self.token_manager = token_manager
         self.max_retries = max_retries
         self.cache = cache
+        self.on_auth_required = on_auth_required
         self._circuit_breaker = CircuitBreaker()
         self._client = httpx.Client(
             timeout=timeout,
@@ -418,6 +420,27 @@ def _request(
 
         return api_response
 
+    def _obtain_auth_headers(
+        self, headers: dict[str, str] | None
+    ) -> dict[str, str]:
+        """Get merged auth + caller headers, with auto-login fallback.
+
+        If getting the access token raises a TokenError (e.g. refresh
+        token expired), invoke the ``on_auth_required`` callback to let
+        the user re-authenticate interactively, then retry once.
+        """
+        from mfbt.token_manager import TokenError
+
+        try:
+            auth_headers = self._get_auth_headers()
+        except TokenError:
+            if self.on_auth_required is None or not self.on_auth_required():
+                raise
+            # Callback succeeded — retry getting headers
+            auth_headers = self._get_auth_headers()
+
+        return {**auth_headers, **(headers or {})}
+
     def _execute_with_retry(
         self,
         method: str,
@@ -430,12 +453,11 @@ def _execute_with_retry(
         """Execute an HTTP request with retry logic and circuit breaker."""
         self._circuit_breaker.check_state()
 
+        merged_headers = self._obtain_auth_headers(headers)
+
         last_error: Exception | None = None
 
         for attempt in range(self.max_retries):
-            auth_headers = self._get_auth_headers()
-            merged_headers = {**auth_headers, **(headers or {})}
-
             try:
                 response = self._client.request(
                     method,
@@ -461,6 +483,36 @@ def _execute_with_retry(
                     technical_details=str(exc),
                 ) from exc
 
+            # Handle 401 — try token refresh, then interactive auth
+            if response.status_code == 401:
+                logger.info("Received 401, attempting token refresh")
+                merged_headers = self._handle_auth_failure(headers)
+                if merged_headers is not None:
+                    # Retry the request once with fresh headers
+                    try:
+                        response = self._client.request(
+                            method,
+                            url,
+                            params=params,
+                            json=json_body,
+                            headers=merged_headers,
+                        )
+                    except Exception as exc:
+                        self._circuit_breaker.record_failure()
+                        raise NetworkError(
+                            technical_details=str(exc),
+                        ) from exc
+
+                    if response.status_code != 401:
+                        self._circuit_breaker.record_success()
+                        return response
+
+                # Still 401 after recovery attempts — fall through to
+                # normal status handling (will raise AuthenticationRequired
+                # in _raise_for_status)
+                self._circuit_breaker.record_success()
+                return response
+
             # Check for retryable HTTP status
             if _is_retryable_status(response.status_code):
                 last_error = APIError(
@@ -496,6 +548,35 @@ def _execute_with_retry(
             original=last_error,
         ) from last_error
 
+    def _handle_auth_failure(
+        self, original_headers: dict[str, str] | None
+    ) -> dict[str, str] | None:
+        """Attempt to recover from an authentication failure.
+
+        1. Try refreshing the token via ``token_manager.refresh_tokens()``.
+        2. If refresh fails and ``on_auth_required`` is set, invoke it
+           to trigger interactive re-authentication.
+
+        Returns merged headers on success, or None if recovery failed.
+        """
+        from mfbt.token_manager import TokenError
+
+        # First try: refresh the token
+        try:
+            self.token_manager.refresh_tokens()
+            return {**self._get_auth_headers(), **(original_headers or {})}
+        except TokenError:
+            logger.info("Token refresh failed, trying interactive auth")
+
+        # Second try: interactive re-auth
+        if self.on_auth_required is not None and self.on_auth_required():
+            try:
+                return {**self._get_auth_headers(), **(original_headers or {})}
+            except TokenError:
+                logger.warning("Auth headers still invalid after interactive login")
+
+        return None
+
     def _raise_for_status(self, response: httpx.Response) -> None:
         """Map HTTP error status codes to specific exceptions."""
         status = response.status_code

src/mfbt/auth.py

@@ -572,13 +572,8 @@ def exchange_code_for_tokens(
     )
 
 
-def store_tokens(project_root: Path, token_response: TokenResponse) -> None:
-    """Store OAuth tokens using the existing config system.
-
-    Args:
-        project_root: The project root containing .mfbt/.
-        token_response: The token response to store.
-    """
+def store_tokens(token_response: TokenResponse) -> None:
+    """Store OAuth tokens using the existing config system."""
     from mfbt.config import save_auth
 
     auth_data = {
@@ -588,5 +583,5 @@ def store_tokens(project_root: Path, token_response: TokenResponse) -> None:
         "expires_at": token_response.expires_at,
         "issued_at": token_response.issued_at,
     }
-    save_auth(project_root, auth_data)
-    logger.info("Tokens stored in .mfbt/auth.json")
+    save_auth(auth_data)
+    logger.info("Tokens stored in ~/.mfbt/auth.json")

src/mfbt/auth_flow.py

@@ -0,0 +1,111 @@
+"""Shared browser-based OAuth login flow.
+
+Extracted from tui/__init__.py so both TUI and CLI commands can
+trigger re-authentication when token refresh fails.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import Any
+
+logger = logging.getLogger("mfbt.auth_flow")
+
+
+def run_interactive_auth(
+    base_url: str,
+    token_manager: Any,
+    console: Any,
+) -> bool:
+    """Run the browser-based OAuth login flow.
+
+    Opens the user's browser for OAuth authentication with PKCE,
+    waits for the callback, and saves the resulting tokens.
+
+    Args:
+        base_url: The mfbt API base URL.
+        token_manager: A TokenManager instance for saving tokens.
+        console: A Rich Console instance for user feedback.
+
+    Returns:
+        True if authentication succeeded, False otherwise.
+    """
+    import webbrowser
+    from urllib.parse import urlencode
+
+    from mfbt.auth import (
+        OAUTH_SCOPES,
+        AuthError,
+        AuthTimeoutError,
+        ClientRegistrationError,
+        TokenExchangeError,
+        exchange_code_for_tokens,
+        generate_pkce_params,
+        start_callback_server,
+        wait_for_callback,
+    )
+
+    pkce = generate_pkce_params()
+
+    try:
+        server, port = start_callback_server(pkce.state)
+    except AuthError as exc:
+        console.print(f"[red]Error:[/red] {exc}")
+        return False
+
+    redirect_uri = f"http://127.0.0.1:{port}/callback"
+
+    # Register OAuth client (RFC 7591 Dynamic Client Registration)
+    console.print("Registering OAuth client...")
+    try:
+        client_id = token_manager.register_and_save_client([redirect_uri])
+    except ClientRegistrationError as exc:
+        server.server_close()
+        console.print(f"[red]Error:[/red] {exc}")
+        return False
+
+    auth_params = {
+        "client_id": client_id,
+        "redirect_uri": redirect_uri,
+        "code_challenge": pkce.code_challenge,
+        "code_challenge_method": pkce.code_challenge_method,
+        "response_type": "code",
+        "state": pkce.state,
+        "scope": OAUTH_SCOPES,
+    }
+    auth_url = f"{base_url.rstrip('/')}/oauth/authorize?{urlencode(auth_params)}"
+
+    console.print("Opening browser for authentication...")
+    webbrowser.open(auth_url)
+    console.print(f"Waiting for callback (timeout: 120s)...\n")
+
+    try:
+        result = wait_for_callback(server, timeout=120)
+    except AuthTimeoutError:
+        console.print("[red]Error:[/red] Authentication timed out. Please try again.")
+        return False
+    except AuthError as exc:
+        console.print(f"[red]Error:[/red] {exc}")
+        return False
+    except KeyboardInterrupt:
+        console.print("\nAuthentication cancelled.")
+        return False
+    finally:
+        server.server_close()
+
+    console.print("Exchanging authorization code for tokens...")
+    try:
+        token_response = exchange_code_for_tokens(
+            base_url=base_url,
+            code=result.code,
+            code_verifier=pkce.code_verifier,
+            redirect_uri=redirect_uri,
+            client_id=client_id,
+        )
+    except TokenExchangeError as exc:
+        console.print(f"[red]Error:[/red] {exc}")
+        return False
+
+    token_manager.save_tokens(token_response)
+    console.print("[green]Successfully authenticated![/green]\n")
+    return True