Add server mode with live reload (rdoc --server) (ruby#1620)

A better attempt of ruby#1151

Implement `rdoc --server[=PORT]` for previewing documentation with
automatic browser refresh on source file changes. The server parses all
sources on startup, serves pages from memory via the Aliki generator,
and watches for file modifications, additions, and deletions —
re-parsing only what changed.


https://github.com/user-attachments/assets/06d5ae7b-cc54-4cc5-a8b0-1cc30177cadb


## Changes

- **New `RDoc::Server`** (`lib/rdoc/server.rb`) — minimal HTTP server
using Ruby's built-in `TCPServer` (no WEBrick or external dependencies)
  - Thread-per-connection with `Connection: close`
  - Persistent Aliki generator instance rendering pages to strings
  - In-memory page cache with full invalidation on changes
- Live reload via inline JS polling `/__status` endpoint every 1 second
  - Background file watcher polling source file mtimes every 1 second
- Incremental re-parse: only changed files are re-parsed, old data
removed first
  - Detection of new and deleted source files
- **`--server[=PORT]`** CLI option (default port 4000) and `rdoc:server`
Rake task
- **`RDoc::Store#remove_file`** — removes a file's entries from the
store hashes
- **`RDoc::Store#clear_file_contributions`** — surgically removes a
file's methods, constants, comments, includes, extends, and aliases from
its classes/modules, preserving classes that span multiple files.
Supports `keep_position:` for server re-parse to preserve comment
ordering.
- **`RDoc::RDoc#relative_path_for`** — extracted path normalization
(against `options.root` and `options.page_dir`) shared by `parse_file`
and the server
- **`Darkfish#refresh_store_data`** — extracted for reuse by the server
after re-parsing
- **`RDoc::Servlet` → `RDoc::RI::Servlet`** — moved to clarify
RI-specific usage

## Security & robustness

- Binds to `127.0.0.1` only (localhost)
- Path traversal protection in asset serving (`File.expand_path`
containment check)
- Proper HTTP error responses: 400, 404, 405, 500
- 5-second `IO.select` read timeout on client sockets
- Mutex protects all store mutations, generator refresh, and cache
invalidation atomically
- Individual `parse_file` errors rescued so one failure doesn't block
remaining files
- Watcher thread uses `@running` flag with clean shutdown via
`Thread#join`

## Known limitations

- **Full cache invalidation:** any file change clears all cached pages.
Rendering is fast (~ms per page); parsing is the expensive part and is
done incrementally.
- **Template/CSS changes:** require server restart (only source files
are watched).

---------

Co-authored-by: Sutou Kouhei <kou@clear-code.com>

Author: st0012

.claude/skills/test-server/SKILL.md

@@ -0,0 +1,120 @@
+---
+name: test-server
+description: E2E testing workflow for the RDoc live-reload server (rdoc --server)
+---
+
+# Test Server
+
+End-to-end testing workflow for the RDoc live-reload server. Use after modifying server code, templates, generators, or routing.
+
+## Steps
+
+### 1. Start the server
+
+```bash
+bundle exec rdoc --server &
+SERVER_PID=$!
+sleep 2  # wait for TCP server to bind
+```
+
+Or on a custom port:
+
+```bash
+bundle exec rdoc --server=8080 &
+```
+
+Default port is 4000.
+
+### 2. Verify core endpoints
+
+Run these curl checks against the running server:
+
+```bash
+# Root → 200, HTML index page
+curl -s -o /dev/null -w '%{http_code}' http://localhost:4000/
+# Expected: 200
+
+# Status endpoint → 200, JSON with last_change float
+curl -s http://localhost:4000/__status
+# Expected: {"last_change":1234567890.123}
+
+# Class page → 200, HTML with live-reload script
+curl -s http://localhost:4000/RDoc.html | head -5
+# Expected: HTML containing class documentation
+
+# CSS asset → 200, stylesheet
+curl -s -o /dev/null -w '%{http_code}' http://localhost:4000/css/rdoc.css
+# Expected: 200
+
+# JS search index → 200, search data
+curl -s -o /dev/null -w '%{http_code}' http://localhost:4000/js/search_data.js
+# Expected: 200
+
+# Missing page → 404, still has live-reload script
+curl -s -w '\n%{http_code}' http://localhost:4000/Missing.html | tail -1
+# Expected: 404
+
+# Path traversal via asset route → 404 (blocked by expand_path check)
+curl -s -o /dev/null -w '%{http_code}' 'http://localhost:4000/css/../../etc/passwd'
+# Expected: 404
+```
+
+### 3. Verify live-reload
+
+HTML pages should contain the live-reload polling script:
+
+```bash
+# Check for live-reload script in a class page
+curl -s http://localhost:4000/RDoc.html | grep 'var lastChange'
+# Expected: var lastChange = <float>;
+
+# Check that 404 pages also get live-reload
+curl -s http://localhost:4000/Missing.html | grep 'var lastChange'
+# Expected: var lastChange = <float>;
+```
+
+The script polls `/__status` and reloads when `data.last_change > lastChange`.
+
+### 4. Verify file change detection
+
+Confirm the server detects source file changes and invalidates its cache:
+
+```bash
+# Record the current last_change timestamp
+BEFORE=$(curl -s http://localhost:4000/__status | grep -o '"last_change":[0-9.]*' | cut -d: -f2)
+
+# Touch a source file to trigger the file watcher
+touch lib/rdoc.rb
+sleep 2  # watcher polls every 1 second
+
+# Check that last_change has advanced
+AFTER=$(curl -s http://localhost:4000/__status | grep -o '"last_change":[0-9.]*' | cut -d: -f2)
+echo "before=$BEFORE after=$AFTER"
+# Expected: AFTER > BEFORE
+```
+
+### 5. (Optional) Visual testing with Playwright CLI
+
+For visual inspection of rendered pages, use Playwright CLI commands directly:
+
+```bash
+# Install browsers (one-time)
+npx playwright install chromium
+
+# Take a screenshot of the index page
+npx playwright screenshot http://localhost:4000/ /tmp/rdoc-index.png
+
+# Take a screenshot of a specific class page
+npx playwright screenshot http://localhost:4000/RDoc.html /tmp/rdoc-class.png
+
+# Full-page screenshot
+npx playwright screenshot --full-page http://localhost:4000/RDoc.html /tmp/rdoc-full.png
+```
+
+Review the screenshots to verify layout, styling, and content rendering.
+
+### 6. Stop the server
+
+```bash
+kill $SERVER_PID 2>/dev/null
+```

AGENTS.md

@@ -125,6 +125,12 @@ bundle exec rake rerdoc
 # Show documentation coverage
 bundle exec rake rdoc:coverage
 bundle exec rake coverage
+
+# Start live-reloading preview server (port 4000)
+bundle exec rake rdoc:server
+
+# Or via CLI with custom port
+bundle exec rdoc --server=8080
 ```
 
 **Output Directory:** `_site/` (GitHub Pages compatible)
@@ -176,6 +182,7 @@ lib/rdoc/
 │   ├── c.rb                   # C extension parser
 │   ├── prism_ruby.rb          # Prism-based Ruby parser
 │   └── ...
+├── server.rb                  # Live-reloading preview server (rdoc --server)
 ├── generator/                 # Documentation generators
 │   ├── aliki.rb               # HTML generator (default theme)
 │   ├── darkfish.rb            # HTML generator (deprecated, will be removed in v8.0)
@@ -232,6 +239,30 @@ exe/
 - **Parsers:** Ruby, C, Markdown, RD, Prism-based Ruby (experimental)
 - **Generators:** HTML/Aliki (default), HTML/Darkfish (deprecated), RI, POT (gettext), JSON, Markup
 
+### Live Preview Server (`RDoc::Server`)
+
+The server (`lib/rdoc/server.rb`) provides `rdoc --server` for live documentation preview.
+
+**Architecture:**
+- Uses Ruby's built-in `TCPServer` (`socket` stdlib) — no WEBrick or external dependencies
+- Creates a persistent `RDoc::Generator::Aliki` instance with `file_output = false` (renders to strings)
+- Thread-per-connection HTTP handling with `Connection: close` (no keep-alive)
+- Background watcher thread polls file mtimes every 1 second
+- Live reload via inline JS polling `/__status` endpoint
+
+**Key files:**
+- `lib/rdoc/server.rb` — HTTP server, routing, caching, file watcher
+- `lib/rdoc/rdoc.rb` — `start_server` method, server branch in `document`
+- `lib/rdoc/options.rb` — `--server[=PORT]` option
+- `lib/rdoc/generator/darkfish.rb` — `refresh_store_data` (extracted for server reuse)
+- `lib/rdoc/store.rb` — `remove_file` (for deleted file handling)
+- `lib/rdoc/task.rb` — `rdoc:server` Rake task
+
+**Known limitations:**
+- Reopened classes: deleting a file that partially defines a class removes the entire class from the store (save the other file to restore)
+- Template/CSS changes require server restart (only source files are watched)
+- Full page cache invalidation on any change (rendering is fast, so this is acceptable)
+
 ## Common Workflows
 
 Do NOT commit anything. Ask the developer to review the changes after tasks are finished.
@@ -319,20 +350,36 @@ When editing markup reference documentation, such as `doc/markup_reference/markd
 
 When making changes to theme CSS or templates (e.g., Darkfish or Aliki themes):
 
-1. **Generate documentation**: Run `bundle exec rake rerdoc` to create baseline
-2. **Start HTTP server**: Run `cd _site && python3 -m http.server 8000` (use different port if 8000 is in use)
-3. **Investigate with Playwright**: Ask the AI assistant to take screenshots and inspect the documentation visually
-   - Example: "Navigate to the docs at localhost:8000 and screenshot the RDoc module page"
-   - See "Playwright MCP for Testing Generated Documentation" section below for details
-4. **Make changes**: Edit files in `lib/rdoc/generator/template/<theme>/` as needed
-5. **Regenerate**: Run `bundle exec rake rerdoc` to rebuild documentation with changes
-6. **Verify with Playwright**: Take new screenshots and compare to original issues
-7. **Lint changes** (if modified):
+1. **Start the live-reloading server**: Run `bundle exec rdoc --server` (or `bundle exec rake rdoc:server`)
+2. **Make changes**: Edit files in `lib/rdoc/generator/template/<theme>/` or source code
+3. **Browser auto-refreshes**: The server detects file changes and refreshes the browser automatically
+4. **Verify with `/test-server`**: Use the test-server skill for endpoint checks, live-reload verification, and optional Playwright screenshots
+5. **Lint changes** (if modified):
    - ERB templates: `npx @herb-tools/linter "lib/rdoc/generator/template/**/*.rhtml"`
    - CSS files: `npm run lint:css -- --fix`
-8. **Stop server**: Kill the HTTP server process when done
 
-**Tip:** Keep HTTP server running during iteration. Just regenerate with `bundle exec rake rerdoc` between changes.
+**Note:** The server watches source files, not template files. If you modify `.rhtml` templates or CSS in the template directory, restart the server to pick up those changes.
+
+## Visual Testing with Playwright CLI
+
+Use `npx playwright` to take screenshots of generated documentation — works with both the live-reload server and static `_site/` output.
+
+```bash
+# Install browsers (one-time)
+npx playwright install chromium
+
+# Screenshot a live server page
+npx playwright screenshot http://localhost:4000/RDoc.html /tmp/rdoc-class.png
+
+# Screenshot static output (start a file server first)
+cd _site && python3 -m http.server 8000 &
+npx playwright screenshot http://localhost:8000/index.html /tmp/rdoc-index.png
+
+# Full-page screenshot
+npx playwright screenshot --full-page http://localhost:4000/RDoc.html /tmp/rdoc-full.png
+```
+
+For server-specific E2E testing (endpoint checks, live-reload verification, file change detection), use the `/test-server` skill.
 
 ## Notes for AI Agents
 
@@ -345,64 +392,3 @@ When making changes to theme CSS or templates (e.g., Darkfish or Aliki themes):
 4. **Use `rake rerdoc`** to regenerate documentation (not just `rdoc`)
 5. **Verify generated files** with `rake verify_generated`
 6. **Don't edit generated files** directly (in `lib/rdoc/markdown/` and `lib/rdoc/rd/`)
-
-## Playwright MCP for Testing Generated Documentation
-
-The Playwright MCP server enables visual inspection and interaction with generated HTML documentation. This is useful for verifying CSS styling, layout issues, and overall appearance.
-
-**MCP Server:** `@playwright/mcp` (Microsoft's official browser automation server)
-
-### Setup
-
-The Playwright MCP server can be used with any MCP-compatible AI tool (Claude Code, Cursor, GitHub Copilot, OpenAI Agents, etc.).
-
-**Claude Code:**
-
-```bash
-/plugin playwright
-```
-
-**Other MCP-compatible tools:**
-
-```bash
-npx @playwright/mcp@latest
-```
-
-Configure your tool to connect to this MCP server. Playwright launches its own browser instance automatically - no manual browser setup or extensions required.
-
-### Troubleshooting: Chrome Remote Debugging Blocked
-
-If you encounter `DevTools remote debugging is disallowed by the system admin`, Chrome's debugging is blocked by the machine's policy. Use Firefox instead:
-
-```bash
-# Install Firefox for Playwright
-npx playwright install firefox
-
-# Add Playwright MCP with Firefox to your project (creates/updates .mcp.json)
-claude mcp add playwright --scope project -- npx -y @playwright/mcp@latest --browser firefox
-```
-
-Restart Claude Code after running these commands.
-
-### Testing Generated Documentation
-
-To test the generated documentation:
-
-```bash
-# Generate documentation
-bundle exec rake rerdoc
-
-# Start a simple HTTP server in the _site directory (use an available port)
-cd _site && python3 -m http.server 8000
-```
-
-If port 8000 is already in use, try another port (e.g., `python3 -m http.server 9000`).
-
-Then ask the AI assistant to inspect the documentation. It will use the appropriate Playwright tools (`browser_navigate`, `browser_snapshot`, `browser_take_screenshot`, etc.) based on your request.
-
-**Example requests:**
-
-- "Navigate to `http://localhost:8000` and take a screenshot"
-- "Take a screenshot of the RDoc module page"
-- "Check if code blocks are rendering properly on the Markup page"
-- "Compare the index page before and after my CSS changes"

README.md

@@ -180,6 +180,41 @@ There are also a few community-maintained themes for RDoc:
 
 Please follow the theme's README for usage instructions.
 
+## Live Preview Server
+
+RDoc includes a built-in server for previewing documentation while you edit source files. It parses your code once on startup, then watches for changes and auto-refreshes the browser.
+
+```shell
+rdoc --server
+```
+
+This starts a server at `http://localhost:4000`. You can specify a different port:
+
+```shell
+rdoc --server=8080
+```
+
+Or use the Rake task:
+
+```shell
+rake rdoc:server
+```
+
+### How It Works
+
+- Parses all source files on startup and serves pages from memory using the Aliki theme
+- A background thread polls file mtimes every second
+- When a file changes, only that file is re-parsed — the browser refreshes automatically
+- New files are detected and added; deleted files are removed
+
+**No external dependencies.** The server uses Ruby's built-in `TCPServer` (`socket` stdlib) — no WEBrick or other gems required.
+
+### Limitations
+
+- **Reopened classes and file deletion.** If a class is defined across multiple files (e.g. `Foo` in both `a.rb` and `b.rb`), deleting one file removes the entire class from the store, including parts from the other file. Saving the remaining file triggers a re-parse that restores it.
+- **Full cache invalidation.** Any file change clears all cached pages. This is simple and correct — rendering is fast (~ms per page), parsing is the expensive part and is done incrementally.
+- **No HTTPS or HTTP/2.** The server is intended for local development preview only.
+
 ## Bugs
 
 See [CONTRIBUTING.md](CONTRIBUTING.md) for information on filing a bug report.  It's OK to file a bug report for anything you're having a problem with.  If you can't figure out how to make RDoc produce the output you like that is probably a documentation bug.

lib/rdoc.rb

@@ -160,7 +160,7 @@ def self.home
   autoload :Generator,      "#{__dir__}/rdoc/generator"
   autoload :Options,        "#{__dir__}/rdoc/options"
   autoload :Parser,         "#{__dir__}/rdoc/parser"
-  autoload :Servlet,        "#{__dir__}/rdoc/servlet"
+  autoload :Server,         "#{__dir__}/rdoc/server"
   autoload :RI,             "#{__dir__}/rdoc/ri"
   autoload :Stats,          "#{__dir__}/rdoc/stats"
   autoload :Store,          "#{__dir__}/rdoc/store"

lib/rdoc/code_object/class_module.rb

@@ -747,6 +747,18 @@ def search_snippet
     snippet(first_comment)
   end
 
+  ##
+  # Rebuilds +@comment+ from the current +@comment_location+ entries,
+  # skipping any empty placeholders.
+
+  def rebuild_comment_from_location
+    texts = @comment_location.each_value.flat_map { |comments|
+      comments.filter_map { |c| c.to_s unless c.empty? }
+    }
+    merged = texts.join("\n---\n")
+    @comment = merged.empty? ? '' : RDoc::Comment.new(merged)
+  end
+
   ##
   # Sets the store for this class or module and its contained code objects.
 

lib/rdoc/generator/darkfish.rb

@@ -577,6 +577,15 @@ def setup
 
     return unless @store
 
+    refresh_store_data
+  end
+
+  ##
+  # Refreshes the generator's data from the store.  Called by #setup and
+  # can be called again after the store has been updated (e.g. in server
+  # mode after re-parsing changed files).
+
+  def refresh_store_data
     @classes = @store.all_classes_and_modules.sort
     @files   = @store.all_files.sort
     @methods = @classes.flat_map { |m| m.method_list }.sort