Skip to content

Commit 5b64643

Browse files
authored
Export: latest-day filter, project-scoped CLI, and bulk API hardening (#34)
* feat(export): enhance export functionality with incremental updates and latest-day slice - Updated README to clarify bulk export options, including incremental updates and latest-day slice. - Modified CLI export flags to support `--since incremental` for exporting only new or changed sessions since the last export. - Implemented logic in the export API to handle the new export options, including returning a 422 error when there is nothing to export. - Enhanced session filtering for the latest activity day and improved project matching for exports. - Added unit tests for new export features and validation of state management. * refactor(export): centralize export state management and enhance error handling - Introduced a new utility module for managing export state with atomic I/O and locking mechanisms. - Updated the export API to validate request bodies and handle invalid 'since' parameters, returning appropriate error responses. - Enhanced the README to clarify bulk export options and CLI export flag behaviors. - Added unit tests for new error handling scenarios in the export API and improved logging for session parsing failures. - Refactored existing code to utilize the new export state management utilities, ensuring consistency across API and CLI. * test(export): add unit tests for load_export_state_from_disk validation * test(export): add additional unit tests for iso_timestamp_to_date function
1 parent 07f7cb0 commit 5b64643

11 files changed

Lines changed: 1196 additions & 181 deletions

README.md

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -19,13 +19,14 @@ Browse and export Claude Code chat history — Web GUI and CLI.
1919
- **Smooth transitions** — staggered card/message animations, crossfade content swaps
2020
- **Scroll-to-top button** in bottom-right corner
2121
- **Per-model badges** in session header
22-
- **Bulk export** — download all sessions as a zip
22+
- **Bulk export** — download all sessions, incremental updates, or latest-day slice as a zip; if there is nothing to export, the API returns **422** with JSON body `{"error": "Nothing to export", "since": "<mode>"}` (the `since` field echoes your request: `"all"`, `"last"`, or `"incremental"`) instead of an empty zip
2323

2424
### CLI Export
2525
- Standalone script to export all sessions to Markdown with YAML frontmatter
2626
- Rich Markdown: token usage, tool calls, thinking blocks, model info, timestamps
27-
- `--since last` flag for incremental export (only new/updated sessions)
28-
- `--project` flag to export a specific project
27+
- `--since last` — export every session that overlaps the **latest UTC calendar day** present in your history (default zip name: `claude-code-export-last-MM-DD-YYYY-MM-DD.zip` — the first `MM-DD` is that latest UTC day, and `YYYY-MM-DD` is the export date)
28+
- `--since incremental` — export only sessions **new or changed since the last export** (file mtime + saved state)
29+
- `--project` flag to export a subset of projects
2930

3031
## Quick Start
3132

@@ -60,7 +61,7 @@ python app.py --base-dir /path/to/claude/projects
6061
```bash
6162
# Activate venv first (see above), then:
6263

63-
# List all projects (shows directory names you can use with --project)
64+
# List all projects (first column is a friendly name; --project accepts that or the dir slug)
6465
python scripts/export.py list
6566

6667
# Export all sessions as zip
@@ -69,14 +70,17 @@ python scripts/export.py
6970
# Export to specific directory, no zip
7071
python scripts/export.py --out ./exports --no-zip
7172

72-
# Incremental export (only new sessions since last run)
73+
# Latest calendar day (UTC): all sessions active on that day; zip pattern claude-code-export-last-MM-DD-YYYY-MM-DD.zip (e.g. claude-code-export-last-04-06-2026-05-08.zip — 04-06 = latest UTC day, 2026-05-08 = export date)
7374
python scripts/export.py --since last
7475

75-
# Export specific project only (substring match on directory name)
76+
# Incremental (only new/updated sessions since last run, using export state)
77+
python scripts/export.py --since incremental
78+
79+
# Export specific project only (substring on friendly name from list and/or dir name under ~/.claude/projects/)
7680
python scripts/export.py --project boost-capy
7781
```
7882

79-
The `--project` flag matches against the directory names under `~/.claude/projects/`. These are path-based names like `F--boost-capy` or `d--harbor-forge`. You can use any substring — for example `boost-capy` will match `F--boost-capy`. Run `python scripts/export.py list` to see all available project names.
83+
The `--project` flag matches a **case-insensitive substring** of either the **Project** column from `list` (derived from the session working directory) or the internal directory name under `~/.claude/projects/` (for example `F--boost-capy` or `d--harbor-forge`). A substring like `boost-capy` matches `F--boost-capy`; you can also paste the friendly name shown in `list`.
8084

8185
## Data Source
8286

api/export_api.py

Lines changed: 189 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -8,120 +8,245 @@
88

99
from flask import Blueprint, current_app, jsonify, request, send_file
1010

11-
from utils.session_path import get_claude_projects_dir, list_projects, list_sessions
11+
from utils.export_state_store import (
12+
EXPORT_STATE_FILE,
13+
atomic_write_export_state,
14+
export_state_lock,
15+
load_export_state_from_disk,
16+
)
17+
from utils.session_path import (
18+
get_claude_projects_dir,
19+
list_projects,
20+
list_sessions,
21+
)
1222
from utils.jsonl_parser import parse_session
1323
from utils.session_stats import compute_stats
1424
from utils.md_exporter import session_to_markdown
1525
from utils.json_exporter import session_to_json
1626
from utils.exclusion_rules import is_session_excluded
1727
from utils.slugify import slugify
28+
from utils.export_day_filter import collect_sessions_for_latest_activity_day
1829

1930
export_bp = Blueprint("export", __name__)
2031

21-
_STATE_FILE = os.path.join(os.path.expanduser("~"), ".claude-code-chat-browser", "export_state.json")
32+
# Tests monkeypatch this path; keep in sync with utils.export_state_store.
33+
_STATE_FILE = EXPORT_STATE_FILE
34+
35+
36+
def _state_lock():
37+
return export_state_lock(_STATE_FILE)
38+
39+
40+
def _load_state_from_disk() -> dict:
41+
return load_export_state_from_disk(_STATE_FILE)
42+
43+
44+
def _atomic_write_state(state: dict) -> None:
45+
atomic_write_export_state(state, _STATE_FILE)
2246

2347

2448
def _read_state() -> dict:
25-
if os.path.exists(_STATE_FILE):
26-
try:
27-
with open(_STATE_FILE) as f:
28-
return json.load(f)
29-
except Exception:
30-
pass
31-
return {}
49+
with _state_lock():
50+
return _load_state_from_disk()
3251

3352

34-
def _write_state(sessions_map: dict, count: int):
35-
os.makedirs(os.path.dirname(_STATE_FILE), exist_ok=True)
36-
state = _read_state()
37-
state["lastExportTime"] = datetime.now().isoformat()
38-
state["exportedCount"] = count
39-
state.setdefault("sessions", {}).update(sessions_map)
40-
with open(_STATE_FILE, "w") as f:
41-
json.dump(state, f, indent=2)
53+
def _write_state(sessions_map: dict, count: int) -> None:
54+
"""Persist merge of *sessions_map* and update last-export metadata (*count* = this run only)."""
55+
with _state_lock():
56+
state = _load_state_from_disk()
57+
state["lastExportTime"] = datetime.now().isoformat()
58+
state["exportedCount"] = count
59+
state.setdefault("sessions", {}).update(sessions_map)
60+
_atomic_write_state(state)
4261

4362

4463
@export_bp.route("/api/export/state")
4564
def get_export_state():
4665
state = _read_state()
47-
return jsonify({
48-
"last_export_time": state.get("lastExportTime"),
49-
"export_count": state.get("exportedCount", 0),
50-
})
66+
n = state.get("exportedCount", 0)
67+
return jsonify(
68+
{
69+
"last_export_time": state.get("lastExportTime"),
70+
# Sessions exported in the last completed bulk export (not a lifetime total).
71+
"last_export_session_count": n,
72+
"export_count": n,
73+
}
74+
)
5175

5276

5377
@export_bp.route("/api/export", methods=["POST"])
5478
def bulk_export():
55-
body = request.get_json(silent=True) or {}
56-
since = "last" if body.get("since") == "last" else "all"
79+
body = request.get_json(silent=True)
80+
if body is None:
81+
body = {}
82+
if not isinstance(body, dict):
83+
return jsonify({"error": "Invalid request body"}), 400
5784

58-
base = current_app.config.get("CLAUDE_PROJECTS_DIR") or get_claude_projects_dir()
85+
since = body.get("since", "all")
86+
if since not in ("all", "last", "incremental"):
87+
return jsonify({"error": "Invalid since mode", "since": since}), 400
88+
89+
base = (
90+
current_app.config.get("CLAUDE_PROJECTS_DIR")
91+
or get_claude_projects_dir()
92+
)
5993
projects = list_projects(base)
6094
rules = current_app.config.get("EXCLUSION_RULES") or []
6195

6296
state = _read_state()
63-
last_export_sessions: dict = state.get("sessions", {}) if since == "last" else {}
97+
last_export_sessions: dict = (
98+
state.get("sessions", {}) if since == "incremental" else {}
99+
)
64100

65101
buf = io.BytesIO()
66102
count = 0
67103
manifest = []
68104
new_sessions_map: dict = {}
105+
latest_day = None
106+
69107
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
70-
for project in projects:
71-
sessions = list_sessions(project["path"])
72-
for sess_info in sessions:
108+
if since == "last":
109+
d, rows, _n = collect_sessions_for_latest_activity_day(
110+
projects,
111+
list_sessions=list_sessions,
112+
parse_session=parse_session,
113+
is_session_excluded=is_session_excluded,
114+
rules=rules,
115+
)
116+
latest_day = d
117+
for project, sess_info, session, _st, _en in rows:
73118
sid = sess_info["id"]
74119
try:
75-
if since == "last":
76-
prev_mtime = last_export_sessions.get(sid, 0)
77-
curr_mtime = sess_info.get("modified", 0)
78-
if curr_mtime and curr_mtime <= prev_mtime:
79-
continue
80-
81-
session = parse_session(sess_info["path"])
82-
if session["title"] == "Untitled Session":
83-
continue
84-
85-
if is_session_excluded(
86-
rules,
87-
session,
88-
project.get("display_name") or project["name"],
89-
):
90-
continue
91-
92120
stats = compute_stats(session)
93121
md = session_to_markdown(session, stats)
94122
title_slug = slugify(session["title"], default="session")
95123
short_id = sid[:8]
96124
proj_slug = slugify(project["name"], default="project")
97125
ts = session["metadata"].get("first_timestamp", "")
98-
ts_file = ts[:19].replace(":", "-") if ts else "0000-00-00T00-00-00"
99-
rel_path = f"{proj_slug}/{ts_file}__{title_slug}__{short_id}.md"
126+
ts_file = (
127+
ts[:19].replace(":", "-")
128+
if ts
129+
else "0000-00-00T00-00-00"
130+
)
131+
rel_path = (
132+
f"{proj_slug}/{ts_file}__{title_slug}__{short_id}.md"
133+
)
100134
zf.writestr(rel_path, md)
101-
manifest.append({
102-
"session_id": sid,
103-
"title": session["title"],
104-
"project": project["name"],
105-
"tokens": session["metadata"]["total_input_tokens"]
106-
+ session["metadata"]["total_output_tokens"],
107-
"tool_calls": session["metadata"]["total_tool_calls"],
108-
"cost_estimate_usd": stats.get("cost_estimate_usd"),
109-
})
135+
manifest.append(
136+
{
137+
"session_id": sid,
138+
"title": session["title"],
139+
"project": project["name"],
140+
"tokens": session["metadata"]["total_input_tokens"]
141+
+ session["metadata"]["total_output_tokens"],
142+
"tool_calls": session["metadata"][
143+
"total_tool_calls"
144+
],
145+
"cost_estimate_usd": stats.get(
146+
"cost_estimate_usd"
147+
),
148+
}
149+
)
110150
new_sessions_map[sid] = sess_info.get("modified", 0)
111151
count += 1
112152
except Exception as e:
113-
current_app.logger.warning("Failed to export %s: %s", sid[:10], e)
153+
current_app.logger.warning(
154+
"Failed to export %s: %s", sid[:10], e
155+
)
114156
continue
157+
else:
158+
for project in projects:
159+
sessions = list_sessions(project["path"])
160+
for sess_info in sessions:
161+
sid = sess_info["id"]
162+
try:
163+
if since == "incremental":
164+
prev_mtime = last_export_sessions.get(sid, 0)
165+
curr_mtime = sess_info.get("modified", 0)
166+
if curr_mtime and curr_mtime <= prev_mtime:
167+
continue
168+
169+
session = parse_session(sess_info["path"])
170+
if session["title"] == "Untitled Session":
171+
continue
172+
173+
if is_session_excluded(
174+
rules,
175+
session,
176+
project.get("display_name") or project["name"],
177+
):
178+
continue
179+
180+
stats = compute_stats(session)
181+
md = session_to_markdown(session, stats)
182+
title_slug = slugify(
183+
session["title"], default="session"
184+
)
185+
short_id = sid[:8]
186+
proj_slug = slugify(project["name"], default="project")
187+
ts = session["metadata"].get("first_timestamp", "")
188+
ts_file = (
189+
ts[:19].replace(":", "-")
190+
if ts
191+
else "0000-00-00T00-00-00"
192+
)
193+
rel_path = f"{proj_slug}/{ts_file}__{title_slug}__{short_id}.md"
194+
zf.writestr(rel_path, md)
195+
manifest.append(
196+
{
197+
"session_id": sid,
198+
"title": session["title"],
199+
"project": project["name"],
200+
"tokens": session["metadata"][
201+
"total_input_tokens"
202+
]
203+
+ session["metadata"]["total_output_tokens"],
204+
"tool_calls": session["metadata"][
205+
"total_tool_calls"
206+
],
207+
"cost_estimate_usd": stats.get(
208+
"cost_estimate_usd"
209+
),
210+
}
211+
)
212+
new_sessions_map[sid] = sess_info.get("modified", 0)
213+
count += 1
214+
except Exception as e:
215+
current_app.logger.warning(
216+
"Failed to export %s: %s", sid[:10], e
217+
)
218+
continue
115219
if manifest:
116-
manifest_str = "\n".join(json.dumps(e, default=str) for e in manifest)
220+
manifest_str = "\n".join(
221+
json.dumps(e, default=str) for e in manifest
222+
)
117223
zf.writestr("manifest.jsonl", manifest_str)
118224

119225
if count > 0:
120226
_write_state(new_sessions_map, count)
121227

228+
if count == 0:
229+
return (
230+
jsonify(
231+
{
232+
"error": "Nothing to export",
233+
"since": since,
234+
}
235+
),
236+
422,
237+
)
238+
122239
buf.seek(0)
123240
date_tag = datetime.now().strftime("%Y-%m-%d")
124-
suffix = "-since-last" if since == "last" else ""
241+
if since == "last":
242+
if latest_day is not None:
243+
suffix = f"-last-{latest_day.strftime('%m-%d')}"
244+
else:
245+
suffix = "-last"
246+
elif since == "incremental":
247+
suffix = "-incremental"
248+
else:
249+
suffix = ""
125250
return send_file(
126251
buf,
127252
mimetype="application/zip",
@@ -134,7 +259,11 @@ def bulk_export():
134259
def export_session(project_name, session_id):
135260
import os
136261
from utils.session_path import safe_join
137-
base = current_app.config.get("CLAUDE_PROJECTS_DIR") or get_claude_projects_dir()
262+
263+
base = (
264+
current_app.config.get("CLAUDE_PROJECTS_DIR")
265+
or get_claude_projects_dir()
266+
)
138267
try:
139268
filepath = safe_join(base, project_name, f"{session_id}.jsonl")
140269
except ValueError:
@@ -171,5 +300,3 @@ def export_session(project_name, session_id):
171300
as_attachment=True,
172301
download_name=f"{title_slug}.md",
173302
)
174-
175-

0 commit comments

Comments
 (0)