Skip to content

Commit fa661e6

Browse files
Address release readiness blockers
Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent e484d6a commit fa661e6

8 files changed

Lines changed: 168 additions & 335 deletions

File tree

.github/workflows/ci.yml

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
- "codex/**"
8+
pull_request:
9+
10+
jobs:
11+
python:
12+
name: Python validation
13+
runs-on: ubuntu-latest
14+
15+
steps:
16+
- name: Check out repository
17+
uses: actions/checkout@v4
18+
19+
- name: Set up Python
20+
uses: actions/setup-python@v5
21+
with:
22+
python-version: "3.11"
23+
24+
- name: Install Python dependencies
25+
run: |
26+
python -m pip install --upgrade pip
27+
pip install -r requirements.txt
28+
pip install -r python_sidecar/requirements.txt
29+
30+
- name: Run unit tests
31+
run: python -m unittest discover -s tests
32+
33+
- name: Compile Python modules
34+
run: python -m compileall main.py scraper.py document_generator.py export_schema.py diagnostics.py python_sidecar
35+
36+
frontend:
37+
name: Frontend build
38+
runs-on: ubuntu-latest
39+
40+
steps:
41+
- name: Check out repository
42+
uses: actions/checkout@v4
43+
44+
- name: Set up Node
45+
uses: actions/setup-node@v4
46+
with:
47+
node-version: "20"
48+
cache: "npm"
49+
50+
- name: Install frontend dependencies
51+
run: npm ci
52+
53+
- name: Build frontend
54+
run: npm run build
55+
56+
rust:
57+
name: Rust check
58+
runs-on: ubuntu-latest
59+
60+
steps:
61+
- name: Check out repository
62+
uses: actions/checkout@v4
63+
64+
- name: Set up Rust
65+
uses: dtolnay/rust-toolchain@stable
66+
67+
- name: Install Linux Tauri dependencies
68+
run: |
69+
sudo apt-get update
70+
sudo apt-get install -y \
71+
libwebkit2gtk-4.1-dev \
72+
libappindicator3-dev \
73+
librsvg2-dev \
74+
patchelf
75+
76+
- name: Check Tauri crate
77+
working-directory: src-tauri
78+
run: cargo check --locked

RELEASE_CHECKLIST.md

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -19,25 +19,34 @@ Use this checklist before tagging or publishing a desktop release. A release is
1919
| Export validation completed for JSON/Markdown/DOCX | Not verified | Yes |
2020
| Known limitations and disclaimers included | Not verified | Yes |
2121

22+
## Sprint Progress Notes
23+
24+
- Implemented: placeholder license, paid-tier, AI, CSV/Excel, and scheduled-scraping claims were removed from Settings and first-run Onboarding.
25+
- Implemented: JSON export generation now accepts `scrape_type`, and CLI/sidecar JSON exports pass profile/bookmarks metadata through.
26+
- Implemented: browser-free tests now cover bookmark `scrape_type` metadata in payloads and saved JSON exports.
27+
- Implemented: `.github/workflows/ci.yml` runs existing Python unit tests, Python compile checks, frontend build, and Rust `cargo check`.
28+
- Still not complete: installer packaging, live X manual testing, privacy/security review, sidecar bundling, history re-export, and broader desktop release gates.
29+
2230
## Pre-Release Code Validation
2331

2432
### Python
25-
- [ ] Run `python -m unittest discover -s tests`.
26-
- [ ] Run `python -m compileall main.py scraper.py document_generator.py export_schema.py diagnostics.py python_sidecar`.
27-
- [ ] Verify `tests/test_exports.py` covers JSON schema version, safe paths, Markdown, and DOCX output.
28-
- [ ] Add or verify a test that bookmark exports can emit `scrape_type: "bookmarks"`.
33+
- [x] CI added for `python -m unittest discover -s tests`.
34+
- [x] CI added for `python -m compileall main.py scraper.py document_generator.py export_schema.py diagnostics.py python_sidecar`.
35+
- [x] Verify `tests/test_exports.py` covers JSON schema version, safe paths, Markdown, and DOCX output.
36+
- [x] Add or verify a test that bookmark exports can emit `scrape_type: "bookmarks"`.
2937
- [ ] Add or verify sidecar command tests for `login`, `scrape`, `export`, `diagnostics`, `pause`, `resume`, `cancel`, `status`, and `stop` without launching Chrome.
3038
- [ ] Confirm `python_sidecar/requirements.txt` and packaged sidecar dependencies match the release runtime.
3139

3240
### Frontend
33-
- [ ] Run `npm install` on a clean checkout.
34-
- [ ] Run `npm run build`.
41+
- [x] CI added using `npm ci`.
42+
- [x] CI added for `npm run build`.
3543
- [ ] Verify TypeScript build has no hidden errors from unused or stale hooks.
3644
- [ ] Verify `src/hooks/useScraper.ts` is either removed, covered, or clearly unused.
3745
- [ ] Verify result export UI uses the intended source of truth for JSON/Markdown/DOCX.
38-
- [ ] Verify hardcoded feature copy does not claim unshipped CSV/Excel/scheduling/paid-tier behavior.
46+
- [x] Verify Settings and Onboarding copy do not claim unshipped CSV/Excel/scheduling/paid-tier behavior.
3947

4048
### Rust/Tauri
49+
- [x] CI added for `cargo check --locked` in `src-tauri`.
4150
- [ ] Run `cargo test` in `src-tauri` after Rust tests exist.
4251
- [ ] Run `npm run tauri build`.
4352
- [ ] Verify command argument naming between React invokes and Rust command structs.

document_generator.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -198,6 +198,7 @@ def create_json_document(
198198
output_path: str,
199199
target_username: str,
200200
output_dir: Optional[str] = None,
201+
scrape_type: str = "profile",
201202
) -> str:
202203
"""
203204
Tweetleri JSON formatında kaydet (MCP-ready)
@@ -207,11 +208,12 @@ def create_json_document(
207208
output_path: Çıktı dosya yolu
208209
target_username: Kullanıcı adı
209210
output_dir: Özel output dizini (None ise default output kullanılır)
211+
scrape_type: Export metadata source type ("profile" or "bookmarks")
210212
211213
Returns:
212214
Dosya yolu
213215
"""
214-
data = build_export_payload(tweets, target_username)
216+
data = build_export_payload(tweets, target_username, scrape_type=scrape_type)
215217

216218
full_path = resolve_output_path(
217219
target_username, output_path, ".json", BASE_OUTPUT_DIR, output_dir

main.py

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -446,7 +446,10 @@ def main():
446446
if output_format == "json":
447447
print("JSON dosyası oluşturuluyor...")
448448
output_path = create_json_document(
449-
tweets, config["output_file"], config["target_username"]
449+
tweets,
450+
config["output_file"],
451+
config["target_username"],
452+
scrape_type=scrape_type,
450453
)
451454
elif output_format == "md":
452455
print("Markdown dosyası oluşturuluyor...")
@@ -515,7 +518,12 @@ def main():
515518

516519
if output_format == "json":
517520
output_file = f"{base_name}_PARTIAL.json"
518-
output_path = create_json_document(tweets, output_file, config["target_username"])
521+
output_path = create_json_document(
522+
tweets,
523+
output_file,
524+
config["target_username"],
525+
scrape_type=config.get("scrape_type", "profile"),
526+
)
519527
elif output_format == "md":
520528
output_file = f"{base_name}_PARTIAL.md"
521529
output_path = create_markdown_document(tweets, output_file, config["target_username"])
@@ -565,7 +573,12 @@ def main():
565573
try:
566574
if output_format == "json":
567575
output_file = f"{base_name}_ERROR.json"
568-
output_path = create_json_document(tweets, output_file, config["target_username"])
576+
output_path = create_json_document(
577+
tweets,
578+
output_file,
579+
config["target_username"],
580+
scrape_type=config.get("scrape_type", "profile"),
581+
)
569582
elif output_format == "md":
570583
output_file = f"{base_name}_ERROR.md"
571584
output_path = create_markdown_document(tweets, output_file, config["target_username"])

python_sidecar/service.py

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -323,9 +323,16 @@ def handle_export(self, cmd: dict):
323323
)
324324

325325
fmt = cmd.get("format", "json")
326-
target = cmd.get("target", "export")
326+
target = cmd.get("target") or (
327+
self.current_run_log.target if self.current_run_log else "export"
328+
)
327329
filename = cmd.get("filename", "")
328330
output_dir = cmd.get("output_dir", "")
331+
scrape_type = (
332+
self.current_run_log.scrape_type
333+
if self.current_run_log
334+
else cmd.get("scrape_type", "profile")
335+
)
329336

330337
self.emit(
331338
"log",
@@ -348,7 +355,11 @@ def handle_export(self, cmd: dict):
348355
)
349356
if fmt == "json":
350357
path = create_json_document(
351-
self.current_tweets, filename, target, output_dir
358+
self.current_tweets,
359+
filename,
360+
target,
361+
output_dir,
362+
scrape_type=scrape_type,
352363
)
353364
elif fmt == "md":
354365
path = create_markdown_document(

0 commit comments

Comments
 (0)