Open
Conversation
Align the CLI with ScrapeGraphAI/scrapegraph-js#11 (v2 SDK migration): - Rename smart-scraper → extract, search-scraper → search - Remove commands dropped from the API: agentic-scraper, generate-schema, sitemap, validate - Add client factory (src/lib/client.ts) using the new scrapegraphai({ apiKey }) pattern - Update scrape command with --format flag (markdown, html, screenshot, branding) - Update crawl to use crawl.start/status polling lifecycle - Update history to use v2 service names and parameters - All commands now use try/catch (v2 throws on error) and self-timed elapsed BREAKING CHANGE: CLI commands have been renamed and removed to match the v2 API surface. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Aligns CLI with scrapegraph-js v2 SDK change (b570a57) that replaced stealth/render booleans with a unified fetch mode enum: auto, fast, js, direct+stealth, js+stealth. - All commands: --stealth boolean → --mode <mode> string - Pin SDK to commit b570a57 (includes fetch mode change) - Update README and SKILL.md with new flag syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Member
Author
CLI Validation with
|
| Command | Status | Notes |
|---|---|---|
credits |
✅ Pass | Returns balance correctly (remaining, used, plan) |
scrape |
✅ Pass | Returns markdown for https://example.com in ~1.1s |
extract |
✅ Pass | Extracts structured JSON with prompt — title & description correct |
markdownify |
✅ Pass | Converts page to markdown in ~430ms |
search |
✅ Pass | Returns search results with URLs and content in ~3.8s |
history |
✅ Pass | Lists request history, both interactive and --json mode work |
crawl (start) |
✅ Pass | Job starts and returns ID; status polling returns "Crawl not found" — server-side issue, not SDK/CLI |
Setup
# Built SDK from PR branch
git clone --branch feat/sdk-v2-migration https://github.com/ScrapeGraphAI/scrapegraph-js.git
cd scrapegraph-js && bun install && bun run build
# Symlinked into CLI
ln -s /tmp/scrapegraph-js node_modules/scrapegraph-js
# Ran CLI against local server
SGAI_API_URL=http://localhost:3002 SGAI_API_KEY=<key> bun run src/cli.ts <command>All v2 SDK methods work correctly through the CLI. The only issue found (crawl status polling) is server-side, not related to the SDK or CLI code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
smart-scraper→extract,search-scraper→searchagentic-scraper,generate-schema,sitemap,validatescrapegraphai({ apiKey })patternscrapewith--formatflag (markdown, html, screenshot, branding)crawlto usecrawl.start/crawl.statuspolling lifecycle--stealthboolean with--modefetch mode enum (auto,fast,js,direct+stealth,js+stealth)Breaking Changes
smart-scraperextractsearch-scrapersearchscrapescrape--formatflagcrawlcrawl--stealth--mode direct+stealthagentic-scrapergenerate-schemasitemapvalidateFetch Modes
autofastjsdirect+stealthjs+stealthTest plan
bun run dev -- extract https://example.com -p "Extract title" --jsonbun run dev -- search "test query" --jsonbun run dev -- scrape https://example.com --jsonbun run dev -- scrape https://example.com -f html --jsonbun run dev -- scrape https://example.com -m js --jsonbun run dev -- markdownify https://example.com --jsonbun run dev -- crawl https://example.com --max-pages 5 --jsonbun run dev -- credits --jsonbun run dev -- history extract --jsonbunx tsc --noEmitpassesbun run buildsucceeds🤖 Generated with Claude Code