44
55Command-line interface for [ ScrapeGraph AI] ( https://scrapegraphai.com ) — AI-powered web scraping, data extraction, search, and crawling.
66
7+ ## Installation
8+
9+ ### From npm (recommended)
10+
11+ Install globally to use ` just-scrape ` from anywhere:
12+
13+ ``` bash
14+ npm install -g just-scrape
15+ ```
16+
17+ Or use it directly without installing via ` npx ` :
18+
19+ ``` bash
20+ npx just-scrape --help
21+ ```
22+
23+ You can also install with other package managers:
24+
25+ ``` bash
26+ # pnpm
27+ pnpm add -g just-scrape
28+
29+ # yarn
30+ yarn global add just-scrape
31+
32+ # bun
33+ bun add -g just-scrape
34+ ```
35+
36+ Package: [ just-scrape] ( https://www.npmjs.com/package/just-scrape ) on npm.
37+
38+ ### From source (local development)
39+
40+ Requires [ Bun] ( https://bun.sh ) and Node.js 22+.
41+
42+ ``` bash
43+ # Clone the repository
44+ git clone https://github.com/ScrapeGraphAI/just-scrape.git
45+ cd just-scrape
46+
47+ # Install dependencies
48+ bun install
49+
50+ # Run directly from source (no build needed)
51+ bun run dev --help
52+
53+ # Or build and link globally
54+ bun run build
55+ npm link
56+ just-scrape --help
57+ ```
58+
59+ ### Verify installation
60+
61+ ``` bash
62+ just-scrape --help
63+ just-scrape validate # check your API key
64+ ```
65+
766## Tech Stack
867
968| Concern | Tool |
@@ -37,35 +96,31 @@ Four ways to provide it (checked in order):
37963 . ** Config file** : stored in ` ~/.scrapegraphai/config.json `
38974 . ** Interactive prompt** : if none of the above are set, the CLI prompts you and saves it to the config file
3998
40- ### Timeout
41-
42- Set ` SGAI_CLI_TIMEOUT_S ` to override the default 120s request/polling timeout:
43-
44- ``` bash
45- export SGAI_CLI_TIMEOUT_S=300
46- ```
99+ ### Environment Variables
47100
48- ### Debug Logging
49-
50- Set ` SGAI_CLI_DEBUG=1 ` to enable debug logging (outputs to stderr):
101+ | Variable | Default | Description |
102+ | ---| ---| ---|
103+ | ` JUST_SCRAPE_TIMEOUT_S ` | ` 120 ` | Request/polling timeout in seconds |
104+ | ` JUST_SCRAPE_DEBUG ` | ` 0 ` | Set to ` 1 ` to enable debug logging (outputs to stderr) |
51105
52106``` bash
53- SGAI_CLI_DEBUG=1 scrapegraphai smart-scraper https://example.com -p " Extract data"
107+ export JUST_SCRAPE_TIMEOUT_S=300
108+ JUST_SCRAPE_DEBUG=1 just-scrape smart-scraper https://example.com -p " Extract data"
54109```
55110
56111## Commands
57112
58113### ` smart-scraper ` — Extract structured data from a URL [ docs] ( https://docs.scrapegraphai.com/services/smartscraper )
59114
60115``` bash
61- scrapegraphai smart-scraper < url> -p " Extract all product names and prices"
116+ just-scrape smart-scraper < url> -p " Extract all product names and prices"
62117
63118# With JSON schema
64- scrapegraphai smart-scraper https://example.com/products -p " Extract products" \
119+ just-scrape smart-scraper https://example.com/products -p " Extract products" \
65120 --schema ' {"type":"object","properties":{"products":{"type":"array","items":{"type":"object","properties":{"name":{"type":"string"},"price":{"type":"number"}}}}}}'
66121
67122# With options
68- scrapegraphai smart-scraper https://example.com -p " Extract data" \
123+ just-scrape smart-scraper https://example.com -p " Extract data" \
69124 --stealth --render-js --scrolls 10 --pages 5
70125```
71126
@@ -84,10 +139,10 @@ scrapegraphai smart-scraper https://example.com -p "Extract data" \
84139### ` search-scraper ` — Search the web and extract data [ docs] ( https://docs.scrapegraphai.com/services/searchscraper )
85140
86141``` bash
87- scrapegraphai search-scraper " What are the top Python web frameworks?"
142+ just-scrape search-scraper " What are the top Python web frameworks?"
88143
89144# Markdown only (cheaper)
90- scrapegraphai search-scraper " Python frameworks" --no-extraction --num-results 5
145+ just-scrape search-scraper " Python frameworks" --no-extraction --num-results 5
91146```
92147
93148| Option | Description |
@@ -101,8 +156,8 @@ scrapegraphai search-scraper "Python frameworks" --no-extraction --num-results 5
101156### ` markdownify ` — Convert a webpage to markdown [ docs] ( https://docs.scrapegraphai.com/services/markdownify )
102157
103158``` bash
104- scrapegraphai markdownify https://example.com/article
105- scrapegraphai markdownify https://example.com --render-js --stealth
159+ just-scrape markdownify https://example.com/article
160+ just-scrape markdownify https://example.com --render-js --stealth
106161```
107162
108163| Option | Description |
@@ -114,13 +169,13 @@ scrapegraphai markdownify https://example.com --render-js --stealth
114169### ` crawl ` — Crawl and extract from multiple pages [ docs] ( https://docs.scrapegraphai.com/services/smartcrawler )
115170
116171``` bash
117- scrapegraphai crawl https://example.com -p " Extract article titles" --max-pages 5 --depth 2
172+ just-scrape crawl https://example.com -p " Extract article titles" --max-pages 5 --depth 2
118173
119174# Markdown only
120- scrapegraphai crawl https://example.com --no-extraction --max-pages 10
175+ just-scrape crawl https://example.com --no-extraction --max-pages 10
121176
122177# With crawl rules
123- scrapegraphai crawl https://example.com -p " Extract data" \
178+ just-scrape crawl https://example.com -p " Extract data" \
124179 --rules ' {"include_paths":["/blog/*"],"same_domain":true}'
125180```
126181
@@ -139,14 +194,14 @@ scrapegraphai crawl https://example.com -p "Extract data" \
139194### ` sitemap ` — Get all URLs from a website's sitemap [ docs] ( https://docs.scrapegraphai.com/services/sitemap )
140195
141196``` bash
142- scrapegraphai sitemap https://example.com
197+ just-scrape sitemap https://example.com
143198```
144199
145200### ` scrape ` — Get raw HTML content [ docs] ( https://docs.scrapegraphai.com/services/scrape )
146201
147202``` bash
148- scrapegraphai scrape https://example.com
149- scrapegraphai scrape https://example.com --stealth --branding --country-code US
203+ just-scrape scrape https://example.com
204+ just-scrape scrape https://example.com --stealth --branding --country-code US
150205```
151206
152207| Option | Description |
@@ -159,7 +214,7 @@ scrapegraphai scrape https://example.com --stealth --branding --country-code US
159214### ` agentic-scraper ` — Browser automation with AI [ docs] ( https://docs.scrapegraphai.com/services/agenticscraper )
160215
161216``` bash
162- scrapegraphai agentic-scraper https://example.com/login \
217+ just-scrape agentic-scraper https://example.com/login \
163218 -s " Fill email with user@test.com,Fill password with pass123,Click Sign In" \
164219 --ai-extraction -p " Extract dashboard data"
165220```
@@ -175,7 +230,7 @@ scrapegraphai agentic-scraper https://example.com/login \
175230### ` generate-schema ` — Generate JSON schema from a prompt
176231
177232``` bash
178- scrapegraphai generate-schema " Schema for an e-commerce product with name, price, and reviews"
233+ just-scrape generate-schema " Schema for an e-commerce product with name, price, and reviews"
179234```
180235
181236| Option | Description |
@@ -185,13 +240,13 @@ scrapegraphai generate-schema "Schema for an e-commerce product with name, price
185240### ` credits ` — Check credit balance
186241
187242``` bash
188- scrapegraphai credits
243+ just-scrape credits
189244```
190245
191246### ` validate ` — Validate your API key
192247
193248``` bash
194- scrapegraphai validate
249+ just-scrape validate
195250```
196251
197252## Testing
@@ -207,7 +262,7 @@ Covers all SDK functions: success paths, polling, HTTP error mapping (401/402/42
207262## Project Structure
208263
209264```
210- scrapegraph-cli /
265+ just-scrape /
211266├── src/
212267│ ├── cli.ts # Entry point, citty main command + subcommands
213268│ ├── lib/
@@ -259,12 +314,16 @@ All commands output pretty-printed JSON to stdout (pipeable). Errors go to stder
259314
260315``` bash
261316# Pipe output to jq
262- scrapegraphai credits | jq ' .remaining_credits'
317+ just-scrape credits | jq ' .remaining_credits'
263318
264319# Save to file
265- scrapegraphai smart-scraper https://example.com -p " Extract data" > result.json
320+ just-scrape smart-scraper https://example.com -p " Extract data" > result.json
266321```
267322
268323## License
269324
270325ISC
326+
327+ ---
328+
329+ Made with love by the [ ScrapeGraphAI] ( https://scrapegraphai.com ) team.
0 commit comments