Skip to content

Commit 25ea5fe

Browse files
Merge pull request #2 from ScrapeGraphAI/rebranding-branch
feat: rebranding and improved installation docs
2 parents a666f6e + d96f509 commit 25ea5fe

8 files changed

Lines changed: 107 additions & 44 deletions

File tree

README.md

Lines changed: 90 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,65 @@
44

55
Command-line interface for [ScrapeGraph AI](https://scrapegraphai.com) — AI-powered web scraping, data extraction, search, and crawling.
66

7+
## Installation
8+
9+
### From npm (recommended)
10+
11+
Install globally to use `just-scrape` from anywhere:
12+
13+
```bash
14+
npm install -g just-scrape
15+
```
16+
17+
Or use it directly without installing via `npx`:
18+
19+
```bash
20+
npx just-scrape --help
21+
```
22+
23+
You can also install with other package managers:
24+
25+
```bash
26+
# pnpm
27+
pnpm add -g just-scrape
28+
29+
# yarn
30+
yarn global add just-scrape
31+
32+
# bun
33+
bun add -g just-scrape
34+
```
35+
36+
Package: [just-scrape](https://www.npmjs.com/package/just-scrape) on npm.
37+
38+
### From source (local development)
39+
40+
Requires [Bun](https://bun.sh) and Node.js 22+.
41+
42+
```bash
43+
# Clone the repository
44+
git clone https://github.com/ScrapeGraphAI/just-scrape.git
45+
cd just-scrape
46+
47+
# Install dependencies
48+
bun install
49+
50+
# Run directly from source (no build needed)
51+
bun run dev --help
52+
53+
# Or build and link globally
54+
bun run build
55+
npm link
56+
just-scrape --help
57+
```
58+
59+
### Verify installation
60+
61+
```bash
62+
just-scrape --help
63+
just-scrape validate # check your API key
64+
```
65+
766
## Tech Stack
867

968
| Concern | Tool |
@@ -37,35 +96,31 @@ Four ways to provide it (checked in order):
3796
3. **Config file**: stored in `~/.scrapegraphai/config.json`
3897
4. **Interactive prompt**: if none of the above are set, the CLI prompts you and saves it to the config file
3998

40-
### Timeout
41-
42-
Set `SGAI_CLI_TIMEOUT_S` to override the default 120s request/polling timeout:
43-
44-
```bash
45-
export SGAI_CLI_TIMEOUT_S=300
46-
```
99+
### Environment Variables
47100

48-
### Debug Logging
49-
50-
Set `SGAI_CLI_DEBUG=1` to enable debug logging (outputs to stderr):
101+
| Variable | Default | Description |
102+
|---|---|---|
103+
| `JUST_SCRAPE_TIMEOUT_S` | `120` | Request/polling timeout in seconds |
104+
| `JUST_SCRAPE_DEBUG` | `0` | Set to `1` to enable debug logging (outputs to stderr) |
51105

52106
```bash
53-
SGAI_CLI_DEBUG=1 scrapegraphai smart-scraper https://example.com -p "Extract data"
107+
export JUST_SCRAPE_TIMEOUT_S=300
108+
JUST_SCRAPE_DEBUG=1 just-scrape smart-scraper https://example.com -p "Extract data"
54109
```
55110

56111
## Commands
57112

58113
### `smart-scraper` — Extract structured data from a URL [docs](https://docs.scrapegraphai.com/services/smartscraper)
59114

60115
```bash
61-
scrapegraphai smart-scraper <url> -p "Extract all product names and prices"
116+
just-scrape smart-scraper <url> -p "Extract all product names and prices"
62117

63118
# With JSON schema
64-
scrapegraphai smart-scraper https://example.com/products -p "Extract products" \
119+
just-scrape smart-scraper https://example.com/products -p "Extract products" \
65120
--schema '{"type":"object","properties":{"products":{"type":"array","items":{"type":"object","properties":{"name":{"type":"string"},"price":{"type":"number"}}}}}}'
66121

67122
# With options
68-
scrapegraphai smart-scraper https://example.com -p "Extract data" \
123+
just-scrape smart-scraper https://example.com -p "Extract data" \
69124
--stealth --render-js --scrolls 10 --pages 5
70125
```
71126

@@ -84,10 +139,10 @@ scrapegraphai smart-scraper https://example.com -p "Extract data" \
84139
### `search-scraper` — Search the web and extract data [docs](https://docs.scrapegraphai.com/services/searchscraper)
85140

86141
```bash
87-
scrapegraphai search-scraper "What are the top Python web frameworks?"
142+
just-scrape search-scraper "What are the top Python web frameworks?"
88143

89144
# Markdown only (cheaper)
90-
scrapegraphai search-scraper "Python frameworks" --no-extraction --num-results 5
145+
just-scrape search-scraper "Python frameworks" --no-extraction --num-results 5
91146
```
92147

93148
| Option | Description |
@@ -101,8 +156,8 @@ scrapegraphai search-scraper "Python frameworks" --no-extraction --num-results 5
101156
### `markdownify` — Convert a webpage to markdown [docs](https://docs.scrapegraphai.com/services/markdownify)
102157

103158
```bash
104-
scrapegraphai markdownify https://example.com/article
105-
scrapegraphai markdownify https://example.com --render-js --stealth
159+
just-scrape markdownify https://example.com/article
160+
just-scrape markdownify https://example.com --render-js --stealth
106161
```
107162

108163
| Option | Description |
@@ -114,13 +169,13 @@ scrapegraphai markdownify https://example.com --render-js --stealth
114169
### `crawl` — Crawl and extract from multiple pages [docs](https://docs.scrapegraphai.com/services/smartcrawler)
115170

116171
```bash
117-
scrapegraphai crawl https://example.com -p "Extract article titles" --max-pages 5 --depth 2
172+
just-scrape crawl https://example.com -p "Extract article titles" --max-pages 5 --depth 2
118173

119174
# Markdown only
120-
scrapegraphai crawl https://example.com --no-extraction --max-pages 10
175+
just-scrape crawl https://example.com --no-extraction --max-pages 10
121176

122177
# With crawl rules
123-
scrapegraphai crawl https://example.com -p "Extract data" \
178+
just-scrape crawl https://example.com -p "Extract data" \
124179
--rules '{"include_paths":["/blog/*"],"same_domain":true}'
125180
```
126181

@@ -139,14 +194,14 @@ scrapegraphai crawl https://example.com -p "Extract data" \
139194
### `sitemap` — Get all URLs from a website's sitemap [docs](https://docs.scrapegraphai.com/services/sitemap)
140195

141196
```bash
142-
scrapegraphai sitemap https://example.com
197+
just-scrape sitemap https://example.com
143198
```
144199

145200
### `scrape` — Get raw HTML content [docs](https://docs.scrapegraphai.com/services/scrape)
146201

147202
```bash
148-
scrapegraphai scrape https://example.com
149-
scrapegraphai scrape https://example.com --stealth --branding --country-code US
203+
just-scrape scrape https://example.com
204+
just-scrape scrape https://example.com --stealth --branding --country-code US
150205
```
151206

152207
| Option | Description |
@@ -159,7 +214,7 @@ scrapegraphai scrape https://example.com --stealth --branding --country-code US
159214
### `agentic-scraper` — Browser automation with AI [docs](https://docs.scrapegraphai.com/services/agenticscraper)
160215

161216
```bash
162-
scrapegraphai agentic-scraper https://example.com/login \
217+
just-scrape agentic-scraper https://example.com/login \
163218
-s "Fill email with user@test.com,Fill password with pass123,Click Sign In" \
164219
--ai-extraction -p "Extract dashboard data"
165220
```
@@ -175,7 +230,7 @@ scrapegraphai agentic-scraper https://example.com/login \
175230
### `generate-schema` — Generate JSON schema from a prompt
176231

177232
```bash
178-
scrapegraphai generate-schema "Schema for an e-commerce product with name, price, and reviews"
233+
just-scrape generate-schema "Schema for an e-commerce product with name, price, and reviews"
179234
```
180235

181236
| Option | Description |
@@ -185,13 +240,13 @@ scrapegraphai generate-schema "Schema for an e-commerce product with name, price
185240
### `credits` — Check credit balance
186241

187242
```bash
188-
scrapegraphai credits
243+
just-scrape credits
189244
```
190245

191246
### `validate` — Validate your API key
192247

193248
```bash
194-
scrapegraphai validate
249+
just-scrape validate
195250
```
196251

197252
## Testing
@@ -207,7 +262,7 @@ Covers all SDK functions: success paths, polling, HTTP error mapping (401/402/42
207262
## Project Structure
208263

209264
```
210-
scrapegraph-cli/
265+
just-scrape/
211266
├── src/
212267
│ ├── cli.ts # Entry point, citty main command + subcommands
213268
│ ├── lib/
@@ -259,12 +314,16 @@ All commands output pretty-printed JSON to stdout (pipeable). Errors go to stder
259314

260315
```bash
261316
# Pipe output to jq
262-
scrapegraphai credits | jq '.remaining_credits'
317+
just-scrape credits | jq '.remaining_credits'
263318

264319
# Save to file
265-
scrapegraphai smart-scraper https://example.com -p "Extract data" > result.json
320+
just-scrape smart-scraper https://example.com -p "Extract data" > result.json
266321
```
267322

268323
## License
269324

270325
ISC
326+
327+
---
328+
329+
Made with love by the [ScrapeGraphAI](https://scrapegraphai.com) team.

assets/demo.gif

-159 KB
Loading

assets/demo.mp4

627 KB
Binary file not shown.

bun.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

package.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
{
22
"name": "just-scrape",
3-
"version": "0.1.0",
3+
"version": "0.1.1",
44
"description": "ScrapeGraph AI CLI tool",
55
"type": "module",
66
"main": "dist/cli.mjs",
77
"bin": {
8-
"scrapegraphai": "dist/cli.mjs"
8+
"just-scrape": "dist/cli.mjs"
99
},
1010
"scripts": {
1111
"dev": "bun run src/cli.ts",

src/cli.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ showBanner();
66

77
const main = defineCommand({
88
meta: {
9-
name: "scrapegraphai",
9+
name: "just-scrape",
1010
version: getVersion(),
1111
description: "ScrapeGraph AI CLI tool",
1212
},

src/lib/env.ts

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,10 @@ function resolve(): Env {
2828

2929
return EnvSchema.parse({
3030
apiKey: process.env.SGAI_API_KEY || (config["api-key"] as string) || undefined,
31-
debug: process.env.SGAI_CLI_DEBUG === "1",
32-
timeoutS: process.env.SGAI_CLI_TIMEOUT_S ? Number(process.env.SGAI_CLI_TIMEOUT_S) : undefined,
31+
debug: process.env.JUST_SCRAPE_DEBUG === "1",
32+
timeoutS: process.env.JUST_SCRAPE_TIMEOUT_S
33+
? Number(process.env.JUST_SCRAPE_TIMEOUT_S)
34+
: undefined,
3335
});
3436
}
3537

src/utils/banner.ts

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,19 +14,20 @@ export function getVersion(): string {
1414
}
1515

1616
const BANNER = [
17-
"╔═╗╔═╗╦═╗╔═╗╔═╗╔═╗╔═╗╦═╗╔═╗╔═╗╦ ╦╔═╗╦",
18-
"╚═╗║ ╠╦╝╠═╣╠═╝║╣ ║ ╦╠╦╝╠═╣╠═╝╠═╣╠═╣║",
19-
"╚═╝╚═╝╩╚═╩ ╩╩ ╚═╝╚═╝╩╚═╩ ╩╩ ╩ ╩╩ ╩╩",
20-
].join("\n");
17+
"╔═╗╦ ╦╔═╗╔╦╗ ╔═╗╔═╗╦═╗╔═╗╔═╗╔═╗",
18+
" ║║ ║╚═╗ ║ ═══╚═╗║ ╠╦╝╠═╣╠═╝║╣ ",
19+
"╚═╝╚═╝╚═╝ ╩ ╚═╝╚═╝╩╚═╩ ╩╩ ╚═╝",
20+
];
21+
22+
const TAGLINE = " made with ♥ from scrapegraphai team";
2123

2224
const BANNER_COLOR = "#bd93f9";
2325

2426
export function showBanner() {
25-
const colored = BANNER.split("\n")
26-
.map((line) => chalk.hex(BANNER_COLOR)(line))
27-
.join("\n");
27+
const text = BANNER.map((line) => chalk.hex(BANNER_COLOR)(line)).join("\n");
2828

29-
console.log(colored);
29+
console.log(text);
30+
console.log(chalk.hex(BANNER_COLOR)(TAGLINE));
3031
console.log(chalk.hex(BANNER_COLOR)(`v${getVersion()}`));
3132
console.log();
3233
}

0 commit comments

Comments
 (0)