Skip to content

Commit adf64a0

Browse files
authored
Merge pull request #43 from ScrapeGraphAI/revert-v2-docs-pr39
Revert "feat: v2 documentation with versioned navigation and updated SDKs" (PR #39)
2 parents b978c1b + 697f89f commit adf64a0

103 files changed

Lines changed: 2862 additions & 4627 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +0,0 @@
1-
DS_Store

api-reference/endpoint/smartcrawler/start.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -229,7 +229,7 @@ sha256={your_webhook_secret}
229229

230230
To verify that a webhook request is authentic:
231231

232-
1. Retrieve your webhook secret from the [dashboard](https://scrapegraphai.com/dashboard)
232+
1. Retrieve your webhook secret from the [dashboard](https://dashboard.scrapegraphai.com)
233233
2. Compare the `X-Webhook-Signature` header value with `sha256={your_secret}`
234234

235235
<CodeGroup>
@@ -305,5 +305,5 @@ The webhook POST request contains the following JSON payload:
305305
| result | string | The crawl result data (null if failed) |
306306

307307
<Note>
308-
Make sure to configure your webhook secret in the [dashboard](https://scrapegraphai.com/dashboard) before using webhooks. Each user has a unique webhook secret for secure verification.
308+
Make sure to configure your webhook secret in the [dashboard](https://dashboard.scrapegraphai.com) before using webhooks. Each user has a unique webhook secret for secure verification.
309309
</Note>

api-reference/errors.mdx

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -139,17 +139,17 @@ except APIError as e:
139139
```
140140

141141
```javascript JavaScript
142-
import { extract } from 'scrapegraph-js';
142+
import { smartScraper } from 'scrapegraph-js';
143143

144-
const result = await extract('your-api-key', {
145-
url: 'https://example.com',
146-
prompt: 'Extract data',
144+
const apiKey = 'your-api-key';
145+
146+
const response = await smartScraper(apiKey, {
147+
website_url: 'https://example.com',
148+
user_prompt: 'Extract data',
147149
});
148150

149-
if (result.status === 'success') {
150-
console.log('Data:', result.data);
151-
} else {
152-
console.error('Error:', result.error);
151+
if (response.status === 'error') {
152+
console.error('Error:', response.error);
153153
}
154154
```
155155
</CodeGroup>

api-reference/introduction.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ The ScrapeGraphAI API provides powerful endpoints for AI-powered web scraping an
99

1010
## Authentication
1111

12-
All API requests require authentication using an API key. You can get your API key from the [dashboard](https://scrapegraphai.com/dashboard).
12+
All API requests require authentication using an API key. You can get your API key from the [dashboard](https://dashboard.scrapegraphai.com).
1313

1414
```bash
1515
SGAI-APIKEY: your-api-key-here

cookbook/examples/pagination.mdx

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -349,16 +349,21 @@ if __name__ == "__main__":
349349
## JavaScript SDK Example
350350

351351
```javascript
352-
import { extract } from 'scrapegraph-js';
352+
import { smartScraper } from 'scrapegraph-js';
353353
import 'dotenv/config';
354354

355-
const result = await extract(process.env.SGAI_APIKEY, {
356-
url: 'https://www.amazon.in/s?k=tv&crid=1TEF1ZFVLU8R8&sprefix=t%2Caps%2C390&ref=nb_sb_noss_2',
357-
prompt: 'Extract all product info including name, price, rating, and image_url',
355+
const apiKey = process.env.SGAI_APIKEY;
356+
357+
const response = await smartScraper(apiKey, {
358+
website_url: 'https://www.amazon.in/s?k=tv&crid=1TEF1ZFVLU8R8&sprefix=t%2Caps%2C390&ref=nb_sb_noss_2',
359+
user_prompt: 'Extract all product info including name, price, rating, and image_url',
360+
total_pages: 3,
358361
});
359362

360-
if (result.status === 'success') {
361-
console.log('Response:', JSON.stringify(result.data?.json, null, 2));
363+
if (response.status === 'error') {
364+
console.error('Error:', response.error);
365+
} else {
366+
console.log('Response:', JSON.stringify(response.data, null, 2));
362367
}
363368
```
364369

cookbook/introduction.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ Each example is available in multiple implementations:
8787
4. Experiment and adapt the code for your needs
8888

8989
<Note>
90-
Make sure to have your ScrapeGraphAI API key ready. Get one from the [dashboard](https://scrapegraphai.com/dashboard) if you haven't already.
90+
Make sure to have your ScrapeGraphAI API key ready. Get one from the [dashboard](https://dashboard.scrapegraphai.com) if you haven't already.
9191
</Note>
9292

9393
## Additional Resources

dashboard/overview.mdx

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,21 @@ The ScrapeGraphAI dashboard is your central hub for managing all your web scrapi
1919
- **Last Used**: Timestamp of your most recent API request
2020
- **Quick Actions**: Buttons to start new scraping jobs or access common features
2121

22+
## Usage Analytics
23+
24+
Track your API usage patterns with our detailed analytics view:
25+
26+
<Frame>
27+
<img src="/images/dashboard/dashboard-2.png" alt="Usage Analytics Graph" />
28+
</Frame>
29+
30+
The usage graph provides:
31+
- **Service-specific metrics**: Track usage for SmartScraper, SearchScraper, and Markdownify separately
32+
- **Time-based analysis**: View usage patterns over different time periods
33+
- **Interactive tooltips**: Hover over data points to see detailed information
34+
- **Trend analysis**: Identify usage patterns and optimize your API consumption
35+
36+
2237
## Key Features
2338

2439
- **Usage Statistics**: Monitor your API usage and remaining credits
@@ -28,7 +43,7 @@ The ScrapeGraphAI dashboard is your central hub for managing all your web scrapi
2843

2944
## Getting Started
3045

31-
1. Log in to your [dashboard](https://scrapegraphai.com/dashboard)
46+
1. Log in to your [dashboard](https://dashboard.scrapegraphai.com)
3247
2. View your API key in the settings section
3348
3. Check your available credits
3449
4. Start your first scraping job

developer-guides/llm-sdks-and-frameworks/anthropic.mdx

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -27,24 +27,24 @@ If using Node < 20, install `dotenv` and add `import 'dotenv/config'` to your co
2727
This example demonstrates a simple workflow: scrape a website and summarize the content using Claude.
2828

2929
```typescript
30-
import { extract } from 'scrapegraph-js';
30+
import { smartScraper } from 'scrapegraph-js';
3131
import Anthropic from '@anthropic-ai/sdk';
3232

33+
const apiKey = process.env.SGAI_APIKEY;
3334
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
3435

35-
const result = await extract(process.env.SGAI_APIKEY!, {
36-
url: 'https://scrapegraphai.com',
37-
prompt: 'Extract all content from this page',
36+
const scrapeResult = await smartScraper(apiKey, {
37+
website_url: 'https://scrapegraphai.com',
38+
user_prompt: 'Extract all content from this page',
3839
});
3940

40-
const data = result.data?.json;
41-
console.log('Scraped content length:', JSON.stringify(data).length);
41+
console.log('Scraped content length:', JSON.stringify(scrapeResult.data.result).length);
4242

4343
const message = await anthropic.messages.create({
4444
model: 'claude-haiku-4-5',
4545
max_tokens: 1024,
4646
messages: [
47-
{ role: 'user', content: `Summarize in 100 words: ${JSON.stringify(data)}` }
47+
{ role: 'user', content: `Summarize in 100 words: ${JSON.stringify(scrapeResult.data.result)}` }
4848
]
4949
});
5050

@@ -56,11 +56,12 @@ console.log('Response:', message);
5656
This example shows how to use Claude's tool use feature to let the model decide when to scrape websites based on user requests.
5757

5858
```typescript
59-
import { extract } from 'scrapegraph-js';
59+
import { smartScraper } from 'scrapegraph-js';
6060
import { Anthropic } from '@anthropic-ai/sdk';
6161
import { z } from 'zod';
6262
import { zodToJsonSchema } from 'zod-to-json-schema';
6363

64+
const apiKey = process.env.SGAI_APIKEY;
6465
const anthropic = new Anthropic({
6566
apiKey: process.env.ANTHROPIC_API_KEY
6667
});
@@ -90,13 +91,12 @@ if (toolUse && toolUse.type === 'tool_use') {
9091
const input = toolUse.input as { url: string };
9192
console.log(`Calling tool: ${toolUse.name} | URL: ${input.url}`);
9293

93-
const result = await extract(process.env.SGAI_APIKEY!, {
94-
url: input.url,
95-
prompt: 'Extract all content from this page',
94+
const result = await smartScraper(apiKey, {
95+
website_url: input.url,
96+
user_prompt: 'Extract all content from this page',
9697
});
9798

98-
const data = result.data?.json;
99-
console.log(`Scraped content preview: ${JSON.stringify(data)?.substring(0, 300)}...`);
99+
console.log(`Scraped content preview: ${JSON.stringify(result.data.result)?.substring(0, 300)}...`);
100100
// Continue with the conversation or process the scraped content as needed
101101
}
102102
```
@@ -106,10 +106,11 @@ if (toolUse && toolUse.type === 'tool_use') {
106106
This example demonstrates how to use Claude to extract structured data from scraped website content.
107107

108108
```typescript
109-
import { extract } from 'scrapegraph-js';
109+
import { smartScraper } from 'scrapegraph-js';
110110
import Anthropic from '@anthropic-ai/sdk';
111111
import { z } from 'zod';
112112

113+
const apiKey = process.env.SGAI_APIKEY;
113114
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
114115

115116
const CompanyInfoSchema = z.object({
@@ -118,11 +119,10 @@ const CompanyInfoSchema = z.object({
118119
description: z.string().optional()
119120
});
120121

121-
const result = await extract(process.env.SGAI_APIKEY!, {
122-
url: 'https://stripe.com',
123-
prompt: 'Extract all content from this page',
122+
const scrapeResult = await smartScraper(apiKey, {
123+
website_url: 'https://stripe.com',
124+
user_prompt: 'Extract all content from this page',
124125
});
125-
const data = result.data?.json;
126126

127127
const prompt = `Extract company information from this website content.
128128
@@ -135,7 +135,7 @@ Output ONLY valid JSON in this exact format (no markdown, no explanation):
135135
}
136136
137137
Website content:
138-
${JSON.stringify(data)}`;
138+
${JSON.stringify(scrapeResult.data.result)}`;
139139

140140
const message = await anthropic.messages.create({
141141
model: 'claude-haiku-4-5',

developer-guides/llm-sdks-and-frameworks/gemini.mdx

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -27,22 +27,22 @@ If using Node < 20, install `dotenv` and add `import 'dotenv/config'` to your co
2727
This example demonstrates a simple workflow: scrape a website and summarize the content using Gemini.
2828

2929
```typescript
30-
import { extract } from 'scrapegraph-js';
30+
import { smartScraper } from 'scrapegraph-js';
3131
import { GoogleGenAI } from '@google/genai';
3232

33+
const apiKey = process.env.SGAI_APIKEY;
3334
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });
3435

35-
const result = await extract(process.env.SGAI_APIKEY!, {
36-
url: 'https://scrapegraphai.com',
37-
prompt: 'Extract all content from this page',
36+
const scrapeResult = await smartScraper(apiKey, {
37+
website_url: 'https://scrapegraphai.com',
38+
user_prompt: 'Extract all content from this page',
3839
});
39-
const data = result.data?.json;
4040

41-
console.log('Scraped content length:', JSON.stringify(data).length);
41+
console.log('Scraped content length:', JSON.stringify(scrapeResult.data.result).length);
4242

4343
const response = await ai.models.generateContent({
4444
model: 'gemini-2.5-flash',
45-
contents: `Summarize: ${JSON.stringify(data)}`,
45+
contents: `Summarize: ${JSON.stringify(scrapeResult.data.result)}`,
4646
});
4747

4848
console.log('Summary:', response.text);
@@ -53,26 +53,26 @@ console.log('Summary:', response.text);
5353
This example shows how to analyze website content using Gemini's multi-turn conversation capabilities.
5454

5555
```typescript
56-
import { extract } from 'scrapegraph-js';
56+
import { smartScraper } from 'scrapegraph-js';
5757
import { GoogleGenAI } from '@google/genai';
5858

59+
const apiKey = process.env.SGAI_APIKEY;
5960
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });
6061

61-
const result = await extract(process.env.SGAI_APIKEY!, {
62-
url: 'https://news.ycombinator.com/',
63-
prompt: 'Extract all content from this page',
62+
const scrapeResult = await smartScraper(apiKey, {
63+
website_url: 'https://news.ycombinator.com/',
64+
user_prompt: 'Extract all content from this page',
6465
});
65-
const data = result.data?.json;
6666

67-
console.log('Scraped content length:', JSON.stringify(data).length);
67+
console.log('Scraped content length:', JSON.stringify(scrapeResult.data.result).length);
6868

6969
const chat = ai.chats.create({
7070
model: 'gemini-2.5-flash'
7171
});
7272

7373
// Ask for the top 3 stories on Hacker News
7474
const result1 = await chat.sendMessage({
75-
message: `Based on this website content from Hacker News, what are the top 3 stories right now?\n\n${JSON.stringify(data)}`
75+
message: `Based on this website content from Hacker News, what are the top 3 stories right now?\n\n${JSON.stringify(scrapeResult.data.result)}`
7676
});
7777
console.log('Top 3 Stories:', result1.text);
7878

@@ -88,22 +88,22 @@ console.log('4th and 5th Stories:', result2.text);
8888
This example demonstrates how to extract structured data using Gemini's JSON mode from scraped website content.
8989

9090
```typescript
91-
import { extract } from 'scrapegraph-js';
91+
import { smartScraper } from 'scrapegraph-js';
9292
import { GoogleGenAI, Type } from '@google/genai';
9393

94+
const apiKey = process.env.SGAI_APIKEY;
9495
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });
9596

96-
const result = await extract(process.env.SGAI_APIKEY!, {
97-
url: 'https://stripe.com',
98-
prompt: 'Extract all content from this page',
97+
const scrapeResult = await smartScraper(apiKey, {
98+
website_url: 'https://stripe.com',
99+
user_prompt: 'Extract all content from this page',
99100
});
100-
const data = result.data?.json;
101101

102-
console.log('Scraped content length:', JSON.stringify(data).length);
102+
console.log('Scraped content length:', JSON.stringify(scrapeResult.data.result).length);
103103

104104
const response = await ai.models.generateContent({
105105
model: 'gemini-2.5-flash',
106-
contents: `Extract company information: ${JSON.stringify(data)}`,
106+
contents: `Extract company information: ${JSON.stringify(scrapeResult.data.result)}`,
107107
config: {
108108
responseMimeType: 'application/json',
109109
responseSchema: {

0 commit comments

Comments
 (0)