Skip to content

feat: v2 documentation with versioned navigation and updated SDKs #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
VinciGit00 wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat: v2 documentation with versioned navigation and updated SDKs #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e3f3927
feat: add v2 documentation with versioned navigation and updated SDKs
VinciGit00 Mar 31, 2026
0289ac3
feat: update docs for v2 API - new services, langchain integration, a…
VinciGit00 Apr 1, 2026
d480ac8
feat: update CLI docs for v2 API (just-scrape PR #13)
VinciGit00 Apr 1, 2026
5b7297e
feat: update v2 nav and integrations
VinciGit00 Apr 1, 2026
3e564ec
feat: align v2 docs with Python SDK PR #82 and JS SDK PR #11
VinciGit00 Apr 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions api-reference/errors.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,17 +139,17 @@ except APIError as e:
```

```javascript JavaScript
import { smartScraper } from 'scrapegraph-js';
import { scrapegraphai } from 'scrapegraph-js';

const apiKey = 'your-api-key';
const sgai = scrapegraphai({ apiKey: 'your-api-key' });

const response = await smartScraper(apiKey, {
website_url: 'https://example.com',
user_prompt: 'Extract data',
});

if (response.status === 'error') {
console.error('Error:', response.error);
try {
const { data } = await sgai.extract('https://example.com', {
prompt: 'Extract data',
});
console.log('Data:', data);
} catch (error) {
console.error('Error:', error.message);
}
```
</CodeGroup>
Expand Down
22 changes: 10 additions & 12 deletions cookbook/examples/pagination.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,22 +349,20 @@ if __name__ == "__main__":
## JavaScript SDK Example

```javascript
import { smartScraper } from 'scrapegraph-js';
import { scrapegraphai } from 'scrapegraph-js';
import 'dotenv/config';

const apiKey = process.env.SGAI_APIKEY;
const sgai = scrapegraphai({ apiKey: process.env.SGAI_APIKEY });

const response = await smartScraper(apiKey, {
website_url: 'https://www.amazon.in/s?k=tv&crid=1TEF1ZFVLU8R8&sprefix=t%2Caps%2C390&ref=nb_sb_noss_2',
user_prompt: 'Extract all product info including name, price, rating, and image_url',
total_pages: 3,
});
const { data } = await sgai.extract(
'https://www.amazon.in/s?k=tv&crid=1TEF1ZFVLU8R8&sprefix=t%2Caps%2C390&ref=nb_sb_noss_2',
{
prompt: 'Extract all product info including name, price, rating, and image_url',
totalPages: 3,
}
);

if (response.status === 'error') {
console.error('Error:', response.error);
} else {
console.log('Response:', JSON.stringify(response.data, null, 2));
}
console.log('Response:', JSON.stringify(data, null, 2));
```

## Example Output
Expand Down
35 changes: 16 additions & 19 deletions developer-guides/llm-sdks-and-frameworks/anthropic.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,24 +27,23 @@ If using Node < 20, install `dotenv` and add `import 'dotenv/config'` to your co
This example demonstrates a simple workflow: scrape a website and summarize the content using Claude.

```typescript
import { smartScraper } from 'scrapegraph-js';
import { scrapegraphai } from 'scrapegraph-js';
import Anthropic from '@anthropic-ai/sdk';

const apiKey = process.env.SGAI_APIKEY;
const sgai = scrapegraphai({ apiKey: process.env.SGAI_APIKEY });
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

const scrapeResult = await smartScraper(apiKey, {
website_url: 'https://scrapegraphai.com',
user_prompt: 'Extract all content from this page',
const { data } = await sgai.extract('https://scrapegraphai.com', {
prompt: 'Extract all content from this page',
});

console.log('Scraped content length:', JSON.stringify(scrapeResult.data.result).length);
console.log('Scraped content length:', JSON.stringify(data).length);

const message = await anthropic.messages.create({
model: 'claude-haiku-4-5',
max_tokens: 1024,
messages: [
{ role: 'user', content: `Summarize in 100 words: ${JSON.stringify(scrapeResult.data.result)}` }
{ role: 'user', content: `Summarize in 100 words: ${JSON.stringify(data)}` }
]
});

Expand All @@ -56,12 +55,12 @@ console.log('Response:', message);
This example shows how to use Claude's tool use feature to let the model decide when to scrape websites based on user requests.

```typescript
import { smartScraper } from 'scrapegraph-js';
import { scrapegraphai } from 'scrapegraph-js';
import { Anthropic } from '@anthropic-ai/sdk';
import { z } from 'zod';
import { zodToJsonSchema } from 'zod-to-json-schema';

const apiKey = process.env.SGAI_APIKEY;
const sgai = scrapegraphai({ apiKey: process.env.SGAI_APIKEY });
const anthropic = new Anthropic({
apiKey: process.env.ANTHROPIC_API_KEY
});
Expand Down Expand Up @@ -91,12 +90,11 @@ if (toolUse && toolUse.type === 'tool_use') {
const input = toolUse.input as { url: string };
console.log(`Calling tool: ${toolUse.name} | URL: ${input.url}`);

const result = await smartScraper(apiKey, {
website_url: input.url,
user_prompt: 'Extract all content from this page',
const { data } = await sgai.extract(input.url, {
prompt: 'Extract all content from this page',
});

console.log(`Scraped content preview: ${JSON.stringify(result.data.result)?.substring(0, 300)}...`);
console.log(`Scraped content preview: ${JSON.stringify(data)?.substring(0, 300)}...`);
// Continue with the conversation or process the scraped content as needed
}
```
Expand All @@ -106,11 +104,11 @@ if (toolUse && toolUse.type === 'tool_use') {
This example demonstrates how to use Claude to extract structured data from scraped website content.

```typescript
import { smartScraper } from 'scrapegraph-js';
import { scrapegraphai } from 'scrapegraph-js';
import Anthropic from '@anthropic-ai/sdk';
import { z } from 'zod';

const apiKey = process.env.SGAI_APIKEY;
const sgai = scrapegraphai({ apiKey: process.env.SGAI_APIKEY });
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

const CompanyInfoSchema = z.object({
Expand All @@ -119,9 +117,8 @@ const CompanyInfoSchema = z.object({
description: z.string().optional()
});

const scrapeResult = await smartScraper(apiKey, {
website_url: 'https://stripe.com',
user_prompt: 'Extract all content from this page',
const { data } = await sgai.extract('https://stripe.com', {
prompt: 'Extract all content from this page',
});

const prompt = `Extract company information from this website content.
Expand All @@ -135,7 +132,7 @@ Output ONLY valid JSON in this exact format (no markdown, no explanation):
}

Website content:
${JSON.stringify(scrapeResult.data.result)}`;
${JSON.stringify(data)}`;

const message = await anthropic.messages.create({
model: 'claude-haiku-4-5',
Expand Down
39 changes: 18 additions & 21 deletions developer-guides/llm-sdks-and-frameworks/gemini.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,22 +27,21 @@ If using Node < 20, install `dotenv` and add `import 'dotenv/config'` to your co
This example demonstrates a simple workflow: scrape a website and summarize the content using Gemini.

```typescript
import { smartScraper } from 'scrapegraph-js';
import { scrapegraphai } from 'scrapegraph-js';
import { GoogleGenAI } from '@google/genai';

const apiKey = process.env.SGAI_APIKEY;
const sgai = scrapegraphai({ apiKey: process.env.SGAI_APIKEY });
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const scrapeResult = await smartScraper(apiKey, {
website_url: 'https://scrapegraphai.com',
user_prompt: 'Extract all content from this page',
const { data } = await sgai.extract('https://scrapegraphai.com', {
prompt: 'Extract all content from this page',
});

console.log('Scraped content length:', JSON.stringify(scrapeResult.data.result).length);
console.log('Scraped content length:', JSON.stringify(data).length);

const response = await ai.models.generateContent({
model: 'gemini-2.5-flash',
contents: `Summarize: ${JSON.stringify(scrapeResult.data.result)}`,
contents: `Summarize: ${JSON.stringify(data)}`,
});

console.log('Summary:', response.text);
Expand All @@ -53,26 +52,25 @@ console.log('Summary:', response.text);
This example shows how to analyze website content using Gemini's multi-turn conversation capabilities.

```typescript
import { smartScraper } from 'scrapegraph-js';
import { scrapegraphai } from 'scrapegraph-js';
import { GoogleGenAI } from '@google/genai';

const apiKey = process.env.SGAI_APIKEY;
const sgai = scrapegraphai({ apiKey: process.env.SGAI_APIKEY });
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const scrapeResult = await smartScraper(apiKey, {
website_url: 'https://news.ycombinator.com/',
user_prompt: 'Extract all content from this page',
const { data } = await sgai.extract('https://news.ycombinator.com/', {
prompt: 'Extract all content from this page',
});

console.log('Scraped content length:', JSON.stringify(scrapeResult.data.result).length);
console.log('Scraped content length:', JSON.stringify(data).length);

const chat = ai.chats.create({
model: 'gemini-2.5-flash'
});

// Ask for the top 3 stories on Hacker News
const result1 = await chat.sendMessage({
message: `Based on this website content from Hacker News, what are the top 3 stories right now?\n\n${JSON.stringify(scrapeResult.data.result)}`
message: `Based on this website content from Hacker News, what are the top 3 stories right now?\n\n${JSON.stringify(data)}`
});
console.log('Top 3 Stories:', result1.text);

Expand All @@ -88,22 +86,21 @@ console.log('4th and 5th Stories:', result2.text);
This example demonstrates how to extract structured data using Gemini's JSON mode from scraped website content.

```typescript
import { smartScraper } from 'scrapegraph-js';
import { scrapegraphai } from 'scrapegraph-js';
import { GoogleGenAI, Type } from '@google/genai';

const apiKey = process.env.SGAI_APIKEY;
const sgai = scrapegraphai({ apiKey: process.env.SGAI_APIKEY });
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const scrapeResult = await smartScraper(apiKey, {
website_url: 'https://stripe.com',
user_prompt: 'Extract all content from this page',
const { data } = await sgai.extract('https://stripe.com', {
prompt: 'Extract all content from this page',
});

console.log('Scraped content length:', JSON.stringify(scrapeResult.data.result).length);
console.log('Scraped content length:', JSON.stringify(data).length);

const response = await ai.models.generateContent({
model: 'gemini-2.5-flash',
contents: `Extract company information: ${JSON.stringify(scrapeResult.data.result)}`,
contents: `Extract company information: ${JSON.stringify(data)}`,
config: {
responseMimeType: 'application/json',
responseSchema: {
Expand Down
Loading