Skip to content

Commit ca369f0

Browse files
committed
2 parents 0474c09 + 6aa662c commit ca369f0

2 files changed

Lines changed: 161 additions & 0 deletions

File tree

docs.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,7 @@
8989
{
9090
"group": "Integrations",
9191
"pages": [
92+
"integrations/claude-code-skill",
9293
"integrations/langchain",
9394
"integrations/llamaindex",
9495
"integrations/crewai",

integrations/claude-code-skill.mdx

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
---
2+
title: 'Claude Code Skill'
3+
description: 'Give Claude Code and AI coding agents the power of web scraping'
4+
icon: '/logo/claude-color.svg'
5+
---
6+
7+
## Overview
8+
9+
The ScrapeGraphAI [Claude Code Skill](https://github.com/ScrapeGraphAI/skill) gives AI coding agents full access to ScrapeGraphAI's web scraping, search, and crawling APIs. Once installed, agents like Claude Code, Cursor, Copilot, and Cline can scrape websites, extract structured data, and crawl pages — all from natural language prompts.
10+
11+
<Card
12+
title="GitHub Repository"
13+
icon="github"
14+
href="https://github.com/ScrapeGraphAI/skill"
15+
>
16+
View the skill source code and documentation
17+
</Card>
18+
19+
## Installation
20+
21+
<Steps>
22+
23+
### Option 1: Install via skills.sh (Recommended)
24+
25+
The fastest way to install. Requires [Node.js](https://nodejs.org).
26+
27+
```bash
28+
npx skills add ScrapeGraphAI/skill
29+
```
30+
31+
This clones the skill and symlinks it into your `~/.claude/skills/` directory automatically.
32+
33+
### Option 2: Manual install
34+
35+
Clone the repository and create the symlink yourself:
36+
37+
```bash
38+
git clone https://github.com/ScrapeGraphAI/skill.git ~/.claude/skills/scrapegraphai
39+
ln -sf ~/.claude/skills/scrapegraphai/SKILL.md ~/.claude/skills/scrapegraphai.md
40+
```
41+
42+
### Option 3: Project-level install
43+
44+
Install the skill for a single project only:
45+
46+
```bash
47+
mkdir -p .claude/skills
48+
git clone https://github.com/ScrapeGraphAI/skill.git .claude/skills/scrapegraphai
49+
ln -sf .claude/skills/scrapegraphai/SKILL.md .claude/skills/scrapegraphai.md
50+
```
51+
52+
</Steps>
53+
54+
## Setup
55+
56+
Set your ScrapeGraphAI API key as an environment variable:
57+
58+
```bash
59+
export SGAI_API_KEY="sgai-..."
60+
```
61+
62+
<Note>
63+
Get your API key from the [dashboard](https://dashboard.scrapegraphai.com).
64+
</Note>
65+
66+
## What's Included
67+
68+
The skill installs the following files:
69+
70+
| File | Description |
71+
|------|-------------|
72+
| `SKILL.md` | Main skill file with API reference, examples, and decision guide |
73+
| `references/api-endpoints.md` | Full parameter tables for all endpoints |
74+
| `references/sdk-examples.md` | Python and JavaScript SDK examples |
75+
| `references/advanced-features.md` | Stealth mode, schemas, scrolling, pagination, and more |
76+
77+
## Capabilities
78+
79+
<CardGroup cols={2}>
80+
<Card title="SmartScraper" icon="wand-magic-sparkles">
81+
Extract structured data from any webpage using natural language prompts
82+
</Card>
83+
<Card title="SearchScraper" icon="magnifying-glass">
84+
Search the web and extract results with AI or as markdown
85+
</Card>
86+
<Card title="Markdownify" icon="file-code">
87+
Convert any webpage into clean, formatted markdown
88+
</Card>
89+
<Card title="SmartCrawler" icon="spider">
90+
Crawl multiple pages from a website with depth and path controls
91+
</Card>
92+
<Card title="Sitemap" icon="sitemap">
93+
Extract all URLs from a website's sitemap
94+
</Card>
95+
<Card title="Agentic Scraper" icon="robot">
96+
Browser automation — login, click, navigate, fill forms, then extract
97+
</Card>
98+
</CardGroup>
99+
100+
## Example Prompts
101+
102+
Once the skill is installed, you can use natural language prompts directly in your AI coding agent:
103+
104+
```text
105+
Scrape https://example.com/products and extract all product names, prices, and ratings
106+
```
107+
108+
```text
109+
Search the web for "best React UI libraries 2025" and extract the top 5 with pros and cons
110+
```
111+
112+
```text
113+
Convert https://example.com/docs/getting-started to markdown and save it to docs/
114+
```
115+
116+
```text
117+
Crawl https://example.com/blog with depth 2 and extract the title and summary from each post
118+
```
119+
120+
```text
121+
Get all URLs from the sitemap of https://example.com
122+
```
123+
124+
```text
125+
Log into https://example.com/dashboard, click "Reports", and extract the table data
126+
```
127+
128+
The agent will automatically select the right ScrapeGraphAI endpoint, handle authentication, poll for async results, and return structured data.
129+
130+
## Supported Agents
131+
132+
This skill works with any AI coding agent that supports the skills protocol:
133+
134+
- **Claude Code** — Anthropic's CLI agent
135+
- **Cursor** — AI-powered code editor
136+
- **GitHub Copilot** — AI pair programmer
137+
- **Cline** — VS Code AI agent
138+
- **Windsurf** — AI coding assistant
139+
- And any other agent supporting [skills.sh](https://skills.sh)
140+
141+
## Support
142+
143+
Need help with the skill?
144+
145+
<CardGroup cols={2}>
146+
<Card
147+
title="GitHub Issues"
148+
icon="github"
149+
href="https://github.com/ScrapeGraphAI/skill/issues"
150+
>
151+
Report bugs and request features
152+
</Card>
153+
<Card
154+
title="Discord Community"
155+
icon="discord"
156+
href="https://discord.gg/uJN7TYcpNa"
157+
>
158+
Get help from our community
159+
</Card>
160+
</CardGroup>

0 commit comments

Comments
 (0)