Skip to content

Commit b439958

Browse files
domantas-jurkus-dcddomantas-jurkus-dcd
authored
Update readme (#50)
1 parent 52b75c5 commit b439958

1 file changed

Lines changed: 54 additions & 29 deletions

File tree

‎README.md‎

Lines changed: 54 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -6,26 +6,32 @@
66
<p align="center">
77
<a href="https://dashboard.decodo.com/integrations?utm_source=github&utm_medium=social&utm_campaign=mcp_server"> <img src="https://github.com/user-attachments/assets/a1e52a9e-3da1-4081-b3c6-053aafb8f196"/></a>
88

9-
Connect LLMs and AI agents to live web data using MCP (Model Context Protocol). The Decodo MCP Server lets you scrape websites, search engines, eCommerce platforms, and social media directly from AI tools like Claude, Cursor, and Windsurf, all without the need to build scraping infrastructure from scratch.
9+
Connect LLMs and AI agents to live web data using MCP (Model Context Protocol). The Decodo MCP
10+
Server lets you scrape websites, search engines, eCommerce platforms, and social media directly from
11+
AI tools like Claude, Cursor, and Windsurf, all without the need to build scraping infrastructure
12+
from scratch.
1013

1114
- Structured outputs in JSON, Markdown, and screenshots
1215
- Server-side JavaScript rendering and anti-bot handling
1316
- 125M+ IPs across 195+ locations
1417

15-
## What is Decodo MCP server?
18+
## What is Decodo MCP server?
1619

17-
The Decodo MCP Server is a web scraping layer for AI agents. It connects MCP-compatible clients to Decodo's Web Scraping API, enabling:
20+
The Decodo MCP Server is a web scraping layer for AI agents. It connects MCP-compatible clients to
21+
Decodo's Web Scraping API, enabling:
1822

1923
- Web scraping for LLMs
2024
- Real-time data retrieval for RAG
2125
- AI agent browsing and research
2226
- Structured data extraction from dynamic websites
2327

24-
Instead of maintaining proxies, parsers, and retry logic, you get a single integration point for reliable web data access.
28+
Instead of maintaining proxies, parsers, and retry logic, you get a single integration point for
29+
reliable web data access.
2530

2631
## Why use MCP for web scraping?
2732

28-
Model Context Protocol (MCP) is the emerging standard for connecting AI agents to external tools and data sources. With MCP:
33+
Model Context Protocol (MCP) is the emerging standard for connecting AI agents to external tools and
34+
data sources. With MCP:
2935

3036
- Agents can call tools dynamically
3137
- Integrations stay standardized
@@ -35,34 +41,49 @@ The Decodo MCP Server gives your agents reliable, production-ready web access th
3541

3642
## Key features
3743

38-
**Web scraping for AI agents, no infrastructure required.** Scrape any website, including JavaScript-heavy pages, without handling proxy rotation, CAPTCHA solving, or anti-bot systems.
44+
**Web scraping for AI agents, no infrastructure required.** Scrape any website, including
45+
JavaScript-heavy pages, without handling proxy rotation, CAPTCHA solving, or anti-bot systems.
3946

40-
**Structured outputs for LLM workflows.** Markdown (LLM-ready), JSON (for structured pipelines), and screenshots (for visual context), built for RAG pipelines, AI research agents, and automation flows.
47+
**Structured outputs for LLM workflows.** Markdown (LLM-ready), JSON (for structured pipelines), and
48+
screenshots (for visual context), built for RAG pipelines, AI research agents, and automation flows.
4149

42-
**Built-in support for popular targets.** Ready-made tools for Google and Bing (SERPs), Amazon, Walmart, and Target (eCommerce), Reddit, TikTok, and YouTube (social media), and ChatGPT and Perplexity (AI search).
50+
**Built-in support for popular targets.** Ready-made tools for Google and Bing (SERPs), Amazon,
51+
Walmart, and Target (eCommerce), Reddit, TikTok, and YouTube (social media), and ChatGPT and
52+
Perplexity (AI search).
4353

44-
**Global proxy infrastructure.** 125M+ residential IPs, 195+ geo-locations, and a 99.99% success rate on even the most protected targets.
54+
**Global proxy infrastructure.** 125M+ residential IPs, 195+ geo-locations, and a 99.99% success
55+
rate on even the most protected targets.
4556

46-
**Modular MCP toolsets.** Enable only what you need: `web`, `search`, `ecommerce`, `social_media`, `ai` for cleaner tool selection and better agent performance.
57+
**Modular MCP toolsets.** Enable only what you need: `web`, `search`, `ecommerce`, `social_media`,
58+
`ai` for cleaner tool selection and better agent performance.
4759

4860
**Fast time to value.** From API key to first scrape in minutes, no setup overhead.
4961

5062
## Use cases
5163

52-
Use the Decodo MCP Server when you need web scraping for AI agents, structured data extraction at scale, reliable access to dynamic websites, real-time data for RAG, or an alternative to building scraping infrastructure from scratch. Common scenarios:
53-
54-
- **AI-powered web scraping** – give LLMs the ability to collect fresh data instead of relying on static training data.
55-
- **RAG with live data** – pull real-time Google, Bing, and AI search results into retrieval pipelines.
56-
- **eCommerce intelligence** – track product prices, listings, and sellers across marketplaces without getting blocked.
57-
- **Social media data collection** – gather posts, channels, and engagement data from Reddit, TikTok, and YouTube.
58-
- **Travel and price aggregation** – build tools that collect live pricing and availability across websites.
64+
Use the Decodo MCP Server when you need web scraping for AI agents, structured data extraction at
65+
scale, reliable access to dynamic websites, real-time data for RAG, or an alternative to building
66+
scraping infrastructure from scratch. Common scenarios:
67+
68+
- **AI-powered web scraping** – give LLMs the ability to collect fresh data instead of relying on
69+
static training data.
70+
- **RAG with live data** – pull real-time Google, Bing, and AI search results into retrieval
71+
pipelines.
72+
- **eCommerce intelligence** – track product prices, listings, and sellers across marketplaces
73+
without getting blocked.
74+
- **Social media data collection** – gather posts, channels, and engagement data from Reddit,
75+
TikTok, and YouTube.
76+
- **Travel and price aggregation** – build tools that collect live pricing and availability across
77+
websites.
5978

6079
## Quick start
6180

62-
1. **Create a free account** at [dashboard.decodo.com](https://dashboard.decodo.com/) – up to 2K free requests, no credit card required.
81+
1. **Create a free account** at [dashboard.decodo.com](https://dashboard.decodo.com/) – up to 2K
82+
free requests, no credit card required.
6383
2. **Get your API key.** Obtain a Web Scraping API basic authentication token from the dashboard.
6484
3. **Configure the MCP server** in your AI client (see configuration examples below).
65-
4. **Connect your AI client.** Works with Claude Code, Cursor, Windsurf, and other MCP-compatible tools.
85+
4. **Connect your AI client.** Works with Claude Code, Cursor, Windsurf, and other MCP-compatible
86+
tools.
6687

6788
Your agent can now scrape and retrieve live web data.
6889

@@ -72,10 +93,12 @@ Your agent can now scrape and retrieve live web data.
7293

7394
```
7495
{
75-
"Decodo": {
76-
"url": "https://mcp.decodo.com/mcp",
77-
"headers": {
78-
"Authorization": "Basic <basic_auth_token>"
96+
"mcpServers": {
97+
"Decodo MCP Server": {
98+
"url": "https://mcp.decodo.com/mcp",
99+
"headers": {
100+
"Authorization": "Basic <basic_auth_token>"
101+
}
79102
}
80103
}
81104
}
@@ -88,9 +111,9 @@ Your agent can now scrape and retrieve live web data.
88111
"mcpServers": {
89112
"Decodo MCP Server": {
90113
"command": "npx",
91-
"args": ["@decodo/mcp-server"],
114+
"args": ["-y", "@decodo/mcp-server"],
92115
"env": {
93-
"SCRAPER_API_TOKEN": "<web_scraping_api_base64_token>",
116+
"SCRAPER_API_TOKEN": "<basic_auth_token>",
94117
"TOOLSETS": "web,ai"
95118
}
96119
}
@@ -273,14 +296,16 @@ Scrape hacker news, return full content.
273296

274297
## Related repositories
275298

276-
[Web Scraping API](https://github.com/Decodo/Web-Scraping-API), [Decodo OpenClaw skill](https://github.com/Decodo/decodo-openclaw-skill)
299+
[Web Scraping API](https://github.com/Decodo/Web-Scraping-API),
300+
[Decodo OpenClaw skill](https://github.com/Decodo/decodo-openclaw-skill)
277301

278302
## Try it
279303

280-
Plug Decodo MCP Server into your AI workflow in just a few clicks and equip your AI agents with real-time data from any website.
281-
282-
[Start for free](https://dashboard.decodo.com/) | [Docs](https://help.decodo.com/docs/introduction) | [Discord](https://discord.gg/Ja8dqKgvbZ)
304+
Plug Decodo MCP Server into your AI workflow in just a few clicks and equip your AI agents with
305+
real-time data from any website.
283306

307+
[Start for free](https://dashboard.decodo.com/) | [Docs](https://help.decodo.com/docs/introduction)
308+
| [Discord](https://discord.gg/Ja8dqKgvbZ)
284309

285310
## License
286311

0 commit comments

Comments
 (0)