Skip to content

Commit c66f327

Browse files
committed
docs(0.9.0): flag secure-by-default changes on the self-hosting page
- Add a 0.9.0 secure-by-default callout + migration-guide link at the top. - Mark the inline-Python hooks section as removed/replaced by declarative hooks (it described an unauthenticated code-execution surface). - Note the 0.9.0 secure defaults on the embedded security config block. - Bump docs site_name to v0.9.x. A fuller per-example rewrite of self-hosting.md will follow.
1 parent 60886d1 commit c66f327

2 files changed

Lines changed: 24 additions & 1 deletion

File tree

docs/md_v2/core/self-hosting.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
# Self-Hosting Crawl4AI 🚀
22

3+
> **🔐 0.9.0 is secure-by-default (breaking changes).** The self-hosted Docker
4+
> server now requires authentication by default, binds to loopback unless you
5+
> set a token, validates request bodies against a strict trust boundary, uses
6+
> declarative hooks instead of inline Python, and returns artifact ids for
7+
> screenshot/pdf. If you are upgrading from 0.8.x, read the
8+
> [migration guide](https://github.com/unclecode/crawl4ai/blob/main/deploy/docker/MIGRATION.md)
9+
> first. Some examples below are being updated for 0.9.0; the migration guide is
10+
> the authoritative reference for the new defaults.
11+
312
**Take Control of Your Web Crawling Infrastructure**
413

514
Self-hosting Crawl4AI gives you complete control over your web crawling and data extraction pipeline. Unlike cloud-based solutions, you own your data, infrastructure, and destiny.
@@ -442,6 +451,15 @@ Executes JavaScript snippets on the specified URL and returns the full crawl res
442451
443452
## User-Provided Hooks API
444453
454+
> **⚠️ Changed in 0.9.0.** The inline-Python hook API described below was removed.
455+
> Sending arbitrary Python code to the server is no longer accepted (it was an
456+
> unauthenticated code-execution surface). 0.9.0 replaces it with **declarative
457+
> hooks**: a fixed set of safe, server-validated actions (for example
458+
> `add_cookies`, `set_headers`, `block_resources`) supplied as JSON, with no code
459+
> execution. See the [migration guide](https://github.com/unclecode/crawl4ai/blob/main/deploy/docker/MIGRATION.md)
460+
> for the declarative hook format. The inline-code examples in this section apply
461+
> to 0.8.x only and are kept for reference until this page is fully rewritten.
462+
445463
The Docker API supports user-provided hook functions, allowing you to customize the crawling behavior by injecting your own Python code at specific points in the crawling pipeline. This powerful feature enables authentication, performance optimization, and custom content extraction without modifying the server code.
446464
447465
> ⚠️ **IMPORTANT SECURITY WARNING**:
@@ -2459,6 +2477,11 @@ rate_limiting:
24592477
storage_uri: "memory://" # Use "redis://localhost:6379" if you need persistent/shared limits
24602478
24612479
# Security Configuration
2480+
# NOTE (0.9.0): the defaults below reflect 0.8.x. In 0.9.0 the Docker server is
2481+
# secure-by-default - authentication is required, the server binds loopback
2482+
# unless a token is set, and request bodies are validated against a trust
2483+
# boundary. See the migration guide for the 0.9.0 defaults and config keys:
2484+
# https://github.com/unclecode/crawl4ai/blob/main/deploy/docker/MIGRATION.md
24622485
security:
24632486
enabled: false # Master toggle for security features
24642487
jwt_enabled: false # Enable JWT authentication (requires security.enabled=true)

mkdocs.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
site_name: Crawl4AI Documentation (v0.8.x)
1+
site_name: Crawl4AI Documentation (v0.9.x)
22
site_description: 🚀🤖 Crawl4AI, Open-source LLM-Friendly Web Crawler & Scraper
33
site_url: https://docs.crawl4ai.com
44
repo_url: https://github.com/unclecode/crawl4ai

0 commit comments

Comments
 (0)