Skip to content

Commit 46a89a5

Browse files
arun-fairfetchcursoragent
arun-fairfetch
and
cursoragent
committed
Initial commit: Fairfetch AI-Aware Content Layer v0.2.0
Open-source infrastructure for publishers to serve machine-ready content to AI agents via x402 micro-payments, MCP, and EU AI Act compliance. Three pillars: - Green AI: HTML-to-Markdown pre-processing at source - Legal Safe Harbor: Ed25519-signed Usage Grants for legal indemnity - Direct Pipeline: Bot steering from scraping to official API Includes REST API (FastAPI), MCP server (FastMCP), mock payment facilitator, edge boilerplates (Cloudflare/CloudFront/Fastly/Akamai), publisher and AI agent guides, and 106 tests at 98% coverage. Co-authored-by: Cursor <cursoragent@cursor.com>
0 parents  commit 46a89a5

57 files changed

Lines changed: 6329 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
jobs:
10+
test:
11+
runs-on: ubuntu-latest
12+
strategy:
13+
matrix:
14+
python-version: ["3.11", "3.12", "3.13"]
15+
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- name: Set up Python ${{ matrix.python-version }}
20+
uses: actions/setup-python@v5
21+
with:
22+
python-version: ${{ matrix.python-version }}
23+
24+
- name: Cache pip
25+
uses: actions/cache@v4
26+
with:
27+
path: ~/.cache/pip
28+
key: ${{ runner.os }}-pip-${{ hashFiles('pyproject.toml') }}
29+
restore-keys: |
30+
${{ runner.os }}-pip-
31+
32+
- name: Install dependencies
33+
run: |
34+
python -m pip install --upgrade pip
35+
pip install -e ".[dev]"
36+
37+
- name: Lint
38+
run: |
39+
ruff check .
40+
ruff format --check .
41+
42+
- name: Type check
43+
run: |
44+
mypy core api payments compliance mcp_server interfaces --ignore-missing-imports
45+
46+
- name: Run tests
47+
env:
48+
FAIRFETCH_TEST_MODE: "true"
49+
run: |
50+
pytest tests/ -v --tb=short --cov=core --cov=api --cov=payments --cov=compliance --cov=mcp_server --cov=interfaces --cov-report=xml
51+
52+
- name: Upload coverage
53+
if: matrix.python-version == '3.12'
54+
uses: actions/upload-artifact@v4
55+
with:
56+
name: coverage-report
57+
path: coverage.xml

‎.gitignore‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Python
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
*.so
6+
*.egg
7+
*.egg-info/
8+
dist/
9+
build/
10+
*.whl
11+
12+
# Virtual environments
13+
.venv/
14+
venv/
15+
ENV/
16+
17+
# IDE
18+
.vscode/
19+
.idea/
20+
*.swp
21+
*.swo
22+
*~
23+
.cursor/
24+
25+
# Testing & coverage
26+
.pytest_cache/
27+
.coverage
28+
coverage.xml
29+
htmlcov/
30+
31+
# Type checking & linting
32+
.mypy_cache/
33+
.ruff_cache/
34+
35+
# Environment & secrets
36+
.env
37+
.env.*
38+
!.env.example
39+
credentials.json
40+
*.pem
41+
*.key
42+
43+
# Data files (runtime generated)
44+
data/
45+
46+
# OS
47+
.DS_Store
48+
Thumbs.db
49+
50+
# Node (MCP Inspector)
51+
node_modules/
52+
package-lock.json

‎CODE_OF_CONDUCT.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Contributor Covenant Code of Conduct
2+
3+
## Our Pledge
4+
5+
We as members, contributors, and leaders pledge to make participation in our
6+
community a harassment-free experience for everyone, regardless of age, body
7+
size, visible or invisible disability, ethnicity, sex characteristics, gender
8+
identity and expression, level of experience, education, socio-economic status,
9+
nationality, personal appearance, race, caste, color, religion, or sexual
10+
identity and orientation.
11+
12+
We pledge to act and interact in ways that contribute to an open, welcoming,
13+
diverse, inclusive, and healthy community.
14+
15+
## Our Standards
16+
17+
Examples of behavior that contributes to a positive environment:
18+
19+
* Demonstrating empathy and kindness toward other people
20+
* Being respectful of differing opinions, viewpoints, and experiences
21+
* Giving and gracefully accepting constructive feedback
22+
* Accepting responsibility and apologizing to those affected by our mistakes
23+
* Focusing on what is best not just for us as individuals, but for the overall community
24+
25+
Examples of unacceptable behavior:
26+
27+
* The use of sexualized language or imagery, and sexual attention or advances of any kind
28+
* Trolling, insulting or derogatory comments, and personal or political attacks
29+
* Public or private harassment
30+
* Publishing others' private information without their explicit permission
31+
* Other conduct which could reasonably be considered inappropriate in a professional setting
32+
33+
## Enforcement Responsibilities
34+
35+
Community leaders are responsible for clarifying and enforcing our standards of
36+
acceptable behavior and will take appropriate and fair corrective action in
37+
response to any behavior that they deem inappropriate, threatening, offensive,
38+
or harmful.
39+
40+
## Scope
41+
42+
This Code of Conduct applies within all community spaces, and also applies when
43+
an individual is officially representing the community in public spaces.
44+
45+
## Enforcement
46+
47+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
48+
reported to the community leaders responsible for enforcement at
49+
**conduct@fairfetch.dev**.
50+
51+
All complaints will be reviewed and investigated promptly and fairly.
52+
53+
## Attribution
54+
55+
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/),
56+
version 2.1, available at
57+
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html](https://www.contributor-covenant.org/version/2/1/code_of_conduct.html).

‎CONTRIBUTING.md‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# Contributing to Fairfetch
2+
3+
Thank you for your interest in contributing to Fairfetch. This document provides
4+
guidelines, instructions, and legal information for contributors.
5+
6+
## Contributor License Agreement (CLA)
7+
8+
**By submitting a pull request or otherwise contributing code to this repository,
9+
you agree to the following terms:**
10+
11+
1. You grant FairFetch and its affiliates a perpetual, worldwide, non-exclusive,
12+
royalty-free, irrevocable license to use, reproduce, modify, distribute,
13+
sublicense, and create derivative works from your contribution.
14+
15+
2. You grant FairFetch the right to use your contribution in both:
16+
- The **open-source** FairFetch project (Apache 2.0 License), and
17+
- The **commercial FairFetch Cloud** offering (Managed Clearinghouse service).
18+
19+
3. You represent that you have the legal right to grant these licenses and that
20+
your contribution does not violate any third-party rights.
21+
22+
4. You understand that your contribution is provided "as-is" without warranty.
23+
24+
This CLA is necessary to maintain the Open Core model: the open-source standard
25+
remains freely available while enabling a sustainable commercial offering that
26+
funds continued development.
27+
28+
If you have questions about the CLA, please open an issue or email legal@fairfetch.dev.
29+
30+
## Getting Started
31+
32+
1. Fork the repository on GitHub
33+
2. Clone your fork locally
34+
3. Set up the development environment:
35+
36+
```bash
37+
python -m venv .venv
38+
source .venv/bin/activate
39+
make setup-dev
40+
```
41+
42+
4. Create a feature branch:
43+
44+
```bash
45+
git checkout -b feature/your-feature-name
46+
```
47+
48+
## Development Workflow
49+
50+
1. Make your changes
51+
2. Write or update tests
52+
3. Run the test suite: `make test`
53+
4. Run linting: `make lint`
54+
5. Run type checking: `make typecheck`
55+
56+
## Code Standards
57+
58+
- **Python 3.12+** with strict type annotations
59+
- **Pydantic v2** for all data models
60+
- **Async/await** for all network operations
61+
- **ruff** for linting and formatting
62+
- **mypy** in strict mode for type checking
63+
64+
### Architecture Boundaries
65+
66+
FairFetch follows the **Open Core** model with clear boundaries:
67+
68+
| Layer | Directory | Purpose |
69+
|-------|-----------|---------|
70+
| **Standard (Open)** | `interfaces/` | Abstract base classes anyone can implement |
71+
| **Green AI (Open)** | `core/` | Content extraction, summarization, signing |
72+
| **Protocol (Open)** | `api/`, `mcp_server/` | REST + MCP endpoints, content negotiation |
73+
| **Payments (Open)** | `payments/` | x402 middleware, mock facilitator |
74+
| **Compliance (Open)** | `compliance/` | EU AI Act headers, lineage, copyright |
75+
| **Cloud (Placeholder)** | `plugins/` | Stubs for managed clearinghouse |
76+
77+
When contributing, ensure your changes respect these boundaries. Interface
78+
changes should go in `interfaces/`, not be hardcoded into implementations.
79+
80+
### Style Guidelines
81+
82+
- Use `from __future__ import annotations` in all modules
83+
- Prefer `dataclass(frozen=True, slots=True)` for immutable value types
84+
- Use Pydantic `BaseModel` for serializable/API models
85+
- All public classes and functions must have docstrings
86+
- No comments that merely restate the code
87+
88+
## Testing
89+
90+
- All new features must include tests
91+
- Use `pytest` with `pytest-asyncio` for async tests
92+
- Mock external services (LLM APIs, network calls) in unit tests
93+
- Use `MockFacilitator` and `MockLicenseProvider` for payment/grant tests
94+
- Test fixtures go in `tests/conftest.py`
95+
- Validate that responses include the **Green+Legal+Indemnity** triple
96+
97+
### Running Tests
98+
99+
```bash
100+
make test # Full suite with coverage
101+
make test-unit # Unit tests only
102+
make dev-mcp # MCP server manual testing via Inspector
103+
```
104+
105+
## Pull Request Process
106+
107+
1. Ensure all tests pass
108+
2. Update documentation if adding new features
109+
3. Create a pull request with a clear description of the changes
110+
4. Reference any related issues
111+
112+
### PR Title Convention
113+
114+
- `feat:` New functionality
115+
- `fix:` Bug fixes
116+
- `docs:` Documentation only
117+
- `refactor:` No behavior change
118+
- `test:` Test additions
119+
- `chore:` Maintenance
120+
121+
## Areas for Contribution
122+
123+
### High Priority
124+
125+
- Production `BaseFacilitator` implementation (EIP-3009 on-chain verification)
126+
- Production `BaseLicenseProvider` with persistent grant storage
127+
- Additional edge deployment templates
128+
- Performance benchmarking suite
129+
130+
### Good First Issues
131+
132+
- Improve error messages in the x402 middleware
133+
- Add more crawler User-Agent patterns to the detection list
134+
- Add support for additional content formats (EPUB, PDF)
135+
- Improve test coverage for edge cases
136+
137+
## License
138+
139+
By contributing, you agree that your contributions will be licensed under the
140+
Apache 2.0 License, subject to the CLA terms above.

0 commit comments

Comments
 (0)