Skip to content

Commit ced1c91

Browse files
HavenDVHavenDV
committed
Fix docs sample generation
1 parent 5fbdc28 commit ced1c91

5 files changed

Lines changed: 488 additions & 34 deletions

File tree

‎docs/index.md‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# Firecrawl
2+
3+
[![Nuget package](https://img.shields.io/nuget/vpre/Firecrawl)](https://www.nuget.org/packages/Firecrawl/)
4+
[![dotnet](https://github.com/tryAGI/Firecrawl/actions/workflows/dotnet.yml/badge.svg?branch=main)](https://github.com/tryAGI/Firecrawl/actions/workflows/dotnet.yml)
5+
[![License: MIT](https://img.shields.io/github/license/tryAGI/Firecrawl)](https://github.com/tryAGI/Firecrawl/blob/main/LICENSE.txt)
6+
[![Discord](https://img.shields.io/discord/1115206893015662663?label=Discord&logo=discord&logoColor=white&color=d82679)](https://discord.gg/Ca2xhfBf3v)
7+
8+
## Features 🔥
9+
- Fully generated C# SDK based on [official Firecrawl OpenAPI specification](https://raw.githubusercontent.com/mendableai/firecrawl/main/apps/api/v1-openapi.json) using [AutoSDK](https://github.com/tryAGI/AutoSDK)
10+
- Same day update to support new features
11+
- Updated and supported automatically if there are no breaking changes
12+
- All modern .NET features - nullability, trimming, NativeAOT, etc.
13+
- Support .Net Framework/.Net Standard 2.0
14+
15+
### Usage
16+
```csharp
17+
using Firecrawl;
18+
19+
using var client = new FirecrawlClient(apiKey);
20+
21+
// Scrape
22+
var response = await client.Scraping.ScrapeAndExtractFromUrlAsync("https://docs.firecrawl.dev/features/scrape");
23+
24+
string markdown = response.Data.Markdown;
25+
26+
// Crawl
27+
var response = await client.Crawling.CrawlUrlsAsync(
28+
url: "https://docs.firecrawl.dev/",
29+
limit: 3,
30+
scrapeOptions: new CrawlUrlsRequestScrapeOptions
31+
{
32+
OnlyMainContent = true,
33+
});
34+
35+
var jobResponse = await client.Crawling.WaitJobAsync(
36+
jobId: response.JobId);
37+
38+
foreach (var data in jobResponse.Data)
39+
{
40+
Console.WriteLine($"URL: {data.Metadata.SourceURL}");
41+
Console.WriteLine($"Output file: {data.Markdown}");
42+
}
43+
```
44+
45+
### CLI
46+
```bash
47+
dotnet tool install -g Firecrawl.Cli
48+
firecrawl auth <API_KEY>
49+
firecrawl scrape https://docs.firecrawl.dev/features/scrape // saves it to output.md
50+
firecrawl crawl https://docs.firecrawl.dev/ --limit 5 // saves all .md files to docs.firecrawl.dev folder
51+
```
52+
53+
## Support
54+
55+
Priority place for bugs: https://github.com/tryAGI/Firecrawl/issues
56+
Priority place for ideas and general questions: https://github.com/tryAGI/Firecrawl/discussions
57+
Discord: https://discord.gg/Ca2xhfBf3v
58+
59+
## Acknowledgments
60+
61+
![JetBrains logo](https://resources.jetbrains.com/storage/products/company/brand/logos/jetbrains.png)
62+
63+
This project is supported by JetBrains through the [Open Source Support Program](https://jb.gg/OpenSourceSupport).
64+
65+
![CodeRabbit logo](https://opengraph.githubassets.com/1c51002d7d0bbe0c4fd72ff8f2e58192702f73a7037102f77e4dbb98ac00ea8f/marketplace/coderabbitai)
66+
67+
This project is supported by CodeRabbit through the [Open Source Support Program](https://github.com/marketplace/coderabbitai).

‎docs/samples/Crawl.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
```csharp
2+
using var client = new FirecrawlClient(apiKey);
3+
using var cancellationTokenSource = new CancellationTokenSource(TimeSpan.FromMinutes(5));
4+
var cancellationToken = cancellationTokenSource.Token;
5+
6+
var response = await client.Crawling.CrawlUrlsAsync(
7+
url: "https://docs.firecrawl.dev/",
8+
limit: 3,
9+
scrapeOptions: new ScrapeOptions
10+
{
11+
OnlyMainContent = true,
12+
},
13+
cancellationToken: cancellationToken);
14+
15+
Console.WriteLine($"Success: {response.Success}");
16+
Console.WriteLine($"JobId: {response.Id}");
17+
Console.WriteLine($"Url: {response.Url}");
18+
19+
var jobResponse = await client.Crawling.WaitJobAsync(
20+
jobId: response.Id!,
21+
cancellationToken: cancellationToken);
22+
23+
var index = 0;
24+
foreach (var data in jobResponse.Data ?? [])
25+
{
26+
27+
var fileInfo = new FileInfo($"output{++index}.md");
28+
await File.WriteAllTextAsync(fileInfo.FullName, data.Markdown, cancellationToken);
29+
Console.WriteLine($"Output file: {new Uri(fileInfo.FullName).AbsoluteUri}");
30+
}
31+
```

‎docs/samples/Scrape.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
```csharp
2+
using var client = new FirecrawlClient(apiKey);
3+
4+
var response = await client.Scraping.ScrapeAndExtractFromUrlAsync(new AllOf<ScrapeAndExtractFromUrlRequest2, ScrapeOptions>
5+
{
6+
Value1 = new ScrapeAndExtractFromUrlRequest2
7+
{
8+
Url = "https://docs.firecrawl.dev/features/scrape",
9+
},
10+
});
11+
12+
Console.WriteLine($"Success: {response.Success}");
13+
14+
var fileInfo = new FileInfo("output.md");
15+
await File.WriteAllTextAsync(fileInfo.FullName, response.Data.Markdown);
16+
Console.WriteLine($"Output file: {new Uri(fileInfo.FullName).AbsoluteUri}");
17+
```

‎mkdocs.yml‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,11 @@
11
site_name: Firecrawl .NET Documentation
22
nav:
33
- Overview: index.md
4-
# EXAMPLES #
4+
# EXAMPLES:START
5+
- Examples:
6+
- Crawl: samples/Crawl.md
7+
- Scrape: samples/Scrape.md
8+
# EXAMPLES:END
59

610
theme:
711
name: material
@@ -79,4 +83,4 @@ markdown_extensions:
7983
custom_checkbox: true
8084
- pymdownx.tilde
8185
- pymdownx.tabbed:
82-
alternate_style: true
86+
alternate_style: true

0 commit comments

Comments
 (0)