Skip to content

Commit 1650c45

Browse files
committed
Add firehose streaming endpoints to recovery and redundancy docs
- Add Firehose Streams to intro alongside existing endpoints - Add Firehose, Sample (1%), and Sample10 (Decahose) to backfill table - Add Firehose curl examples for backfill and recovery sections - Document firehose partition-based redundancy (up to 20 connections) - Note that 1% Sample Stream does not support recovery
1 parent 7f1baa6 commit 1650c45

2 files changed

Lines changed: 48 additions & 6 deletions

File tree

enterprise-api/fundamentals/recovery-and-redundancy.mdx

Lines changed: 24 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: Features for recovering missed data and building resilient streamin
55
keywords: ["recovery", "redundancy", "stream recovery", "reconnect", "fault tolerance", "stream redundancy", "error recovery", "backfill"]
66
---
77

8-
Learn how to maximize connection time and recover missed data when using X streaming endpoints including [Filtered Stream](/x-api/posts/filtered-stream/introduction), [Volume Streams](/x-api/posts/volume-streams/introduction), [Powerstream](/x-api/powerstream/introduction), and [Compliance Streams](/x-api/compliance/streams/introduction).
8+
Learn how to maximize connection time and recover missed data when using X streaming endpoints including [Filtered Stream](/x-api/posts/filtered-stream/introduction), [Firehose Streams](/x-api/stream/stream-all-posts), [Volume Streams](/x-api/posts/volume-streams/introduction), [Powerstream](/x-api/powerstream/introduction), and [Compliance Streams](/x-api/compliance/streams/introduction).
99

1010
## Overview
1111

@@ -32,7 +32,7 @@ Benefits:
3232
Simply connect to the same stream URL with a second client. Data will be sent through both connections.
3333

3434
<Note>
35-
Redundant connections are available for Enterprise access. Filtered Stream allows up to two redundant connections for Enterprise projects. Check your specific endpoint documentation for connection limits.
35+
Redundant connections are available for Enterprise access. Filtered Stream allows up to two redundant connections for Enterprise projects. Firehose Streams use a `partition` parameter to support up to 20 concurrent connections, while Sample10 (Decahose) supports up to 2 partitions. Check your specific endpoint documentation for connection limits.
3636
</Note>
3737

3838
---
@@ -48,15 +48,27 @@ Use the **backfill parameter** when reconnecting to receive Posts matched during
4848
| Endpoint | Parameter | Example |
4949
|:---------|:----------|:--------|
5050
| Filtered Stream | `backfill_minutes` | `?backfill_minutes=5` |
51+
| Firehose Stream | `backfill_minutes` | `?backfill_minutes=5` |
52+
| Sample Stream (1%) | `backfill_minutes` | `?backfill_minutes=5` |
53+
| Sample10 Stream (Decahose) | `backfill_minutes` | `?backfill_minutes=5` |
5154
| Powerstream | `backfillMinutes` | `?backfillMinutes=5` |
5255

53-
Example request:
56+
Example requests:
57+
58+
**Filtered Stream:**
5459

5560
```bash
5661
curl 'https://api.x.com/2/tweets/search/stream?backfill_minutes=5' \
5762
-H "Authorization: Bearer $ACCESS_TOKEN"
5863
```
5964

65+
**Firehose Stream:**
66+
67+
```bash
68+
curl 'https://api.x.com/2/tweets/firehose/stream?backfill_minutes=5&partition=1' \
69+
-H "Authorization: Bearer $ACCESS_TOKEN"
70+
```
71+
6072
<Note>
6173
**Important considerations:**
6274
- Older Posts are generally delivered first, before newly matched Posts
@@ -93,6 +105,13 @@ curl 'https://api.x.com/2/tweets/search/stream?start_time=2022-07-12T15:10:00Z&e
93105
-H "Authorization: Bearer $ACCESS_TOKEN"
94106
```
95107

108+
**Firehose Stream:**
109+
110+
```bash
111+
curl 'https://api.x.com/2/tweets/firehose/stream?start_time=2022-07-12T15:10:00Z&end_time=2022-07-12T15:20:00Z&partition=1' \
112+
-H "Authorization: Bearer $ACCESS_TOKEN"
113+
```
114+
96115
**Powerstream:**
97116

98117
```bash
@@ -105,6 +124,8 @@ curl 'https://api.x.com/2/powerstream?startTime=2022-07-12T15:10:00Z&endTime=202
105124
- Available for Enterprise access
106125
- Recovery window: up to 24 hours in the past
107126
- Filtered Stream allows 2 concurrent recovery jobs
127+
- Firehose, Sample10 (Decahose), and language-specific firehose streams also support recovery
128+
- The basic 1% Sample Stream does not support recovery; use the Search endpoint instead
108129
</Note>
109130

110131
---

x-api/fundamentals/recovery-and-redundancy.mdx

Lines changed: 24 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: Features for recovering missed data and building resilient streamin
55
keywords: ["recovery", "redundancy", "stream recovery", "reconnect", "fault tolerance", "stream redundancy", "error recovery", "backfill"]
66
---
77

8-
Learn how to maximize connection time and recover missed data when using X streaming endpoints including [Filtered Stream](/x-api/posts/filtered-stream/introduction), [Volume Streams](/x-api/posts/volume-streams/introduction), [Powerstream](/x-api/powerstream/introduction), and [Compliance Streams](/x-api/compliance/streams/introduction).
8+
Learn how to maximize connection time and recover missed data when using X streaming endpoints including [Filtered Stream](/x-api/posts/filtered-stream/introduction), [Firehose Streams](/x-api/stream/stream-all-posts), [Volume Streams](/x-api/posts/volume-streams/introduction), [Powerstream](/x-api/powerstream/introduction), and [Compliance Streams](/x-api/compliance/streams/introduction).
99

1010
## Overview
1111

@@ -32,7 +32,7 @@ Benefits:
3232
Simply connect to the same stream URL with a second client. Data will be sent through both connections.
3333

3434
<Note>
35-
Redundant connections are available for Enterprise access. Filtered Stream allows up to two redundant connections for Enterprise projects. Check your specific endpoint documentation for connection limits.
35+
Redundant connections are available for Enterprise access. Filtered Stream allows up to two redundant connections for Enterprise projects. Firehose Streams use a `partition` parameter to support up to 20 concurrent connections, while Sample10 (Decahose) supports up to 2 partitions. Check your specific endpoint documentation for connection limits.
3636
</Note>
3737

3838
---
@@ -48,15 +48,27 @@ Use the **backfill parameter** when reconnecting to receive Posts matched during
4848
| Endpoint | Parameter | Example |
4949
|:---------|:----------|:--------|
5050
| Filtered Stream | `backfill_minutes` | `?backfill_minutes=5` |
51+
| Firehose Stream | `backfill_minutes` | `?backfill_minutes=5` |
52+
| Sample Stream (1%) | `backfill_minutes` | `?backfill_minutes=5` |
53+
| Sample10 Stream (Decahose) | `backfill_minutes` | `?backfill_minutes=5` |
5154
| Powerstream | `backfillMinutes` | `?backfillMinutes=5` |
5255

53-
Example request:
56+
Example requests:
57+
58+
**Filtered Stream:**
5459

5560
```bash
5661
curl 'https://api.x.com/2/tweets/search/stream?backfill_minutes=5' \
5762
-H "Authorization: Bearer $ACCESS_TOKEN"
5863
```
5964

65+
**Firehose Stream:**
66+
67+
```bash
68+
curl 'https://api.x.com/2/tweets/firehose/stream?backfill_minutes=5&partition=1' \
69+
-H "Authorization: Bearer $ACCESS_TOKEN"
70+
```
71+
6072
<Note>
6173
**Important considerations:**
6274
- Older Posts are generally delivered first, before newly matched Posts
@@ -93,6 +105,13 @@ curl 'https://api.x.com/2/tweets/search/stream?start_time=2022-07-12T15:10:00Z&e
93105
-H "Authorization: Bearer $ACCESS_TOKEN"
94106
```
95107

108+
**Firehose Stream:**
109+
110+
```bash
111+
curl 'https://api.x.com/2/tweets/firehose/stream?start_time=2022-07-12T15:10:00Z&end_time=2022-07-12T15:20:00Z&partition=1' \
112+
-H "Authorization: Bearer $ACCESS_TOKEN"
113+
```
114+
96115
**Powerstream:**
97116

98117
```bash
@@ -105,6 +124,8 @@ curl 'https://api.x.com/2/powerstream?startTime=2022-07-12T15:10:00Z&endTime=202
105124
- Available for Enterprise access
106125
- Recovery window: up to 24 hours in the past
107126
- Filtered Stream allows 2 concurrent recovery jobs
127+
- Firehose, Sample10 (Decahose), and language-specific firehose streams also support recovery
128+
- The basic 1% Sample Stream does not support recovery; use the Search endpoint instead
108129
</Note>
109130

110131
---

0 commit comments

Comments
 (0)