Skip to content

Commit f465872

Browse files
committed
Refactor LangGraph documentation on StreamIO integration and retry policies
- Updated the section on streaming messages into StreamIO with exponential backoff, clarifying the use of configurable retry policies. - Added context on node-level retry policies in LangGraph, including code examples for implementing exponential backoff. - Enhanced explanations of handling rate limits and optimizing retry logic for better clarity and usability.
1 parent 646d206 commit f465872

2 files changed

Lines changed: 30 additions & 27 deletions

pages/Programming___Error Handling___Retry Logic___Exponential Backoff.md

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,13 +19,29 @@ alias:: [[Exponential Backoff]]
1919
- Reduces network congestion
2020
- More efficient than fixed-interval retries
2121
- ## Example Formula
22-
- delay = min(max_delay, initial_delay * (base ^ attempt_number))
22+
- `delay = min(max_delay, initial_delay * (base ^ attempt_number))`
2323
- For base=2:
2424
- 1st retry: 1s
2525
- 2nd retry: 2s
2626
- 3rd retry: 4s
2727
- 4th retry: 8s
2828
- etc.
29+
- Note:
30+
- The key property of *exponential backoff* is **multiplicative growth** of the wait-time:
31+
32+
\[
33+
t_n = t_0 \times f^{\,n},
34+
\]
35+
36+
where \(f>1\) is a constant factor and \(n\) is the retry count.
37+
Doubling (\(f = 2\)) yields the classic binary-exponential sequence
38+
39+
\[
40+
1\text{ s} \;\to\; 2\text{ s} \;\to\; 4\text{ s} \;\to\; 8\text{ s} \;\dots
41+
\]
42+
43+
which is what the code does with `delay = initial_delay * (base ^ attempt_number)`.
44+
Multiplying by a fixed constant on each failure therefore produces the exponential curve \(t_n = t_0\,2^{\,n}\).
2945
- ## Implementation Examples
3046
- [[LangChain/Blog/25/04/17 LangChain Python Improved Content Blocks Retry Logic and More]] - LangChain's Runnable.with_retry implementation
3147
- ## See also

pages/langgraph___How To___Stream messages into StreamIO with Exponential Backoff.md

Lines changed: 13 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -17,24 +17,10 @@
1717
- **Batching Chunks:** If rate limits are frequently hit, consider batching multiple chunks together and updating StreamIO less frequently. This reduces the number of API calls and better utilizes the allowed quota.
1818
- **Skipping Redundant Updates:** In scenarios where only the latest chunk matters (e.g., replacing message content), it may be optimal to skip intermediate updates that failed due to rate limits and only update with the most recent chunk after the backoff period.
1919
- **Inspect Rate Limit Headers:** Always inspect the `X-RateLimit-Remaining` and `X-RateLimit-Reset` headers in StreamIO responses to dynamically adjust backoff timing and avoid unnecessary retries.
20-
- **Configurable Retry Policy:** In LangGraph, you can configure retry policies for nodes (see [How to add node retry policies](https://langchain-ai.github.io/langgraph/how-tos/node-retries/)), allowing for exponential backoff and custom retry logic on API errors like 429.
21-
- ### Example Retry Policy in LangGraph
22-
- Use the `RetryPolicy` when adding a node that updates StreamIO, specifying `initial_interval`, `backoff_factor`, `max_interval`, and `max_attempts`.
23-
- Example:
24-
~~~python
25-
from langgraph.pregel import RetryPolicy
26-
builder.add_node(
27-
"update_streamio",
28-
update_streamio_fn,
29-
retry=RetryPolicy(initial_interval=1.0, backoff_factor=2.0, max_interval=32.0, max_attempts=5)
30-
)
31-
~~~
32-
- This ensures that if a rate limit error occurs, the node will retry with exponential backoff, up to the specified maximum attempts.
3320
- ### Summary
3421
- When streaming from LangGraph to StreamIO, design your update logic to:
3522
- Handle 429 errors with exponential backoff
3623
- Consider batching or skipping redundant updates
37-
- Use LangGraph's retry policies for robust error handling
3824
- Monitor rate limit headers to optimize retry timing
3925
- This approach balances responsiveness, efficiency, and compliance with StreamIO's rate limits.
4026
- ## Algorithms
@@ -91,15 +77,16 @@
9177
```
9278
- Uses Stream's **partial-update** endpoint so you never overwrite undeclared fields ([Build an AI Assistant Using Python - getstream.io](https://getstream.io/blog/python-assistant/?utm_source=chatgpt.com)).
9379
- Works with any LangGraph streaming mode; just adapt the buffer strategy for "replace" vs "append".
94-
- ### Node-Level Retry Policy (optional)
95-
- ```python
96-
from langgraph.pregel import RetryPolicy
97-
builder.add_node(
98-
"update_streamio",
99-
lambda state: stream_to_streamio(state["run_id"], state["msg_id"]),
100-
retry=RetryPolicy(initial_interval=1.0, backoff_factor=2.0,
101-
max_interval=32.0, max_attempts=5)
102-
)
103-
104-
```
105-
- This lets LangGraph itself re-invoke the node when a 429 bubbles up. ([Streaming](https://langchain-ai.github.io/langgraph/concepts/streaming/))
80+
- ## Additional Context: Node-Level Retry Policies in LangGraph
81+
- **Configurable Retry Policy:** In LangGraph, you can configure retry policies for nodes (see [How to add node retry policies](https://langchain-ai.github.io/langgraph/how-tos/node-retries/)), allowing for exponential backoff and custom retry logic on API errors like 429.
82+
- **Note:** Node-level retry policies in LangGraph are only relevant if a LangGraph node is directly responsible for updating StreamIO. In the main scenario discussed above, streaming from LangGraph and updating StreamIO are decoupled, so node-level retry policies do not apply. If, however, you architect your graph such that a node performs the StreamIO update, you can use a retry policy as shown below.
83+
- ```python
84+
from langgraph.pregel import RetryPolicy
85+
builder.add_node(
86+
"update_streamio",
87+
lambda state: stream_to_streamio(state["run_id"], state["msg_id"]),
88+
retry=RetryPolicy(initial_interval=1.0, backoff_factor=2.0,
89+
max_interval=32.0, max_attempts=5)
90+
)
91+
```
92+
- This lets LangGraph itself re-invoke the node when a 429 bubbles up. ([Streaming](https://langchain-ai.github.io/langgraph/concepts/streaming/))

0 commit comments

Comments
 (0)