When a URL request fails, is the retry count for it always decremented?
I can think of a few cases where the URL is not the problem:
- A bad proxy.
- Target server overload.
- Transient resource exhaustion on the local machine.
Could you clarify how Crawlee handles these situations?
When a URL request fails, is the retry count for it always decremented?
I can think of a few cases where the URL is not the problem:
Could you clarify how Crawlee handles these situations?