Skip to content

Avoid a race-condition on ETag validation.#425

Open
JulienPalard wants to merge 1 commit into
psf:masterfrom
JulienPalard:mdk-race
Open

Avoid a race-condition on ETag validation.#425
JulienPalard wants to merge 1 commit into
psf:masterfrom
JulienPalard:mdk-race

Conversation

@JulienPalard
Copy link
Copy Markdown

For example, in case multiple Python versions are using the same cache and some are <3.14 and other >=3.14, the Accept-Encoding header will vary between gzip, deflate and gzip, deflate, zstd.

If two processes, say p1 (with zstd support) and p2 (without zstd support) do two requests to the same URL, with an already cached result (for the without zstd support request) it could lead to the following race condition:

  • P1 reads the cache, fails because Accept-Encoding does not match
  • P1 sends a normal request
  • P2 reads the cache, succeeds, but needs validation
  • P2 sends a validation request (with If-None-Match)
  • P1 receives a response for its Accept-Encoding: gzip, deflate, zstd
  • P1 updates the cache with it
  • P1 returns the 200 OK to the user
  • P2 receives a 304 Not Modified response
  • P2 wants to return from the cache, but fails to read it (wrong Accept-Encoding!)
  • P2 fallbacks to returning a 304 Not Modified response to the user :(

Closes #424

@JulienPalard
Copy link
Copy Markdown
Author

JulienPalard commented May 19, 2026

As a nice side effect this save one disk read in case the server answers a 304 to a validation request.

But I'm not proud of my edit of conditional_headers, it previously was kind of pure, it now alters a "global state". It would have been better to have this to the adapter. But having the state modification to the adapter is worse: it can't cleanly call self.controller._load_from_cache (private Controller method), and it alters the prototype of conditional_headers (passing it the cached response).

So if someone with a better theory of the project than me has a better idea, don't hesitate. After all that's just the first time I read cachecontrol code.

For example, in case multiple Python versions are using the same cache
and some are <3.14 and other >=3.14, the Accept-Encoding header will
vary between `gzip, deflate` and `gzip, deflate, zstd`.

If two processes, say p1 (with zstd support) and p2 (without zstd
support) do two requests to the same URL, with an already cached
result (for the without zstd support request) it could lead to the
following race condition:

- P1 reads the cache, fails because Accept-Encoding does not match
- P1 sends a normal request
- P2 reads the cache, succeeds, but needs validation
- P2 sends a validation request (with If-None-Match)
- P1 receives a response for its Accept-Encoding: gzip, deflate, zstd
- P1 updates the cache with it
- P1 returns the 200 OK to the user
- P2 receives a 304 Not Modified response
- P2 wants to return from the cache, but fails to read it (wrong Accept-Encoding!)
- P2 fallbacks to returning a 304 Not Modified response  to the user :(
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition in send()

1 participant