Avoid a race-condition on ETag validation.

For example, in case multiple Python versions are using the same cache
and some are <3.14 and other >=3.14, the Accept-Encoding header will
vary between `gzip, deflate` and `gzip, deflate, zstd`.

If two processes, say p1 (with zstd support) and p2 (without zstd
support) do two requests to the same URL, with an already cached
result (for the without zstd support request) it could lead to the
following race condition:

- P1 reads the cache, fails because Accept-Encoding does not match
- P1 sends a normal request
- P2 reads the cache, succeeds, but needs validation
- P2 sends a validation request (with If-None-Match)
- P1 receives a response for its Accept-Encoding: gzip, deflate, zstd
- P1 updates the cache with it
- P1 returns the 200 OK to the user
- P2 receives a 304 Not Modified response
- P2 wants to return from the cache, but fails to read it (wrong Accept-Encoding!)
- P2 fallbacks to returning a 304 Not Modified response  to the user :(

Author: JulienPalard

cachecontrol/controller.py

@@ -61,6 +61,7 @@ def __init__(
         self.cache_etags = cache_etags
         self.serializer = serializer or Serializer()
         self.cacheable_status_codes = status_codes or (200, 203, 300, 301, 308)
+        self._request_to_cache_response = weakref.WeakKeyDictionary()
 
     @classmethod
     def _urlnorm(cls, uri: str) -> str:
@@ -281,6 +282,7 @@ def conditional_headers(self, request: PreparedRequest) -> dict[str, str]:
         new_headers = {}
 
         if resp:
+            self._request_to_cache_response[request] = resp
             headers: CaseInsensitiveDict[str] = CaseInsensitiveDict(resp.headers)
 
             if "etag" in headers:
@@ -479,7 +481,7 @@ def update_cached_response(
         """
         assert request.url is not None
         cache_url = self.cache_url(request.url)
-        cached_response = self._load_from_cache(request)
+        cached_response = self._request_to_cache_response.get(request) or self._load_from_cache(request)
 
         if not cached_response:
             # we didn't have a cached response