Skip to content

feat: allow section-based Surrogate-Key to purge chunks#2851

Merged
JacobCoffee merged 7 commits intomainfrom
surrogate-key-for-rleeases
Feb 19, 2026
Merged

feat: allow section-based Surrogate-Key to purge chunks#2851
JacobCoffee merged 7 commits intomainfrom
surrogate-key-for-rleeases

Conversation

@JacobCoffee
Copy link
Copy Markdown
Member

@JacobCoffee JacobCoffee commented Dec 16, 2025

Description

When working with release changes lately we've needed to purge all releases because cache is wonky with stale CSS.

fastly ui and api doesnt seem to let me just do "root path, also purge all children" but maybe im missing it and we dont need this

Had to do something like this (which notably still missing pages like /downloads/android/ and others..):

Details
while read url; do
  [ -n "$url" ] && curl -s -X PURGE "https://www.python.org$url" -H "Fastly-Key: $FASTLY_API_KEY" &
done < "releases.txt"
wait
echo "Done - purged $(wc -l < "releases.sh") URLs"

against releases.txt:

/downloads/release/python-2716/
/downloads/release/python-230/
/downloads/release/pymanager-252/
/downloads/release/python-3141/
/downloads/release/python-2714/
/downloads/release/python-2717/
/downloads/release/python-2712/
/downloads/release/python-2711rc1/
/downloads/release/python-278/
/downloads/release/python-2711/
/downloads/release/python-2717rc1/
/downloads/release/python-2710/
/downloads/release/python-241/
/downloads/release/python-245/
/downloads/release/python-240/
/downloads/release/python-244/
/downloads/release/python-243/
/downloads/release/python-242/
/downloads/release/pymanager-250/
/downloads/release/python-2713rc1/
/downloads/release/python-3150a3/
/downloads/release/python-2718rc1/
/downloads/release/python-262/
/downloads/release/python-2718/
/downloads/release/python-279rc1/
/downloads/release/python-263/
/downloads/release/python-264/
/downloads/release/python-265/
/downloads/release/python-266/
/downloads/release/python-268/
/downloads/release/python-269/
/downloads/release/python-3137/
/downloads/release/python-246/
/downloads/release/python-250/
/downloads/release/python-251/
/downloads/release/python-3135/
/downloads/release/python-3100b2/
/downloads/release/python-3100b1/
/downloads/release/python-233/
/downloads/release/python-232/
/downloads/release/python-231/
/downloads/release/python-2715/
/downloads/release/python-275/
/downloads/release/python-276/
/downloads/release/python-277/
/downloads/release/python-277rc1/
/downloads/release/python-260/
/downloads/release/python-261/
/downloads/release/python-267/
/downloads/release/python-279/
/downloads/release/python-2712rc1/
/downloads/release/python-2714rc1/
/downloads/release/python-223/
/downloads/release/python-201/
/downloads/release/python-237/
/downloads/release/python-252/
/downloads/release/python-253/
/downloads/release/python-254/
/downloads/release/python-272/
/downloads/release/python-273/
/downloads/release/python-274/
/downloads/release/python-255/
/downloads/release/python-270/
/downloads/release/python-271/
/downloads/release/python-337rc1/
/downloads/release/python-335-rc1/
/downloads/release/python-3120a2/
/downloads/release/python-335rc1/
/downloads/release/python-2715rc1/
/downloads/release/python-2710rc1/
/downloads/release/python-222/
/downloads/release/python-221/
/downloads/release/python-3820/
/downloads/release/python-2716rc1/
/downloads/release/python-2713/
/downloads/release/pymanager-250b13/
/downloads/release/python-3140a2/
/downloads/release/python-314/
/downloads/release/python-322/
/downloads/release/python-213/
/downloads/release/python-220/
/downloads/release/python-235/
/downloads/release/python-31311/
/downloads/release/python-234/
/downloads/release/python-344/
/downloads/release/python-236/
/downloads/release/python-3142/
/downloads/release/python-3140rc3/
/downloads/release/python-256/
/downloads/release/python-3140b3/
/downloads/release/python-3140rc2/
/downloads/release/python-399/
/downloads/release/python-3130b1/
/downloads/release/python-31211/
/downloads/release/pymanager-251/
/downloads/release/python-3612/
/downloads/release/python-3134/
/downloads/release/pymanager-250b14/
/downloads/release/python-3130b2/
/downloads/release/python-384/
/downloads/release/python-396/
/downloads/release/python-380b1/
/downloads/release/python-31210/
/downloads/release/python-3920/
/downloads/release/python-3130a5/
/downloads/release/pymanager-250b10/
/downloads/release/pymanager-250b9/
/downloads/release/python-3911/
/downloads/release/python-31016/
/downloads/release/python-3140a3/
/downloads/release/python-350b2/
/downloads/release/python-3136/
/downloads/release/python-3128/
/downloads/release/python-383/
/downloads/release/pymanager-250b15/
/downloads/release/python-3130a2/
/downloads/release/python-351/
/downloads/release/python-3140/
/downloads/release/pymanager-250b12/
/downloads/release/python-3140b1/
/downloads/release/python-3130/
/downloads/release/python-3100a1/
/downloads/release/python-3812/
/downloads/release/python-3915/
/downloads/release/python-3913/
/downloads/release/python-3121/
/downloads/release/python-3140b2/
/downloads/release/python-3131/
/downloads/release/python-3613/
/downloads/release/python-3132/
/downloads/release/python-3118/
/downloads/release/python-353/
/downloads/release/python-3102/
/downloads/release/python-3117/
/downloads/release/python-352rc1/
/downloads/release/python-368rc1/
/downloads/release/python-3140b4/
/downloads/release/python-3138/
/downloads/release/python-31019/
/downloads/release/python-3810/
/downloads/release/python-370/
/downloads/release/python-360/
/downloads/release/python-362rc2/
/downloads/release/python-31012/
/downloads/release/python-3130a1/
/downloads/release/python-3110rc1/
/downloads/release/python-3120a5/
/downloads/release/python-3100a6/
/downloads/release/python-378/
/downloads/release/python-348rc1/
/downloads/release/python-3108/
/downloads/release/python-378rc1/
/downloads/release/python-377/
/downloads/release/python-3130b3/
/downloads/release/python-350a2/
/downloads/release/python-3130rc3/
/downloads/release/python-390b1/
/downloads/release/python-380b4/
/downloads/release/python-358rc2/
/downloads/release/python-3133/
/downloads/release/python-377rc1/
/downloads/release/python-368/
/downloads/release/python-3104/
/downloads/release/python-360a3/
/downloads/release/python-3150a1/
/downloads/release/pymanager-250b11/
/downloads/release/python-350a1/
/downloads/release/python-31114/
/downloads/release/python-31011/
/downloads/release/python-3110a7/
/downloads/release/python-3120b3/
/downloads/release/python-388rc1/
/downloads/release/python-3120a7/
/downloads/release/python-3110rc2/
/downloads/release/pymanager-251b1/
/downloads/release/python-3116/
/downloads/release/python-390a3/
/downloads/release/python-360b1/
/downloads/release/python-3110b2/
/downloads/release/python-373/
/downloads/release/python-370b1/
/downloads/release/python-3714/
/downloads/release/python-3124/
/downloads/release/python-360a1/
/downloads/release/python-3410/
/downloads/release/pymanager-251b2/
/downloads/release/python-3814/
/downloads/release/python-346rc1/
/downloads/release/python-3130b4/
/downloads/release/python-312/
/downloads/release/python-3100b3/
/downloads/release/python-365rc1/
/downloads/release/python-3100rc2/
/downloads/release/python-3100/
/downloads/release/python-370a2/
/downloads/release/python-3615/
/downloads/release/python-345/
/downloads/release/python-3120a1/
/downloads/release/python-341rc1/
/downloads/release/python-351rc1/
/downloads/release/python-31212/
/downloads/release/python-3109/
/downloads/release/python-3130rc1/
/downloads/release/python-3100rc1/
/downloads/release/python-3120rc3/
/downloads/release/python-350rc2/
/downloads/release/python-3914/
/downloads/release/python-310/
/downloads/release/python-3110a1/
/downloads/release/python-3126/
/downloads/release/python-3924/
/downloads/release/python-3140a1/
/downloads/release/python-3110a2/
/downloads/release/python-31310/
/downloads/release/python-360a4/
/downloads/release/python-3119/
/downloads/release/python-379/
/downloads/release/python-3919/
/downloads/release/python-3120a3/
/downloads/release/python-342/
/downloads/release/python-3110a3/
/downloads/release/python-346/
/downloads/release/python-3110a4/
/downloads/release/python-388/
/downloads/release/python-3139/
/downloads/release/python-350b3/
/downloads/release/python-356rc1/
/downloads/release/python-3100a5/
/downloads/release/python-3816/
/downloads/release/python-3130rc2/
/downloads/release/python-3112/
/downloads/release/python-360rc2/
/downloads/release/python-366/
/downloads/release/python-335rc2/
/downloads/release/python-3510/
/downloads/release/python-3105/
/downloads/release/python-358/
/downloads/release/python-372rc1/
/downloads/release/python-371rc2/
/downloads/release/python-343/
/downloads/release/python-355/
/downloads/release/python-350b1/
/downloads/release/python-3129/
/downloads/release/python-324/
/downloads/release/python-390b4/
/downloads/release/python-3130a4/
/downloads/release/python-370a3/
/downloads/release/python-321/
/downloads/release/python-347rc1/
/downloads/release/python-3103/
/downloads/release/python-391/
/downloads/release/python-340rc3/
/downloads/release/python-397/
/downloads/release/python-325/
/downloads/release/python-360b4/
/downloads/release/python-31110/
/downloads/release/python-31113/
/downloads/release/python-3110b4/
/downloads/release/python-313/
/downloads/release/python-311/
/downloads/release/python-367rc2/
/downloads/release/python-337/
/downloads/release/python-370b5/
/downloads/release/python-343rc1/
/downloads/release/python-3819/
/downloads/release/python-3710/
/downloads/release/python-350rc3/
/downloads/release/python-380a1/
/downloads/release/python-360a2/
/downloads/release/python-370a4/
/downloads/release/python-382/
/downloads/release/python-376/
/downloads/release/python-352/
/downloads/release/python-3713/
/downloads/release/python-354rc1/
/downloads/release/python-3110b1/
/downloads/release/python-390rc2/
/downloads/release/python-3610rc1/
/downloads/release/python-3716/
/downloads/release/python-31112/
/downloads/release/python-392/
/downloads/release/python-390b5/
/downloads/release/python-3912/
/downloads/release/python-362/
/downloads/release/python-31013/
/downloads/release/python-350b4/
/downloads/release/python-336rc1/
/downloads/release/python-387rc1/
/downloads/release/python-31017/
/downloads/release/python-3125/
/downloads/release/python-390a6/
/downloads/release/python-370b2/
/downloads/release/python-3100a4/
/downloads/release/python-365/
/downloads/release/python-3107/
/downloads/release/python-335/
/downloads/release/python-3110b5/
/downloads/release/python-3614/
/downloads/release/python-390a4/
/downloads/release/python-301/
/downloads/release/python-390rc1/
/downloads/release/python-395/
/downloads/release/python-3120a6/
/downloads/release/python-3610/
/downloads/release/python-370rc1/
/downloads/release/python-331/
/downloads/release/python-3925/
/downloads/release/python-3120rc2/
/downloads/release/python-3101/
/downloads/release/python-361/
/downloads/release/python-372/
/downloads/release/python-3110a5/
/downloads/release/python-394/
/downloads/release/python-336/
/downloads/release/python-3813/
/downloads/release/python-3122/
/downloads/release/python-3120b4/
/downloads/release/python-358rc1/
/downloads/release/python-382rc1/
/downloads/release/python-320/
/downloads/release/python-380rc1/
/downloads/release/python-374/
/downloads/release/python-3715/
/downloads/release/python-371rc1/
/downloads/release/python-381rc1/
/downloads/release/python-359/
/downloads/release/python-384rc1/
/downloads/release/python-3130a6/
/downloads/release/python-381/
/downloads/release/python-380b3/
/downloads/release/python-354/
/downloads/release/python-392rc1/
/downloads/release/python-364/
/downloads/release/python-326/
/downloads/release/python-333/
/downloads/release/python-390a5/
/downloads/release/python-364rc1/
/downloads/release/python-3110b3/
/downloads/release/python-363rc1/
/downloads/release/python-361rc1/
/downloads/release/python-386/
/downloads/release/python-3110a6/
/downloads/release/python-362rc1/
/downloads/release/python-3115/
/downloads/release/python-332/
/downloads/release/python-3100b4/
/downloads/release/python-31010/
/downloads/release/python-371/
/downloads/release/python-375/
/downloads/release/python-3140a6/
/downloads/release/python-380a3/
/downloads/release/python-3811/
/downloads/release/python-380b2/
/downloads/release/python-369rc1/
/downloads/release/python-3717/
/downloads/release/python-342rc1/
/downloads/release/python-349/
/downloads/release/python-380/
/downloads/release/python-3711/
/downloads/release/python-380a4/
/downloads/release/python-31015/
/downloads/release/python-369/
/downloads/release/python-383rc1/
/downloads/release/python-391rc1/
/downloads/release/python-3120/
/downloads/release/python-3140rc1/
/downloads/release/python-3916/
/downloads/release/python-3510rc1/
/downloads/release/python-3120b1/
/downloads/release/python-344rc1/
/downloads/release/python-360rc1/
/downloads/release/python-31014/
/downloads/release/python-370a1/
/downloads/release/python-353rc1/
/downloads/release/python-3922/
/downloads/release/python-3817/
/downloads/release/python-347/
/downloads/release/python-3917/
/downloads/release/python-3130a3/
/downloads/release/python-385/
/downloads/release/python-360b3/
/downloads/release/python-355rc1/
/downloads/release/python-389/
/downloads/release/python-3120rc1/
/downloads/release/python-3100a2/
/downloads/release/python-3815/
/downloads/release/python-390a2/
/downloads/release/python-367rc1/
/downloads/release/python-360b2/
/downloads/release/python-315/
/downloads/release/python-3114/
/downloads/release/python-382rc2/
/downloads/release/python-3918/
/downloads/release/python-3110/
/downloads/release/python-3921/
/downloads/release/python-373rc1/
/downloads/release/python-380a2/
/downloads/release/python-334/
/downloads/release/python-390b2/
/downloads/release/python-390/
/downloads/release/python-376rc1/
/downloads/release/python-348/
/downloads/release/python-3923/
/downloads/release/python-3100a7/
/downloads/release/python-300/
/downloads/release/python-3150a2/
/downloads/release/python-3140a5/
/downloads/release/python-3113/
/downloads/release/python-363/
/downloads/release/python-3140a4/
/downloads/release/python-340/
/downloads/release/python-350a3/
/downloads/release/python-356/
/downloads/release/python-386rc1/
/downloads/release/python-330/
/downloads/release/python-350rc4/
/downloads/release/python-3611/
/downloads/release/python-3611rc1/
/downloads/release/python-3140a7/
/downloads/release/python-3111/
/downloads/release/python-3123/
/downloads/release/python-326rc1/
/downloads/release/python-390a1/
/downloads/release/python-3120b2/
/downloads/release/python-374rc1/
/downloads/release/python-366rc1/
/downloads/release/python-3910/
/downloads/release/python-375rc1/
/downloads/release/python-3100a3/
/downloads/release/python-323/
/downloads/release/python-367/
/downloads/release/python-393/
/downloads/release/python-3127/
/downloads/release/python-3712/
/downloads/release/python-350a4/
/downloads/release/python-3818/
/downloads/release/python-398/
/downloads/release/python-3410rc1/
/downloads/release/python-349rc1/
/downloads/release/python-357/
/downloads/release/python-341/
/downloads/release/python-357rc1/
/downloads/release/python-3106/
/downloads/release/python-387/
/downloads/release/python-31018/
/downloads/release/python-390b3/
/downloads/release/python-3120a4/
/downloads/release/python-350/
/downloads/release/python-31111/
/downloads/release/python-345rc1/
/downloads/release/python-350rc1/

but ideally we just purge all of downloads/ when needed

Comment thread pydotorg/middleware.py Outdated
@JacobCoffee JacobCoffee requested a review from hugovk December 16, 2025 16:48
hugovk
hugovk previously approved these changes Dec 16, 2025
Comment thread .github/workflows/purge-cache.yml
corneliusroemer added a commit to corneliusroemer/pythondotorg that referenced this pull request Jan 4, 2026
… cause

The “Please turn on JavaScript” notice can be shown even when JavaScript
is enabled, e.g. due to intermittent asset loading or caching issues
(python#2324, python#2851).

This change updates the copy to explain why the notice is shown and to
list possible causes without asserting that JavaScript is disabled on
the client.
Copilot AI review requested due to automatic review settings February 19, 2026 02:20
@JacobCoffee JacobCoffee force-pushed the surrogate-key-for-rleeases branch from 974c458 to f1d6aea Compare February 19, 2026 02:20
Comment thread .github/workflows/purge-cache.yml Fixed
@JacobCoffee JacobCoffee force-pushed the surrogate-key-for-rleeases branch 2 times, most recently from f3b96dd to f0ebb7a Compare February 19, 2026 02:23
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request implements section-based surrogate key purging for Fastly CDN, enabling more efficient cache invalidation by purging entire sections (e.g., /downloads/*) in a single API request rather than individual URLs. This significantly simplifies cache management when deploying changes that affect multiple pages.

Changes:

  • Added purge_surrogate_key() utility function to purge Fastly cache by surrogate key
  • Enhanced GlobalSurrogateKey middleware to add section-based surrogate keys derived from URL paths
  • Simplified Release post_save signal to use surrogate key purging instead of individual URL purges
  • Added GitHub Actions workflow for manual/automated cache purging via surrogate keys

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
pydotorg/middleware.py Enhanced middleware to extract and append section keys (e.g., "downloads", "events") from URL paths to the Surrogate-Key header
pydotorg/tests/test_middleware.py Added comprehensive tests for section key extraction and Surrogate-Key header generation
fastly/utils.py Added purge_surrogate_key() function to purge all cached content with a specific surrogate key via Fastly API
apps/downloads/models.py Refactored purge_fastly_download_pages to use surrogate key purging instead of individual URL purges
pydotorg/settings/base.py Added FASTLY_SERVICE_ID configuration required for surrogate key API endpoint
.github/workflows/purge-cache.yml Created workflow to manually/automatically purge cache sections on template/static file changes
Comments suppressed due to low confidence (1)

fastly/utils.py:49

  • The new purge_surrogate_key function lacks test coverage. The codebase has test patterns for mocking external API calls (see apps/downloads/tests/test_template_tags.py for examples using @mock.patch on requests.get). Consider adding tests to verify that purge_surrogate_key correctly calls the Fastly API with the right parameters, handles missing configuration, and respects the DEBUG setting.
def purge_surrogate_key(key):
    """Purge all Fastly cached content tagged with a surrogate key.

    Common keys (set by GlobalSurrogateKey middleware):
        - 'pydotorg-app': Purges entire site
        - 'downloads': Purges all /downloads/* pages
        - 'events': Purges all /events/* pages
        - 'sponsors': Purges all /sponsors/* pages
        - etc. (first path segment becomes the surrogate key)

    Returns the response from Fastly API, or None if not configured.
    """
    if settings.DEBUG:
        return None

    api_key = getattr(settings, "FASTLY_API_KEY", None)
    service_id = getattr(settings, "FASTLY_SERVICE_ID", None)
    if not api_key or not service_id:
        return None

    return requests.post(
        f"https://api.fastly.com/service/{service_id}/purge/{key}",
        headers={"Fastly-Key": api_key},
        timeout=30,
    )


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread fastly/utils.py
Comment thread apps/downloads/models.py Outdated
Comment thread .github/workflows/purge-cache.yml Outdated
Comment thread pydotorg/tests/test_middleware.py Outdated
Comment thread pydotorg/tests/test_middleware.py Outdated
@JacobCoffee JacobCoffee force-pushed the surrogate-key-for-rleeases branch from f0ebb7a to d02f8f8 Compare February 19, 2026 02:25
Copilot AI review requested due to automatic review settings February 19, 2026 02:27
@JacobCoffee JacobCoffee force-pushed the surrogate-key-for-rleeases branch from d02f8f8 to 7d5ba7b Compare February 19, 2026 02:27
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apps/downloads/models.py Outdated
Comment thread pydotorg/tests/test_middleware.py Outdated
Comment thread pydotorg/middleware.py
JacobCoffee and others added 2 commits February 18, 2026 20:37
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevents silent no-op regression when surrogate key purging
is not available.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@JacobCoffee JacobCoffee merged commit 40cb9cf into main Feb 19, 2026
14 checks passed
@JacobCoffee JacobCoffee deleted the surrogate-key-for-rleeases branch February 19, 2026 03:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants