Add Cloudflare Pages middleware to handle trailing slash redirects (#1631)

mdlinville · web-flow · commit 89c43b5ab92e · 2025-09-11T17:07:26.000-04:00
## Problem Cloudflare Pages has a limitation where: - `/wandb/config/` → redirects correctly to `/ref/python/sdk/functions/init/` (via _redirects) - `/wandb/config` → returns 404 (doesn't match the redirect rule) Recently, we discovered that this results in a Google Search Console report with over 1000 404s that never hit the Cloudflare redirects at all. ## Solution We've implemented a Cloudflare Pages Function middleware that automatically adds trailing slashes to URLs before they're processed by the redirect rules. ### How it works 1. The middleware intercepts all GET/HEAD requests 2. For URLs without trailing slashes (and not file extensions or system files): - It checks if adding a slash would match a redirect rule - If yes, it redirects to the URL with a trailing slash 3. The browser then follows the redirect and hits the actual redirect rule ### Example flow 1. User visits: `/wandb/config` 2. Middleware redirects to: `/wandb/config/` (301) 3. _redirects rule matches and redirects to: `/ref/python/sdk/functions/init/` (301) 4. Final destination served: `/ref/python/sdk/functions/init/` ## Implementation ### Files created/modified: 1. **`/functions/_middleware.js`** - The edge function that handles trailing slash redirects 2. **`/public/_routes.json`** - Configuration to ensure the middleware runs on all routes ## Tested Scenarios ### ✅ Working correctly: 1. **Redirect rules without trailing slashes** - `/wandb/config` → `/wandb/config/` → `/ref/python/sdk/functions/init/` - Preserves query parameters: `/wandb/config?test=param` → `/wandb/config/?test=param` 2. **Files with extensions (not redirected)** - `/wandb/config.html` → 404 (no redirect) - `/style.css` → served as-is - `/script.js` → served as-is 3. **Paths already with trailing slashes (passed through)** - `/index/` → processed normally - `/wandb/config/` → triggers redirect rule directly 4. **System files (not redirected)** - `/robots.txt` → served as-is - `/favicon.ico` → served as-is 5. **Directory paths** - `/guides` → `/guides/` (when directory exists) 6. **Special paths** - `/` → works correctly (already has slash) - `/path.with.dots` → processed normally (no common file extension) ### Edge cases handled: - Query parameters are preserved during redirect - Hash fragments are preserved during redirect - Only common file extensions skip the middleware (.html, .css, .js, etc.) - Paths with dots but uncommon extensions are processed normally ## Deployment Instructions The middleware will be automatically deployed to production when this PR is merged. These files are the key: - `/functions/_middleware.js` - The middleware code - `/public/_routes.json` - Routes configuration (optional, but recommended) ## Testing This has been tested locally by using Wrangler, since it's not so easy to test it in the Hugo layer. The middleware is deployed in this PR's preview build for easy testing. The Cloudflare build logs in this PR show: ``` 13:33:47.656 | Successfully read wrangler.toml file. -- | -- 13:33:47.657 | Found Functions directory at /functions. Uploading. 13:33:47.665 | ⛅️ wrangler 3.101.0 13:33:47.665 | ------------------- 13:33:48.705 | ✨ Compiled Worker successfully ``` Test these URLs without the trailing slash, and they should redirect as described: From | To ------|----- https://cloudflare-trailing-slash-ed.docodile.pages.dev/wandb/config | https://cloudflare-trailing-slash-ed.docodile.pages.dev/ref/python/sdk/functions/init/ https://cloudflare-trailing-slash-ed.docodile.pages.dev/library/init | https://cloudflare-trailing-slash-ed.docodile.pages.dev/ref/python/sdk/functions/init/ https://cloudflare-trailing-slash-ed.docodile.pages.dev/artifacts/api | https://cloudflare-trailing-slash-ed.docodile.pages.dev/ref/python/artifact/ https://cloudflare-trailing-slash-ed.docodile.pages.dev/sweeps/quickstart | https://cloudflare-trailing-slash-ed.docodile.pages.dev/guides/sweeps/walkthrough/ https://cloudflare-trailing-slash-ed.docodile.pages.dev/frameworks/pytorch | https://cloudflare-trailing-slash-ed.docodile.pages.dev/guides/integrations/pytorch/ https://cloudflare-trailing-slash-ed.docodile.pages.dev/library/hyperparameter-sweeps | https://cloudflare-trailing-slash-ed.docodile.pages.dev/guides/sweeps/ https://cloudflare-trailing-slash-ed.docodile.pages.dev/guides/self-hosted | https://cloudflare-trailing-slash-ed.docodile.pages.dev/guides/hosting/hosting-options/self-managed/ https://cloudflare-trailing-slash-ed.docodile.pages.dev/company/company-faq | https://cloudflare-trailing-slash-ed.docodile.pages.dev/support/ https://cloudflare-trailing-slash-ed.docodile.pages.dev/ref/data_types | https://cloudflare-trailing-slash-ed.docodile.pages.dev/ref/python/data-types/ https://cloudflare-trailing-slash-ed.docodile.pages.dev/weave/boards | https://cloudflare-trailing-slash-ed.docodile.pages.dev/guides/app/features/panels/query-panels/ https://cloudflare-trailing-slash-ed.docodile.pages.dev/wandb/config?test=param | https://cloudflare-trailing-slash-ed.docodile.pages.dev/ref/python/sdk/functions/init/?test=param (is a no-op but preserves the query parameter https://cloudflare-trailing-slash-ed.docodile.pages.dev/ref/release-notes/0.73#v0731 | https://cloudflare-trailing-slash-ed.docodile.pages.dev/ref/release-notes/0.73/#v0731 (preeserves the anchor) If you are observing in the browser inspector or the Cloudflare logs, each test should result in two redirects: 1. 301 redirect adding the trailing slash (example: `/wandb/config` → `/wandb/config/`) 2. 301 redirect to the final destination (example: `/wandb/config/` → `/ref/python/sdk/functions/init/`)
diff --git a/functions/README.md b/functions/README.md
@@ -0,0 +1,26 @@
+# Cloudflare Pages Functions
+
+This directory contains Cloudflare Pages Functions that enhance the functionality of the docs site.
+
+## Middleware
+
+### `_middleware.js` - Trailing Slash Redirect Handler
+
+This middleware addresses a known Cloudflare Pages issue where URLs without trailing slashes don't match redirect rules in the `_redirects` file.
+
+**Problem it solves:**
+- `/wandb/config` → 404 (doesn't match redirect rule)
+- `/wandb/config/` → redirects correctly
+
+**How it works:**
+1. Intercepts requests to URLs without trailing slashes
+2. Checks if adding a slash would match a redirect rule
+3. If yes, redirects to the URL with a trailing slash
+4. The browser then follows the actual redirect rule
+
+**Example:**
+- User visits: `/wandb/config`
+- Middleware redirects to: `/wandb/config/`
+- _redirects rule redirects to: `/ref/python/sdk/functions/init/`
+
+This is a temporary solution until the migration off Cloudflare Pages is completed.
diff --git a/functions/_middleware.js b/functions/_middleware.js
@@ -0,0 +1,95 @@
+/**
+ * Cloudflare Pages middleware to handle trailing slash redirects
+ * 
+ * This middleware addresses the known Cloudflare Pages issue where URLs without
+ * trailing slashes don't match redirect rules that include trailing slashes.
+ */
+
+export async function onRequest(context) {
+  const { request, env, next } = context;
+  const url = new URL(request.url);
+  
+  // Only process GET and HEAD requests
+  if (request.method !== 'GET' && request.method !== 'HEAD') {
+    return next();
+  }
+  
+  // Skip if the path already has a trailing slash
+  if (url.pathname.endsWith('/')) {
+    return next();
+  }
+  
+  // Skip if the path has a file extension
+  const hasFileExtension = /\.[a-zA-Z0-9]+$/.test(url.pathname);
+  if (hasFileExtension) {
+    return next();
+  }
+  
+  // Skip certain system paths
+  const skipPaths = ['/robots.txt', '/sitemap.xml', '/_redirects', '/favicon.ico'];
+  if (skipPaths.includes(url.pathname)) {
+    return next();
+  }
+  
+  // Skip paths with dots that might not be file extensions
+  // but could be version numbers or other valid path segments
+  // Only skip if the dot is followed by common file extensions
+  const commonExtensions = /\.(html|htm|css|js|json|xml|txt|pdf|jpg|jpeg|png|gif|svg|ico|woff|woff2|ttf|eot)$/i;
+  if (commonExtensions.test(url.pathname)) {
+    return next();
+  }
+  
+  // Create URL with trailing slash, preserving query and hash
+  const urlWithSlash = new URL(request.url);
+  urlWithSlash.pathname = url.pathname + '/';
+  
+  // Create a new request with the trailing slash
+  const requestWithSlash = new Request(urlWithSlash.toString(), {
+    method: request.method,
+    headers: request.headers,
+    redirect: 'manual' // Important: don't follow redirects
+  });
+  
+  try {
+    // Fetch with the trailing slash URL
+    const response = await env.ASSETS.fetch(requestWithSlash);
+    
+    // If we get a redirect response (301, 302, etc.), redirect to URL with slash
+    if (response.status >= 301 && response.status <= 308) {
+      // The path with trailing slash matches a redirect rule
+      // Redirect the browser to the URL with slash, which will then trigger the actual redirect
+      // Preserve query parameters and hash
+      const redirectUrl = new URL(request.url);
+      redirectUrl.pathname = url.pathname + '/';
+      
+      return new Response(null, {
+        status: 301,
+        headers: {
+          'Location': redirectUrl.pathname + redirectUrl.search + redirectUrl.hash,
+          'Cache-Control': 'public, max-age=3600'
+        }
+      });
+    }
+    
+    // If the response is successful (200), it means the trailing slash version exists
+    // In this case, we should also redirect to maintain consistency
+    if (response.status === 200) {
+      const redirectUrl = new URL(request.url);
+      redirectUrl.pathname = url.pathname + '/';
+      
+      return new Response(null, {
+        status: 301,
+        headers: {
+          'Location': redirectUrl.pathname + redirectUrl.search + redirectUrl.hash,
+          'Cache-Control': 'public, max-age=3600'
+        }
+      });
+    }
+  } catch (error) {
+    // Log error but continue with original request
+    console.error('Trailing slash middleware error:', error);
+  }
+  
+  // No redirect found, continue with the original request
+  return next();
+}
