v4: Potential 431 Header Too Large when prefetching dynamic content

[robcaldecott]

Checklist

• The issue can be reproduced in the nextjs-auth0 sample app (or N/A).
• I have looked into the Readme, Examples, and FAQ and have not found a suitable solution or answer.
• I have looked into the API documentation and have not found a suitable solution or answer.
• I have searched the issues and have not found a suitable solution or answer.
• I have searched the Auth0 Community forums and have not found a suitable solution or answer.
• I agree to the terms within the Auth0 Code of Conduct.

Description

We are using this toolkit for a flagship application and users are often seeing an issue where txn cookies are building up and are not deleted, eventually causing a 431 error. Once this happens the user cannot use the application until they clear cookies, which is not a great experience.

We are using @auth0/nextjs-auth0 v4.13.0 with NextJS 15.5.7 and have managed to reliably reproduce this issue when using <Link prefetch={true} /> with dynamic pages. Note that this has different behaviour to prefetch (which does not cause dynamic async server components to run.)

Here is what happens.

1. The user leaves their browser open with our app loaded. Eventually the session expires.
2. The user then hovers over links that are then prefetched, or scrolls the page to reveal links that are prefetched. These links cause the middleware to attempt to reload the auth0 session, but as prefetched data seems to be treated special by NextJS, the browser will not actually redirect. Each prefetched link will silently fail and each one will add a new txn cookie.
3. The user then clicks on a link and by now there are multiple txn cookies stored against the app. The middleware attempts to redirect to /auth/login and this flow eventually fails with a 431 error.
4. The user is now stuck until they clear cookies and refresh the page.

More Details
All our app pages are dynamic React Server Components and we use prefetch to load these in two ways:

• Using <Link prefetch={true} href="/page">Page Link</Link>
• Prefetching on hover using the NextJS example here.

When the auth0 session has expired (or we manually delete the session cookies to force it), the prefetching shows up in the Network tab like this:

Notice all the 307 and 404 requests, including CORS errors.

Every time one of these prefetching requests attempts is made, a new txn cookie is created and the application cookies ends up looking something like this:

When you then click on a link, our middleware.ts attempts to redirect to /auth/login and a 431 error occurs.

Our middleware.ts looks something like this:

export async function middleware(req: NextRequest) {
  const authResponse = await auth0.middleware(req);  
  if (req.nextUrl.pathname.startsWith("/auth")) {
    return authResponse;
  }
  const session = await auth0.getSession(req);
  if (!session) {
    return NextResponse.redirect(
      new URL(`/auth/login?returnTo=${req.nextUrl.pathname}`, req.url)
    );
  }
  try {
    await auth0.getAccessToken(req, authResponse);
  } catch (err) {
    if (err instanceof AccessTokenError) {
      return NextResponse.redirect(
        new URL(`/auth/login?returnTo=${req.nextUrl.pathname}`, req.url)
      );
    }
    // Let NextJS handle it
    throw err;
  }
}

export const config = {
  matcher: [
    {
      source:
        "/((?!_next/static|_next/image|favicon.ico|sitemap.xml|robots.txt|hc).*)",
    },
  ],
};

Pretty standard stuff.

To mitigate this issue, we are manually clearing the txn cookies based on advice here.

if (req.nextUrl.pathname === "/auth/login") {
  const cookieNames = req.cookies
    .getAll()
    .map((cookie) => cookie.name)
    .filter((name) => name.startsWith("__txn"));
  cookieNames.forEach((cookie) => {
    authResponse.cookies.delete(cookie);
  });
}

This seems to work but we don't know what consequences this will have.

We have also disabled use of prefetch={true} which has a significant performance impact on our app. Hopefully we can eventually figure out a way to disable prefetching if there is no auth0 session.

We have also regularly seen The state parameter is invalid errors which I suspect are related, but are near impossible to diagnose and reproduce. We make heavy use of async RSCs and are not in a position to switch to the auth0 React SPA toolkit as we're all in on auth0 server-side. Many of our RSCs require a valid access token to make API calls to our services, and refactoring this code to use CSR and data fetching is not an option.

Reproduction

It is possible to reproduce this issue relatively easily using a new NextJS 15 or 16 app.

Use the following middleware.ts (or refactor to proxy.ts). The auth0 client is standard stuff.

import { NextRequest, NextResponse } from "next/server";
import { auth0 } from "./lib/auth0";

export async function middleware(req: NextRequest) {
  const authResponse = await auth0.middleware(req);
  if (req.nextUrl.pathname.startsWith("/auth")) {
    return authResponse;
  }
  const session = await auth0.getSession(req);
  if (!session) {
    return NextResponse.redirect(
      new URL(`/auth/login?returnTo=${req.nextUrl.pathname}`, req.url)
    );
  }
  return authResponse;
}

export const config = {
  matcher: [
    "/((?!_next/static|_next/image|favicon.ico|sitemap.xml|robots.txt).*)",
  ],
};

Drop in a component called hover-prefetch-link.tsx

"use client";

import { useState } from "react";
import Link from "next/link";

export function HoverPrefetchLink(
  props: Omit<React.ComponentProps<typeof Link>, "prefetch">
) {
  const [active, setActive] = useState(false);

  return (
    <Link
      prefetch={active ? true : null}
      onMouseEnter={() => setActive(true)}
      {...props}
    />
  );
}

Opt into dynamic rendering by making the route layout.tsx an async component and drop this in:

export default async function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode;
}>) {
  // Make every page under this layout dynamic
  await cookies();

  return (
    <html lang="en">
      <body className={`${geistSans.variable} ${geistMono.variable}`}>
        {children}
      </body>
    </html>
  );
}

Then create some server component pages and link to them like this:

<HoverPrefetchLink href="/rsc1">Go to RSC Page 1</HoverPrefetchLink>

You need to use a production build (npm run build) to enable prefetching. Then, with the app running and logged in, manually delete the session cookies, force preloading using hover, etc. and watch the txn cookies multiply.

Additional context

No response

nextjs-auth0 version

4.13.0

Next.js version

15.x, 16.x

Node.js version

22

[nandan-bhat]

Hi @robcaldecott ,

Thanks for putting time creating detailed report and the repro steps :).

At the moment, we don’t have an immediate solution for this. Reliably detecting prefetch requests (especially those triggered by hover or automatic preloading) is not straightforward in Next.js, and there isn’t a clean, documented signal we can safely rely on today.

There are a few possible workarounds, but they are either fragile or not aligned with Next.js recommendations. We don’t want add something to the SDK that could break with future Next.js changes. But I totally agree with the issue you described and with the behavior you explained.

We’re aware of this issue and are looking into what we can reasonably handle on our side, at least for the auth/* endpoints, once we have a solid and safe approach.