Parallelize initial page cache population

[FabianKoder]

Introduction

We currently have thousands of statically built pages that need to be populated into the cache handler on startup. This takes about 30-40 seconds in our production instance. This change tries to reduce the time spent registering the initial cache by parallelizing the route population.

Summary

Populate page-cache in parallel instead of sequentially.

Old behavior
Previously, all routes were set one-by-one.

New behavior
Up to os.availableParallelism() pages are now concurrently read from the filesystem and populated into the cache handler.

Changes

• Parallelize initial cache hydration by wrapping route population in p-limit with os.availableParallelism() and awaiting Promise.all, so initial routes/pages are processed concurrently.
• Introduce a promise-backed latch on CacheHandler that guarantees onCreation runs exactly once; #ensureConfigured() now serializes calls while still letting the main set/get/revalidateTag logic run in parallel.
• Preserve previous behavior by resetting the latch if initialization fails, and retain the existing merged-handler guard so repeated calls after the first simply fall through and create a debug log message.

---

OSS @durchblicker

[FabianKoder]

Additional remarks: Maybe it makes sense to add a min parallelism of say 4 or 8 instead of fully relying on os.availableParallelism (). Because most next containers or for example Vercel use 1 vCPU which would probably result in a limit of 1 on prod instances - since we are not really limited by CPU performance but rather I/O and the cache handler itself, it is not the best metric actually.

[FabianKoder]

Hey @AyronK, any updates? :) (+ opinion on minimum parallelism count)

[AyronK]

@FabianKoder cool idea, I will test it today and get back to you.

[AyronK]

Additional remarks: Maybe it makes sense to add a min parallelism of say 4 or 8 instead of fully relying on os.availableParallelism (). Because most next containers or for example Vercel use 1 vCPU which would probably result in a limit of 1 on prod instances - since we are not really limited by CPU performance but rather I/O and the cache handler itself, it is not the best metric actually.

I think it's a solid idea @FabianKoder, I would go with 4 as default but you could introduce a parameter in instrumentation config (RegisterInitialCacheOptions).

[AyronK]

Very good addition, please consider adding a config for minimum parallelism and a default value (4 for example). Tested and works fine 👌🏻

[FabianKoder]

I added a respective config option :)