feat(rsc): support observing imports/exports during build scan

[james-elicx]

background (by me):

I basically need a way to be able to track which server functions are consumed by which files, and then build up a manifest of the routes that consume them and then gate calling server actions behind the routes that consume them.

I was originally thinking I'd just parse or lex in vinext and use that to build the manifest. the only problem with that though, is that it could get expensive if we're going over every module, and it ideally needs to be slotted into the right place in-between rsc's plugins in order to build up the right map.

When looking further, the import/export info that i'd need to do this is already obtained with es-module-lexer here, at the point where i'd ideally want to obtain it.

so that led to experimenting with the idea of being able to observe modules being parsed and feed the imports/exports that are discovered during the scan step back to the framework.

is this the kind of thing you'd be open to adding here @hi-ogawa?

for us, this ends up looking like:

1. vinext observes imports/exports with the rsc plugin's new scan observers, and records the import/export relationships between module ids
2. rsc plugin identifies server actions and provides serverReferenceMetaMap with module ids and actions exported
3. vinext computes which action ids each module consumes from the observed relationships.
4. vinext walks through the imported module ids from rolldown's module info for each route to discover the reachable modules for a route
5. vinext uses the observed relationships and the discovered modules to determine which exported action ids are consumed within each route.

---

codex:

Description

Expose opt-in scan-build observers from the RSC plugin manager so framework integrations can reuse plugin-RSC's existing es-module-lexer pass and Rollup ModuleInfo instead of performing a second source crawl.

Observers receive:

• a reset event for each scan environment build
• the original module source
• the existing lexer import/export metadata
• the resolved Rollup module information
• the environment name

The scan path does no additional bookkeeping when no observers are registered. The implementation also deduplicates module events across plugin-RSC's existing shared scan plugin instances.

This additionally records inlineExportNames in server reference metadata. Consumers can distinguish generated inline server-action exports from module-level 'use server' exports without inferring that distinction from generated names.

The immediate consumer is Cloudflare vinext, which uses this data to derive server-action ownership during the existing RSC scans rather than reparsing and traversing the application afterward.

Tests cover:

• unchanged scan stripping behavior
• raw lexer callback metadata
• no observer events outside scan builds
• reset and module event ordering
• multiple observer consistency
• raw source, import/export, environment, and ModuleInfo fidelity
• deduplication across shared scan plugin instances
• inline versus module-level server reference metadata

[hi-ogawa]

Do you have a reference on Next.js (perhaps their e2e)?
The motivation of reusing scan lexer sounds reasonable to me even though such server function protection mechanism is technically an out of scope of bare React RSC convention.
To have better understanding of the framework side mechanism, can you whip up entirely new example and e2e? I'd probably prefer bigger vibe coded rsc app for demo than surgical unit test since unit test doesn't express rsc app motivation and makes it hard to discuss the alternatives as it locks internal wiring 😃

[james-elicx]

Do you have a reference on Next.js (perhaps their e2e)? The motivation of reusing scan lexer sounds reasonable to me even though such server function protection mechanism is technically an out of scope of bare React RSC convention. To have better understanding of the framework side mechanism, can you whip up entirely new example and e2e? I'd probably prefer bigger vibe coded rsc app for demo than surgical unit test since unit test doesn't express rsc app motivation and makes it hard to discuss the alternatives as it locks internal wiring 😃

I can point you to the source code where they do some of the stuff, not sure if it's explicitly covered in an e2e though - https://github.com/vercel/next.js/blob/9097ef6f8ae83d9472533cc4e4c8d59021543990/packages/next/src/server/app-render/manifests-singleton.ts#L246-L272

Here, they're checking if the manifest for the route's worker has the action, and if it doesn't, the post request gets forwarded on to one that does own the action.

I started a POC in vinext using these changes here - cloudflare/vinext#2520

But there's a decent bit of work going on there to make it all work 😅
The action-owner-manifest.ts‎ file is the one where the bulk of the work is happening using this change

[hi-ogawa]

Here, they're checking if the manifest for the route's worker has the action, and if it doesn't, the post request gets forwarded on to one that does own the action.

Thanks for the pointer. That's helpful. I think there's still something unclear and what selectWorkerForForwarding is for at high level product feature (i.e. ideally something visible to users, thus something exercised as e2e).
If this is more tangled with server side code/bundle splitting, I think it does motivates vite rsc plugin to support such paradigm of server action manifest. It may not be fully framework agnostic from the start, but just some bits and pieces to make it possible for framework.

[james-elicx]

Here, they're checking if the manifest for the route's worker has the action, and if it doesn't, the post request gets forwarded on to one that does own the action.

Thanks for the pointer. That's helpful. I think there's still something unclear and what selectWorkerForForwarding is for at high level product feature (i.e. ideally something visible to users, thus something exercised as e2e). If this is more tangled with server side code/bundle splitting, I think it does motivates vite rsc plugin to support such paradigm of server action manifest. It may not be fully framework agnostic from the start, but just some bits and pieces to make it possible for framework.

I've found this one where where it tests that the server action requests gets forwarded to somewhere it's registered - https://github.com/vercel/next.js/blob/9097ef6f8ae83d9472533cc4e4c8d59021543990/test/e2e/app-dir/actions/app-action.test.ts#L899-L937

Are you thinking it might be worth building this manifest in here instead of in the framework?

I can see the motivation behind that and it could make sense for advanced use cases, e.g. splitting each route into an independent entrypoint / bundle and having a separate router in front of them all that forward requests to where they live.

If we want to go in that direction, is there anything I can do to help there?

[hi-ogawa]

Here, they're checking if the manifest for the route's worker has the action, and if it doesn't, the post request gets forwarded on to one that does own the action.

Thanks for the pointer. That's helpful. I think there's still something unclear and what selectWorkerForForwarding is for at high level product feature (i.e. ideally something visible to users, thus something exercised as e2e). If this is more tangled with server side code/bundle splitting, I think it does motivates vite rsc plugin to support such paradigm of server action manifest. It may not be fully framework agnostic from the start, but just some bits and pieces to make it possible for framework.

I've found this one where where it tests that the server action requests gets forwarded to somewhere it's registered - https://github.com/vercel/next.js/blob/9097ef6f8ae83d9472533cc4e4c8d59021543990/test/e2e/app-dir/actions/app-action.test.ts#L899-L937

Are you thinking it might be worth building this manifest in here instead of in the framework?

I can see the motivation behind that and it could make sense for advanced use cases, e.g. splitting each route into an independent entrypoint / bundle and having a separate router in front of them all that forward requests to where they live.

If we want to go in that direction, is there anything I can do to help there?

Yes, that's basically what I'm thinking in terms of scope, but I do worry that the complexity of course (and also the concern of my commitment if there's more work to have in rsc plugin). So, it's totally fine to start with what you have in mind currently. Anyways, I appreciate you sharing the higher level context with me. 🙏

Side question, does Vinext currently have architecture to benefit from such multi workers and forwarding? I don't have much mental model of what Next.js exactly benefit from this either yet though. Is this for them to deploy separate server lighter bundle per route per route? or is this also about their edge/node split?