Add compiler plugin hooks and plugins and move compilation pipeline out of App#6260
Add compiler plugin hooks and plugins and move compilation pipeline out of App#6260FarhanAliRaza wants to merge 25 commits intoreflex-dev:mainfrom
Conversation
Greptile SummaryThis PR lays the plugin-architecture foundations for single-pass page compilation in Reflex, introducing Key changes:
Findings:
Confidence Score: 5/5Safe to merge β all remaining findings are P2 style and documentation suggestions with no impact on runtime correctness. The core plugin machinery is well-designed and the test coverage is thorough. All three findings are P2: a missing inline comment on the intentional prop-component side-effect-only traversal, a defensive assertion that would improve error messages for malformed plugin generators (not triggerable by correctly-typed plugins), and duplicated test helper logic. None affect correctness or reliability. reflex/compiler/plugins.py β the prop-component traversal asymmetry and the unwind-loop None-children path are worth a second look before the plugin API is widely adopted. Important Files Changed
Sequence DiagramsequenceDiagram
participant CC as CompileContext.compile()
participant CH as CompilerHooks
participant PC as PageContext
participant P as Plugin (ordered chain)
CC->>CH: eval_page(page_fn) [stop-on-first]
CH->>P: eval_page() β first plugin returning non-None wins
P-->>CH: PageContext
CH-->>CC: PageContext
CC->>PC: async with page_ctx (attach ContextVar)
loop For each structural child
CC->>CH: compile_component(comp)
CH->>P: plugin.compile_component() β AsyncGenerator
Note over P: PRE: anext(gen) β enter phase (registration order)
CH->>CH: _compile_children(structural_children) [recursive]
CH->>CH: compile_component(prop_components) [side-effects only]
Note over P: POST: gen.asend((comp, children)) β unwind (reverse order)
CH-->>CC: compiled_component
end
CC->>CH: compile_page(page_ctx)
CH->>P: compile_page() β all plugins, registration order
P-->>CH: (void)
CC->>PC: exit async with (detach ContextVar)
CC->>CC: compiled_pages[route] = page_ctx
Reviews (2): Last reviewed commit: "removed the extra stuff no related to is..." | Re-trigger Greptile |
Merging this PR will improve performance by Γ11
|
masenf
left a comment
There was a problem hiding this comment.
we need a test case for the child replacement logic when traversing the tree; we might not strictly need it now, but it will be important when moving the StatefulComponent compilation into the plugin system.
|
Khaleel and I were discussing this a bit further, and I think it would be better to add these new plugin hooks and hook dispatching system to the existing Plugin base in |
Move the frontend compilation pipeline from App._compile into compiler.compile_app(), introducing a CompilerPlugin protocol with enter_component/leave_component/eval_page/compile_page hooks. Remove the ExecutorType/ExecutorSafeFunctions abstractions in favor of a sequential plugin-driven compilation model.
ac4af22 to
15f363b
Compare
Move memo component compilation after app_root resolution so app-wrap components are included. Fix DefaultPagePlugin to preserve Var-backed titles instead of replacing them with the default string.
Move _get_app_wrap_components collection outside the `if stateful_component is None` guard so that app wrap components (e.g. UploadFilesProvider) are collected even when a component is wrapped as a stateful component. Add test verifying upload pages correctly emit UploadFilesProvider in the app root.
|
@masenf i dont know why the tests are failing. |
Remove the class-level tag_to_stateful_component dict and instead thread a compile-scoped stateful_component_cache through compile_from() and create(). This prevents stale cache entries from leaking between independent compilation runs. Also collect imports and app_wrap_components from root components in CompileContext so stateful component libraries and providers (e.g. UploadFilesProvider) are properly propagated. Update benchmarks to inline helpers using the new plugin API and add tests covering shared stateful components across pages and cache isolation between runs.
Component._get_vars had a dead-code cache path: `getattr(self, "__vars", None)` reads the literal attribute `__vars` but `self.__vars = []` writes to the name-mangled `_Component__vars`. The cache branch was never taken, and even if the name-mangling were fixed the missing `return` after `yield from vars` would have caused duplicate yields on repeated calls. Fix the cache (as `_vars_cache`) with a proper early-return. Extend the same per-instance cache pattern to `_get_imports` and `_get_hooks_internal`, which share the same dependency on `event_triggers` / `_get_vars`. Unify invalidation with the existing render-cache clear in `StatefulComponent.create` so all four caches drop together when `_fix_event_triggers` mutates the component.
| Returns: | ||
| A page context when the plugin can evaluate the page, otherwise ``None``. | ||
| """ | ||
| del page_fn, kwargs |
There was a problem hiding this comment.
why does it del the local vars?
There was a problem hiding this comment.
They are gonna be overloaded by the actual plugin. So Claude is i think, just trying to use it. Deleting it.
| ] | ||
|
|
||
|
|
||
| class CompilerPlugin(Protocol): |
There was a problem hiding this comment.
is this protocol needed since we have the base plugin definition?
|
|
||
|
|
||
| @dataclasses.dataclass(kw_only=True) | ||
| class BaseContext: |
There was a problem hiding this comment.
this class also gets defined in my backend event loop PR as reflex_base.context.base depending on whose merges first, we can refactor
| if isinstance(page_ctx.root_component, StatefulComponent): | ||
| self.all_imports = merge_imports( | ||
| self.all_imports, | ||
| page_ctx.root_component._get_all_imports(), |
There was a problem hiding this comment.
is this still making the recursive call? isn't this information cached in the PageContext as imports?
β¦ields Remove the duplicated _compile_component_without_replacements and _compile_component_single_enter_fast_path methods in favor of a single _compile_component_tree walker with inline replacement checks. This eliminates ~200 lines of duplication and removes several optimization-only fields (_regular_leave_component_hook_binders, _stateful_leave_component_hook_binders, _component_hooks_can_replace, _enter_component_hooks, _leave_component_hooks) with negligible benchmark impact.
_compile_stateful_components now returns its collected imports alongside the rendered code. CompileContext merges these into all_imports after compilation, and no longer calls _get_all_imports on the root component (the single-pass walk already collects structural imports).
CompilerPlugin duplicated methods already on Plugin base class. PageDefinition was a structural typing protocol only satisfied by UnevaluatedPage. Replace both with their concrete counterparts. Give UnevaluatedPage fields defaults so it can be used directly in tests and benchmarks, eliminating FakePage and BenchmarkPage classes.
# Conflicts: # packages/reflex-base/src/reflex_base/components/component.py # reflex/app.py
The reflex version in reflex-web's pyproject.toml is of no consequence to us, we want to test the reflex version in the current PR.
Remove the StatefulComponent class (~470 lines) and its two-pass compile model (evaluate β StatefulComponent.compile_from β render). Replace it with MemoizeStatefulPlugin, a single-pass compiler plugin that auto-memoizes stateful subtrees using the experimental memo infrastructure. Key changes: - Delete StatefulComponent from component.py. All memoization logic now lives in reflex/compiler/plugins/memoize.py (MemoizeStatefulPlugin) and reflex_base/components/memoize_helpers.py (event trigger helpers). - Remove the shared stateful_components module. Memoized wrappers are now compiled as experimental memo components into $/utils/components, tracked via CompileContext.memoize_wrappers and auto_memo_components. - Remove CompilerPlugin protocol β Plugin base class already provides the same eval_page/compile_page/enter_component/leave_component interface. - Add PageDefinition protocol so CompileContext.pages is decoupled from UnevaluatedPage, allowing test fixtures to provide minimal page-like objects. - Add three specialized tree-walk methods to CompilerHooks (_compile_component_without_replacements, _single_enter_fast_path, _with_replacements) to avoid replacement-dispatch overhead when no plugin can replace components. - Simplify compile_unevaluated_page to inline page evaluation directly (route, style, theme as positional args) instead of going through the full plugin pipeline. - Remove redundant _get_all_imports() and _get_all_app_wrap_components() calls from the evaluate loop β the tree walk via DefaultCollectorPlugin already collects these into PageContext. - Remove dead code: _compile_page_from_app wrapper, get_stateful_components_path, compile_stateful_components, _compile_stateful_components, _get_shared_components_recursive. - Clean up Plugin.enter_component / leave_component signatures by removing the stateful_component parameter from all hooks. - Remove unused _enter_component_hooks/_leave_component_hooks fields from CompilerHooks (only binders are used at runtime). - Remove dead apply_overlay parameter from CompileContext.compile().
Memo wrappers are transparent in the authored component tree β they should not trigger _valid_parents checks against themselves. Override _validate_component_children on ExperimentalMemoComponent to skip the check, preventing false validation failures when a restricted child (e.g. _valid_parents = ["ValidParent"]) is wrapped in a memo before being placed inside its valid parent.
Summary
enter_component/leave_component/eval_page/compile_pagehooks, enabling third-party plugins to participate in the frontend compilation pipelineApp._compile()intocompiler.compile_app(), decoupling compilation logic from the App classExecutorType/ExecutorSafeFunctionsabstractions fromenvironment.pyin favor of a sequential plugin-driven compilation modelDefaultPagePlugin,ApplyStylePlugin,DefaultCollectorPluginthat replicate the existing compilation behaviorDefaultPagePluginto preserveVar-backed page titles instead of replacing them with the default stringKey Changes
packages/reflex-core/src/reflex_core/plugins/compiler.py(new): Core plugin infrastructure βCompilerPluginprotocol,CompileContext,PageContext,CompilerHooks, and component tree traversal logicreflex/compiler/plugins/builtin.py(new): Built-in plugins (DefaultPagePlugin,ApplyStylePlugin,DefaultCollectorPlugin) that replicate existing compilation behaviorreflex/compiler/compiler.py: Refactoredcompile_app()to use plugin-driven compilation viaCompilerHooksreflex/app.py: Removed ~430 lines of compilation logic,App._compile()now delegates tocompiler.compile_app()packages/reflex-core/src/reflex_core/environment.py: RemovedExecutorTypeandExecutorSafeFunctions(no longer needed)Test Plan
tests/units/compiler/test_plugins.py(855 lines) covering:CompileContextlifecycle and state managementDefaultPagePluginbehavior including Var-backed titlesApplyStylePluginstyle applicationDefaultCollectorPluginimport/custom code collectiontests/units/test_app.pyverifying memo components are written to shared components moduleAll Submissions:
Type of change
Please delete options that are not relevant.
New Feature Submission:
Changes To Core Features:
closes #6210
closes #6211
closes #6212
Closes #6213