Add virtual filesystem support for module imports#481
Open
ganehag wants to merge 2 commits into
Open
Conversation
Script.SetImportFS and Compiler.SetImportFS accept any fs.FS as the
source for file-based imports. Modules are resolved and read through
the FS rather than the OS, which enables sandboxed execution, embedded
script bundles (embed.FS), in-memory testing (fstest.MapFS), and
custom backends such as tar archives or databases.
Relative imports (import("./sibling")) work correctly across the VFS:
the compiler tracks the current module directory as a plain forward-
slash string, making behaviour identical on all platforms.
MultiFS layers multiple fs.FS values in priority order, allowing a
base archive to be patched by an overlay without any changes to the
import statements in the scripts themselves.
Bumps the minimum Go version from 1.13 to 1.16 to pick up io/fs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
File-based imports currently require modules to exist on the OS filesystem. That makes Tengo harder to use in sandboxed environments, when embedding scripts into a binary, or when writing isolated unit tests without temporary files.
This changeset adds
SetImportFS(fsys fs.FS)to bothScriptandCompiler. When set, imports are resolved and read through the providedfs.FSinstead of the OS filesystem. Relative imports such asimport("./sibling")are supported by tracking the current module directory as a forward-slash path, giving consistent behaviour across platforms.The changeset also adds
MultiFS, which layers multiplefs.FSvalues in priority order. This supports the common case where a base module archive needs to be partially overridden without changing import statements in scripts.Because
fs.FSis a standard library interface, this works out of the box withembed.FS,testing/fstest.MapFS,fs.Sub, and third-party filesystem adapters such as zip, tar, S3-backed filesystems, and similar implementations.This requires raising the minimum supported Go version from 1.13 to 1.16. Given that Go 1.13 is now several years old, and Go 1.16 introduced useful standard-library support such as
io/fs, I think this is a reasonable tradeoff.