-
-
Notifications
You must be signed in to change notification settings - Fork 35.5k
vfs: add minimal node:vfs subsystem #63115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mcollina
wants to merge
17
commits into
nodejs:main
Choose a base branch
from
mcollina:vfs-minimal
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 15 commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
5bbcbad
vfs: add minimal node:vfs subsystem
mcollina fff45aa
test: add VFS API tests adapted from mount-based tests
mcollina b8219f9
test: add VFS unit tests for VirtualDir, file handles, and provider base
mcollina d01bfc1
test: adapt more VFS tests to direct API
mcollina 0a0931e
test: add VFS callback, stream, watch, and real-provider async tests
mcollina c1481a4
test: cover remaining VFS API edge cases
mcollina 7e664db
test: push every VFS file to >=95% line coverage
mcollina f23c8d0
test: push branch coverage to 95%+
mcollina c9b5b63
test: rename and split VFS test files into topic-based names
mcollina 03c7bc9
vfs: gate node:vfs behind --experimental-vfs flag
mcollina 48a89f5
vfs: fix lint errors
mcollina cdf2a0d
vfs: import getVirtualFd eagerly in streams
mcollina 779fc37
doc: trim vfs docs to match the minimal API
mcollina 98f37a8
vfs: register node:vfs as an experimental builtin in CI metadata
mcollina 117d09a
test: stabilize test-vfs-watch-directory
mcollina 034e1b2
Apply suggestions from code review
mcollina de4882c
module: exclude node:vfs from builtinModules when flag is disabled
mcollina File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,307 @@ | ||
| # Virtual File System | ||
|
|
||
| <!--introduced_in=REPLACEME--> | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| > Stability: 1 - Experimental | ||
|
|
||
| <!-- source_link=lib/vfs.js --> | ||
|
|
||
| The `node:vfs` module provides an in-memory virtual file system with an | ||
| `fs`-like API. It is useful for tests, fixtures, embedded assets, and other | ||
| scenarios where you need a self-contained file system without touching the | ||
| real disk. | ||
|
|
||
| To access it: | ||
|
|
||
| ```mjs | ||
| import vfs from 'node:vfs'; | ||
| ``` | ||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
| ``` | ||
|
|
||
| This module is only available under the `node:` scheme, and only when Node.js | ||
| is started with the `--experimental-vfs` flag. | ||
|
|
||
| ## Basic usage | ||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
|
|
||
| const myVfs = vfs.create(); | ||
| myVfs.mkdirSync('/dir', { recursive: true }); | ||
| myVfs.writeFileSync('/dir/hello.txt', 'Hello, VFS!'); | ||
|
|
||
| console.log(myVfs.readFileSync('/dir/hello.txt', 'utf8')); // 'Hello, VFS!' | ||
| ``` | ||
|
|
||
| `vfs.create()` returns a [`VirtualFileSystem`][] instance backed by a | ||
| [`MemoryProvider`][] by default. The instance exposes synchronous, | ||
| callback-based, and promise-based file system methods that mirror the | ||
| shape of the [`node:fs`][] API. All paths are POSIX-style and absolute | ||
| (starting with `/`). | ||
|
|
||
| ## `vfs.create([provider][, options])` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * `provider` {VirtualProvider} The provider to use. **Default:** | ||
| `new MemoryProvider()`. | ||
| * `options` {Object} | ||
| * `emitExperimentalWarning` {boolean} Whether to emit the experimental | ||
| warning when the instance is created. **Default:** `true`. | ||
| * Returns: {VirtualFileSystem} | ||
|
|
||
| Convenience factory equivalent to `new VirtualFileSystem(provider, options)`. | ||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
|
|
||
| // Default in-memory provider | ||
| const memoryVfs = vfs.create(); | ||
|
|
||
| // Explicit provider | ||
| const realVfs = vfs.create(new vfs.RealFSProvider('/tmp/sandbox')); | ||
| ``` | ||
|
|
||
| ## Class: `VirtualFileSystem` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| A `VirtualFileSystem` wraps a [`VirtualProvider`][] and exposes an | ||
| `fs`-like API. Each instance maintains its own file tree. | ||
|
mcollina marked this conversation as resolved.
Outdated
|
||
|
|
||
| ### `new VirtualFileSystem([provider][, options])` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * `provider` {VirtualProvider} The provider to use. **Default:** | ||
| `new MemoryProvider()`. | ||
| * `options` {Object} | ||
| * `emitExperimentalWarning` {boolean} Whether to emit the experimental | ||
| warning. **Default:** `true`. | ||
|
|
||
| ### `vfs.provider` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * {VirtualProvider} | ||
|
|
||
| The provider backing this VFS instance. | ||
|
|
||
| ### `vfs.readonly` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * {boolean} | ||
|
|
||
| `true` when the underlying provider is read-only. | ||
|
|
||
| ### File system methods | ||
|
|
||
| `VirtualFileSystem` implements the following methods, with the same | ||
| signatures as their [`node:fs`][] counterparts: | ||
|
|
||
| #### Synchronous methods | ||
|
|
||
| * `existsSync(path)` | ||
| * `statSync(path[, options])` | ||
| * `lstatSync(path[, options])` | ||
| * `readFileSync(path[, options])` | ||
| * `writeFileSync(path, data[, options])` | ||
| * `appendFileSync(path, data[, options])` | ||
| * `readdirSync(path[, options])` | ||
| * `mkdirSync(path[, options])` | ||
| * `rmdirSync(path)` | ||
| * `unlinkSync(path)` | ||
| * `renameSync(oldPath, newPath)` | ||
| * `copyFileSync(src, dest[, mode])` | ||
| * `realpathSync(path[, options])` | ||
| * `readlinkSync(path[, options])` | ||
| * `symlinkSync(target, path[, type])` | ||
| * `accessSync(path[, mode])` | ||
| * `rmSync(path[, options])` | ||
| * `truncateSync(path[, len])` | ||
| * `ftruncateSync(fd[, len])` | ||
| * `linkSync(existingPath, newPath)` | ||
| * `chmodSync(path, mode)` | ||
| * `chownSync(path, uid, gid)` | ||
| * `utimesSync(path, atime, mtime)` | ||
| * `lutimesSync(path, atime, mtime)` | ||
| * `mkdtempSync(prefix)` | ||
| * `opendirSync(path[, options])` | ||
| * `openAsBlob(path[, options])` | ||
| * File-descriptor ops: `openSync`, `closeSync`, `readSync`, `writeSync`, | ||
| `fstatSync` | ||
| * Streams: `createReadStream`, `createWriteStream` | ||
| * Watchers: `watch`, `watchFile`, `unwatchFile` | ||
|
|
||
| #### Callback-style asynchronous methods | ||
|
|
||
| `readFile`, `writeFile`, `stat`, `lstat`, `readdir`, `realpath`, `readlink`, | ||
| `access`, `open`, `close`, `read`, `write`, `rm`, `fstat`, `truncate`, | ||
| `ftruncate`, `link`, `mkdtemp`, `opendir`. Each takes a Node.js-style | ||
| callback `(err, ...result)`. | ||
|
|
||
| #### Promise methods | ||
|
|
||
| `vfs.promises` exposes the promise-based variants: | ||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
|
|
||
| async function example() { | ||
| const myVfs = vfs.create(); | ||
| await myVfs.promises.writeFile('/file.txt', 'hello'); | ||
| const data = await myVfs.promises.readFile('/file.txt', 'utf8'); | ||
| return data; | ||
| } | ||
| example(); | ||
| ``` | ||
|
|
||
| The promise namespace mirrors `fs.promises` and includes `readFile`, | ||
| `writeFile`, `appendFile`, `stat`, `lstat`, `readdir`, `mkdir`, `rmdir`, | ||
| `unlink`, `rename`, `copyFile`, `realpath`, `readlink`, `symlink`, | ||
| `access`, `rm`, `truncate`, `link`, `mkdtemp`, `chmod`, `chown`, `lchown`, | ||
| `utimes`, `lutimes`, `open`, `lchmod`, and `watch`. | ||
|
|
||
| ## Class: `VirtualProvider` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| The base class for all VFS providers. Subclasses implement the essential | ||
| primitives (`open`, `stat`, `readdir`, `mkdir`, `rmdir`, `unlink`, | ||
| `rename`, ...) and inherit default implementations of the derived | ||
| methods (`readFile`, `writeFile`, `exists`, `copyFile`, `access`, ...). | ||
|
|
||
| ### Capability flags | ||
|
|
||
| * `provider.readonly` {boolean} **Default:** `false`. | ||
| * `provider.supportsSymlinks` {boolean} **Default:** `false`. | ||
| * `provider.supportsWatch` {boolean} **Default:** `false`. | ||
|
|
||
| ### Creating custom providers | ||
|
|
||
| ```cjs | ||
| const { VirtualProvider } = require('node:vfs'); | ||
|
|
||
| class StaticProvider extends VirtualProvider { | ||
| get readonly() { return true; } | ||
|
|
||
| statSync(path) { /* ... */ } | ||
| openSync(path, flags) { /* ... */ } | ||
| readdirSync(path, options) { /* ... */ } | ||
| // ... | ||
| } | ||
| ``` | ||
|
|
||
| The base class throws `ERR_METHOD_NOT_IMPLEMENTED` for any primitive | ||
| that has not been overridden, and rejects writes from a `readonly` | ||
| provider with `EROFS`. | ||
|
|
||
| ## Class: `MemoryProvider` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| The default in-memory provider. Stores files, directories, and symbolic | ||
| links in a `Map`-backed tree, supports symlinks (`supportsSymlinks === | ||
| true`), and supports watching (`supportsWatch === true`). | ||
|
|
||
| ### `memoryProvider.setReadOnly()` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| Locks the provider into read-only mode. Subsequent writes through any | ||
| [`VirtualFileSystem`][] using this provider throw `EROFS`. There is no | ||
| way to revert the provider to writable. | ||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
|
|
||
| const provider = new vfs.MemoryProvider(); | ||
| const myVfs = vfs.create(provider); | ||
| myVfs.writeFileSync('/seed.txt', 'initial'); | ||
|
|
||
| provider.setReadOnly(); | ||
|
|
||
| myVfs.writeFileSync('/x.txt', 'fail'); // throws EROFS | ||
| ``` | ||
|
|
||
| ## Class: `RealFSProvider` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| A provider that wraps a real file system directory and exposes its | ||
| contents through the VFS API. All VFS paths are resolved relative to | ||
| the root and verified to stay inside it; symbolic links resolving | ||
| outside the root are rejected. | ||
|
|
||
| ### `new RealFSProvider(rootPath)` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * `rootPath` {string} The absolute file system path to use as the root. | ||
| Must be a non-empty string. | ||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
|
|
||
| const realVfs = vfs.create(new vfs.RealFSProvider('/tmp/sandbox')); | ||
| realVfs.writeFileSync('/file.txt', 'hello'); // writes /tmp/sandbox/file.txt | ||
| ``` | ||
|
|
||
| ### `realFSProvider.rootPath` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * {string} | ||
|
|
||
| The resolved absolute path used as the root. | ||
|
|
||
| ## Implementation details | ||
|
|
||
| ### `Stats` objects | ||
|
|
||
| VFS `Stats` objects are real instances of [`fs.Stats`][] (or | ||
| [`fs.BigIntStats`][] when `{ bigint: true }` is requested). Their | ||
| fields use synthetic but stable values: | ||
|
|
||
| * `dev` is `4085` (the VFS device id). | ||
| * `ino` is monotonically increasing per process. | ||
| * `blksize` is `4096`. | ||
| * `blocks` is `Math.ceil(size / 512)`. | ||
| * Times default to the moment the entry was created/last modified. | ||
|
|
||
| [`MemoryProvider`]: #class-memoryprovider | ||
| [`VirtualFileSystem`]: #class-virtualfilesystem | ||
| [`VirtualProvider`]: #class-virtualprovider | ||
| [`fs.BigIntStats`]: fs.md#class-fsbigintstats | ||
| [`fs.Stats`]: fs.md#class-fsstats | ||
| [`node:fs`]: fs.md | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.