ffi: load libraries from a mounted VFS · nodejs/node@c994e3d · GitHub
Skip to content

Commit c994e3d

Browse files
mcollinamcollina
committed
ffi: load libraries from a mounted VFS
The operating system's dynamic loader cannot open a library that lives in a mounted virtual file system: the reserved mount path has no real inode. Native addons already handle this in require(): the loader hands their bytes to process.dlopen(), which loads them from a private, self-cleaning image - an anonymous in-memory memfd on Linux. Make ffi.dlopen() and new DynamicLibrary() do the same transparently. Mirroring the fs handler integration, the VFS hook installer sets a library reader into node:ffi while at least one VFS is mounted and clears it when the last one unmounts; DynamicLibrary consults it before every load, so the dependency points from the VFS into ffi and ffi never loads any VFS code. The reader hands the library's bytes to the native constructor, which loads them from the same kind of image, released right after the load, while library.path keeps reporting the virtual path. Libraries on the real file system are unaffected and load directly, and pay only a null check while no VFS is mounted. Since the load happens inside the constructor, the image never outlives the call: nothing is left for dlclose() to clean up and no temporary file lingers on POSIX. The AddonImage materializer moves from an anonymous namespace in node_binding.cc to node_binding.h so that node_ffi.cc can reuse it. Signed-off-by: Matteo Collina <hello@matteocollina.com>
1 parent 336166d commit c994e3d

9 files changed

Lines changed: 324 additions & 54 deletions

File tree

‎doc/api/ffi.md‎

Lines changed: 25 additions & 0 deletions

‎doc/api/vfs.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -425,6 +425,12 @@ addon's bytes are read from the VFS and loaded from a private, self-cleaning
425425
temporary image instead. Addons on the real file system are unaffected and
426426
load directly.
427427

428+
Shared libraries opened through [`ffi.dlopen()`][] (or
429+
[`new ffi.DynamicLibrary()`][]) work the same way: a library path inside a
430+
mounted VFS is detected, its bytes are read from the VFS, and the library is
431+
loaded from a private, self-cleaning image while `library.path` keeps
432+
reporting the virtual path. Libraries on the real file system load directly.
433+
428434
## Use with Single Executable Applications
429435

430436
When running as a [Single Executable Application][] built with
@@ -634,9 +640,11 @@ fields use synthetic but stable values:
634640
[`VirtualFileSystem`]: #class-virtualfilesystem
635641
[`VirtualProvider`]: #class-virtualprovider
636642
[`ZipProvider`]: #class-zipprovider
643+
[`ffi.dlopen()`]: ffi.md#ffidlopenpath-definitions
637644
[`fs.BigIntStats`]: fs.md#class-fsstats
638645
[`fs.Stats`]: fs.md#class-fsstats
639646
[`import.meta.resolve()`]: esm.md#importmetaresolvespecifier
647+
[`new ffi.DynamicLibrary()`]: ffi.md#new-dynamiclibrarypath
640648
[`node:fs`]: fs.md
641649
[`require()`]: modules.md#requireid
642650
[`require.resolve()`]: modules.md#requireresolverequest-options

‎lib/ffi.js‎

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ const {
99
ObjectGetOwnPropertyDescriptor,
1010
ObjectKeys,
1111
ObjectPrototypeToString,
12+
ReflectConstruct,
1213
SafeWeakMap,
1314
SafeWeakRef,
1415
SymbolDispose,
@@ -38,7 +39,7 @@ const {
3839
emitExperimentalWarning('FFI');
3940

4041
const {
41-
DynamicLibrary,
42+
DynamicLibrary: NativeDynamicLibrary,
4243
getInt8,
4344
getUint8,
4445
getInt16,
@@ -119,6 +120,37 @@ function wrapFFIFunction(rawFn, owner) {
119120
return wrapped;
120121
}
121122

123+
const { getVfsLibraryReader } = require('internal/ffi/vfs');
124+
125+
// A thin constructor in front of the native class so that a library inside
126+
// a mounted virtual file system loads transparently: its bytes are read
127+
// from the VFS and handed to the native constructor, which loads them from
128+
// a private, self-cleaning image - the same way require() handles a native
129+
// addon in a VFS. The reader is installed by the VFS while it is mounted
130+
// (see internal/ffi/vfs), so no VFS code is ever loaded from here. The
131+
// wrapper shares the native prototype, so instances and instanceof behave
132+
// as if the native class were exposed directly.
133+
function DynamicLibrary(path) {
134+
if (new.target === undefined) {
135+
// Let the native constructor produce its usual error.
136+
return FunctionPrototypeCall(NativeDynamicLibrary, this, path);
137+
}
138+
const readVirtualLibrary = getVfsLibraryReader();
139+
const binary =
140+
readVirtualLibrary === null || typeof path !== 'string' ?
141+
undefined : readVirtualLibrary(path);
142+
return ReflectConstruct(NativeDynamicLibrary,
143+
binary === undefined ? [path] : [path, binary],
144+
new.target);
145+
}
146+
DynamicLibrary.prototype = NativeDynamicLibrary.prototype;
147+
ObjectDefineProperty(DynamicLibrary.prototype, 'constructor', {
148+
__proto__: null,
149+
configurable: true,
150+
value: DynamicLibrary,
151+
writable: true,
152+
});
153+
122154
const rawGetFunction = DynamicLibrary.prototype.getFunction;
123155
const rawGetFunctions = DynamicLibrary.prototype.getFunctions;
124156
const rawClose = DynamicLibrary.prototype.close;

‎lib/internal/ffi/vfs.js‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
'use strict';
2+
3+
// Seam between node:ffi and the virtual file system, mirroring the fs
4+
// handler integration in internal/fs/utils: the VFS hook installer sets a
5+
// library reader while at least one VFS is mounted and clears it when the
6+
// last one unmounts, and DynamicLibrary consults it before every load. The
7+
// dependency points from the VFS into ffi: ffi never loads any VFS code,
8+
// and pays only a null check while no VFS is mounted.
9+
10+
// When reader is null, no VFS is active (zero overhead). Otherwise it is
11+
// (path) => Buffer|undefined: the library's bytes for a path inside a
12+
// mounted VFS, or undefined for a path the dynamic loader should open
13+
// itself.
14+
let vfsLibraryReader = null;
15+
16+
function setVfsLibraryReader(reader) {
17+
vfsLibraryReader = reader;
18+
}
19+
20+
function getVfsLibraryReader() {
21+
return vfsLibraryReader;
22+
}
23+
24+
module.exports = {
25+
getVfsLibraryReader,
26+
setVfsLibraryReader,
27+
};

‎lib/internal/vfs/setup.js‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -976,6 +976,30 @@ function installAddonLoader() {
976976
};
977977
}
978978

979+
/**
980+
* Reads the bytes of a file that lives in a mounted VFS. Returns undefined
981+
* for a path outside the reserved VFS root - the caller should open the
982+
* path itself - and throws ENOENT for a path under the root that no
983+
* mounted VFS serves, since no real file can exist there. Installed into
984+
* node:ffi while hooks are installed, so DynamicLibrary can load a
985+
* VFS-resident library from a private image, the same way the module
986+
* loader handles a native addon in a VFS.
987+
* @param {string} pathStr The path of the library
988+
* @returns {Buffer|undefined} The library's bytes, or undefined
989+
*/
990+
function readVirtualBinary(pathStr) {
991+
const normalized = normalizeMountedPath(pathStr);
992+
if (!StringPrototypeStartsWith(normalized, normalizedVfsRootPrefix)) {
993+
return undefined;
994+
}
995+
const layerId = getLayerIdFromPath(normalized);
996+
const vfs = layerId === -1 ? undefined : activeVFSLayers.get(layerId);
997+
if (vfs === undefined || !vfs.shouldHandleNormalized(normalized)) {
998+
throw createENOENT('open', pathStr);
999+
}
1000+
return vfs.readFileSync(normalized);
1001+
}
1002+
9791003
/**
9801004
* Install all VFS hooks: module loader overrides and fs handlers.
9811005
*/
@@ -985,6 +1009,8 @@ function installHooks() {
9851009
normalizedVfsRootPrefix = getNormalizedVfsRoot() + sep;
9861010
installModuleLoaderOverrides();
9871011
installAddonLoader();
1012+
const { setVfsLibraryReader } = require('internal/ffi/vfs');
1013+
setVfsLibraryReader(readVirtualBinary);
9881014
vfsHandlerObj = createVfsHandlers();
9891015
setVfsHandlers(vfsHandlerObj);
9901016
hooksInstalled = true;
@@ -1002,6 +1028,8 @@ function uninstallHooks() {
10021028
setLoaderOverrides();
10031029
setVfsHandlers(null);
10041030
vfsHandlerObj = undefined;
1031+
const { setVfsLibraryReader } = require('internal/ffi/vfs');
1032+
setVfsLibraryReader(null);
10051033
process.dlopen = originalDlopen;
10061034
hooksInstalled = false;
10071035
}

‎src/node_binding.cc‎

Lines changed: 17 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
#include "util.h"
1010

1111
#include <string>
12+
#include <utility>
1213
#include <vector>
1314

1415
#ifdef _WIN32
@@ -474,64 +475,29 @@ int NodeMemfdCreate(const char* name, unsigned int flags) {
474475
return static_cast<int>(syscall(SYS_memfd_create, name, flags));
475476
}
476477
#endif // __linux__
478+
#else // _WIN32
479+
480+
// Delete-on-close handles kept alive until process exit so their temp files
481+
// outlive the loaded DLLs and are removed once the process ends.
482+
Mutex g_retained_addon_handles_mutex;
483+
std::vector<HANDLE>* g_retained_addon_handles = nullptr;
484+
477485
#endif // !_WIN32
478486

479-
// Materializes native-addon bytes into a form dlopen()/LoadLibrary() can load,
480-
// with the smallest, most private on-disk footprint each platform allows:
481-
// Linux: an anonymous in-memory memfd, loaded via /proc/self/fd/N -
482-
// the bytes never touch the filesystem.
483-
// other POSIX: a 0700 mkdtemp() directory plus an O_EXCL|O_NOFOLLOW file,
484-
// unlink()ed right after the load (the mapping keeps it alive).
485-
// Windows: a temp file opened FILE_FLAG_DELETE_ON_CLOSE; its handle is
486-
// retained for the process lifetime so the file is removed
487-
// automatically once the process (and the loaded DLL) exit.
488-
// Used for an addon that lives somewhere dlopen() cannot open by path, such as
489-
// a virtual file system.
490-
class AddonImage {
491-
public:
492-
AddonImage() = default;
493-
~AddonImage();
494-
AddonImage(const AddonImage&) = delete;
495-
AddonImage& operator=(const AddonImage&) = delete;
496-
497-
// The directory a temporary image would be written to, with a trailing
498-
// separator; empty when it cannot be determined. Names the resource for the
499-
// file-system permission check.
500-
static std::string TempDir();
501-
502-
// On success sets path() to a real, loadable path for `data`.
503-
bool Materialize(const char* data, size_t len);
504-
const std::string& path() const { return path_; }
505-
const std::string& errmsg() const { return errmsg_; }
506-
507-
// Call exactly once, right after DLib::Open(); `opened` says whether the load
508-
// succeeded. Releases the transient resources that are no longer needed (a
509-
// successful load holds its own mapping): on POSIX closes the memfd or
510-
// unlinks the temp file; on Windows retains the delete-on-close handle for
511-
// the process lifetime when opened, or closes it (deleting the file) on
512-
// failure.
513-
void AfterOpen(bool opened);
487+
} // namespace
514488

515-
private:
516-
std::string path_;
517-
std::string errmsg_;
518-
bool consumed_ = false;
489+
// AddonImage is declared in node_binding.h so that the other loader of
490+
// dynamically shared objects, node_ffi.cc, can reuse it; see the header for
491+
// the platform-by-platform description.
492+
493+
AddonImage::AddonImage() {
519494
#ifdef _WIN32
520-
HANDLE handle_ = INVALID_HANDLE_VALUE;
521-
#else
522-
bool MaterializeTempFile(const char* data, size_t len);
523-
int fd_ = -1;
524-
std::string temp_dir_; // non-empty only for the temp-file (non-memfd) path
495+
handle_ = INVALID_HANDLE_VALUE;
525496
#endif
526-
};
497+
}
527498

528499
#ifdef _WIN32
529500

530-
// Delete-on-close handles kept alive until process exit so their temp files
531-
// outlive the loaded DLLs and are removed once the process ends.
532-
Mutex g_retained_addon_handles_mutex;
533-
std::vector<HANDLE>* g_retained_addon_handles = nullptr;
534-
535501
// static
536502
std::string AddonImage::TempDir() {
537503
wchar_t dir[MAX_PATH + 1];
@@ -730,8 +696,6 @@ AddonImage::~AddonImage() {
730696

731697
#endif // _WIN32
732698

733-
} // namespace
734-
735699
// Shared by process.dlopen() and the internal dlopenBinary(). `allow_binary`
736700
// says whether args[3] may carry the addon's bytes; it is false for
737701
// process.dlopen(), whose signature stays (module, filename[, flags]).
@@ -928,6 +892,7 @@ void DLOpenBinary(const FunctionCallbackInfo<Value>& args) {
928892
DLOpenImpl(args, true);
929893
}
930894

895+
931896
inline struct node_module* FindModule(struct node_module* list,
932897
const char* name,
933898
int flag) {

‎src/node_binding.h‎

Lines changed: 54 additions & 0 deletions

0 commit comments

Comments
 (0)