{{ message }}
Support using vcpkg to build dependencies on all platforms#8387
Merged
Conversation
alexreinking
added
the
release_notes
Member
Author
steven-johnson
approved these changes
Aug 12, 2024
Member
|
How big is the vcpkg llvm build tree these days? Last time I tried to install llvm this way, I couldn't, because it needed several hundred gigabytes of free space (which it doesn't clean up once it's done building). |
Member
|
Oh I see, so this explicitly avoids building llvm somehow. How does it get llvm in that case? |
Member
Author
Yes, by setting
The same way as any other dependency: by searching the system. Vcpkg merely inserts its paths at the beginning of the various |
Member
Author
LebedevRI
pushed a commit
to LebedevRI/Halide
that referenced
this pull request
Aug 13, 2024
This PR adds support for using vcpkg to acquire and build Halide's dependencies on all platforms. It adds a top-level `vcpkg.json` file that explains the relationship between Halide's features and its dependencies. These features include the various LLVM `target-`s (which merely imply a dependency on the corresponding LLVM backend), `serialization` (flatbuffers), the `python-bindings` (pybind11), the `wasm-executor` (wabt), and a few meta-features: * `jit`: enables LLVM targets corresponding to the host system * `target-all`: enables all LLVM targets * `tests`: depends on everything needed for the tests and apps * `developer`: includes all other features All of these are optional (since x86 and WebAssembly are forced), but `jit` and `serialization` are on by default. vcpkg is intended to be an eventual replacement for FetchContent, at least on the buildbots. It will accelerate builds beyond ccache by directly restoring binary caches for our dependencies. Unlike FetchContent, it does not pollute our build with third-party CMake code. Indeed, our build has no idea at all when vcpkg is in use. The primary drawback is that vcpkg installation happens during (or ahead of) configuration time, so there is some initial wait. ## Try it! I have provided many CMake presets to ease adoption. As long as you have `VCPKG_ROOT` set to a fresh clone of `vcpkg`, they should work. They come in two flavors: * `vcpkg`: this acquires dependencies from the main vcpkg registry, but applies our own overlay, which disables building Python 3 (really!) and LLVM. The system is searched for these as usual. * `vcpkg-full`: this disables the Halide overlay and attempts to build ALL dependencies. All these presets enable the `developer` feature in `VCPKG_MANIFEST_FEATURES`, which can be overridden in the usual way. Here are the commands you should use to try it locally: * On Linux or Windows: `cmake --preset release-vcpkg` * On macOS: `cmake --preset macOS-vcpkg` * To use Visual Studio: `cmake --preset win32`. Here, `vcpkg` is implied and `-vcpkg-full` can be added to build LLVM.
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds support for using vcpkg to acquire and build Halide's dependencies on all platforms.
It adds a top-level
vcpkg.jsonfile that explains the relationship between Halide's features and its dependencies. These features include the various LLVMtarget-s (which merely imply a dependency on the corresponding LLVM backend),serialization(flatbuffers), thepython-bindings(pybind11), thewasm-executor(wabt), and a few meta-features:jit: enables LLVM targets corresponding to the host systemtarget-all: enables all LLVM targetstests: depends on everything needed for the tests and appsdeveloper: includes all other featuresAll of these are optional (since x86 and WebAssembly are forced), but
jitandserializationare on by default.vcpkg is intended to be an eventual replacement for FetchContent, at least on the buildbots. It will accelerate builds beyond ccache by directly restoring binary caches for our dependencies. Unlike FetchContent, it does not pollute our build with third-party CMake code. Indeed, our build has no idea at all when vcpkg is in use.
The primary drawback is that vcpkg installation happens during (or ahead of) configuration time, so there is some initial wait.
Try it!
I have provided many CMake presets to ease adoption. As long as you have
VCPKG_ROOTset to a fresh clone ofvcpkg, they should work. They come in two flavors:vcpkg: this acquires dependencies from the main vcpkg registry, but applies our own overlay, which disables building Python 3 (really!) and LLVM. The system is searched for these as usual.vcpkg-full: this disables the Halide overlay and attempts to build ALL dependencies.All these presets enable the
developerfeature inVCPKG_MANIFEST_FEATURES, which can be overridden in the usual way.Here are the commands you should use to try it locally:
cmake --preset release-vcpkgcmake --preset macOS-vcpkgcmake --preset win32. Here,vcpkgis implied and-vcpkg-fullcan be added to build LLVM.Buildbot adjustments
We should adjust the buildbots to use vcpkg on all platforms and to set up the overlay and feature settings so we can remove the only bit of vcpkg-aware code in the build.