Build musl from sources#97452
Conversation
Add contrib/musl submodule and a CMake build system for it. Modify toolchains, default_libs, and contrib CMake files to support USE_MUSL flag for building ClickHouse with musl libc. Assisted-by: Claude Sonnet 4.5 via GitHub Copilot
Add complete musl source file coverage and move native tool builds to build-time custom commands so protoc/grpc_cpp_plugin are built without musl's -nostartfiles flag. Assisted-by: Claude Sonnet 4.5 via GitHub Copilot
Add USE_MUSL=1 to AMD_RELEASE, ARM_RELEASE, and AMD_MUSL CI build configurations using the standard toolchain files instead of the standalone musl toolchain. Add USE_MUSL support to the aarch64 toolchain, mirroring the x86_64 implementation. Assisted-by: Claude Opus 4.6 via GitHub Copilot
Reword a comment in contrib/llvm-project-cmake/CMakeLists.txt to not mention 'check_symbol_exists', which triggers the cpp style check that bans dynamic compiler checks in contrib cmake files. Assisted-by: Claude Opus 4.6 via GitHub Copilot
Add toolchain file and -DUSE_MUSL=1 to FastTest cmake configuration, and add contrib/musl to the submodule list. This replaces the old commented-out reference to toolchain-x86_64-musl.cmake. Assisted-by: Claude Opus 4.6 via GitHub Copilot
…mand Revert the native tool build from build-time (add_custom_command) back to configure-time (execute_process) to match the upstream approach. The add_custom_command approach only declared a stamp file as OUTPUT but not the actual native binary paths, causing ninja to fail with 'missing and no known rule to make it' for any cross-compilation build (not just musl). The condition is extended to also trigger native builds when USE_MUSL is set, since musl's -nostartfiles breaks standalone executables like protoc. Remove the now-unnecessary NATIVE_TOOLS_TARGET target-level dependencies from all proto generator files, since with execute_process the native binaries already exist on disk before ninja starts. Assisted-by: Claude Opus 4.6 via GitHub Copilot
Add 14 math functions (log, log2, exp, exp2, expm1, pow, floor, fabs, scalbn, logf, fabsl, scalbnl, copysignl, fmodl) that were previously provided by glibc but are needed when building with musl. Also add explogxf.cpp as a shared dependency for exp/exp2/expm1. Includes both the source files in the SRCS list and corresponding x86_64 runtime dispatch entries for CPU-optimized variant selection. Assisted-by: Claude Opus 4.6 via GitHub Copilot
lexer_test uses plain add_executable instead of clickhouse_add_executable, so it was missing musl CRT objects (crt1, crti, crtn) while the global -nostartfiles flag prevented linking glibc's CRT. This caused the binary to have no _start entry point, resulting in a segfault. Assisted-by: Claude Opus 4.6 via GitHub Copilot
Include all 232 musl math source files instead of a partial subset. Previously 129 math files were excluded, assuming LLVM-libc would provide them, but LLVM-libc only covers ~45 functions. This left 84 math functions (mostly float-precision variants like sinf, cosf, expf) with no implementation, causing 'undefined symbol: nan' linker errors. LLVM-libc's optimized versions still take precedence via link ordering (linked before musl in global-libs). Also fix ksigaction.h include-override for aarch64: only x86_64 has an arch-specific ksigaction.h in musl; all other architectures should use the generic version from src/internal/ksigaction.h. Assisted-by: Claude Opus 4.6 via GitHub Copilot
Create include_linux_aarch64_musl with JEMALLOC_STRERROR_R_RETURNS_CHAR_WITH_GNU_SOURCE undef'd and JEMALLOC_HAVE_PTHREAD_GETNAME_NP disabled. Without this, the aarch64 build uses the glibc header which enables GNU strerror_r (returns char*), but musl always provides POSIX strerror_r (returns int), causing -Wint-conversion error in malloc_io.c. Assisted-by: Claude Opus 4.6 via GitHub Copilot
With USE_MUSL, -nostartfiles is set globally but only clickhouse_add_executable adds musl CRT objects (crt1, crti, crtn). Fix decompressor, parquet tools, and perf_checksum which use plain add_executable and would fail to link with 'cannot find entry symbol _start'. Assisted-by: Claude Opus 4.6 via GitHub Copilot
With USE_MUSL, the target-built compressor binary cannot run on the host because it's statically linked against musl. Extend the cross-compilation check to also trigger when USE_MUSL is set, matching the native build condition in the root CMakeLists.txt. This ensures the native-built pre_compressor (without musl) is used on the host, while the target-built decompressor (with musl) is passed separately for embedding. Assisted-by: Claude Opus 4.6 via GitHub Copilot
Assisted-by: Claude Opus 4.6 via GitHub Copilot
Assisted-by: Claude Opus 4.6 via GitHub Copilot
# Conflicts: # ci/jobs/build_clickhouse.py
Assisted-by: Claude Opus 4.6 via GitHub Copilot
Assisted-by: Claude Opus 4.6 via GitHub Copilot
Add 18 additional math functions from LLVM-libc: `exp`, `exp2`, `expm1`, `fabs`, `fabsl`, `floor`, `fmodl`, `log`, `log2`, `logf`, `pow`, `scalbn`, `scalbnl`, `copysignl`, `nan`, `nanf`, `nanl`, and the `explogxf` shared constants. Also add corresponding x86_64 runtime dispatch entries. Taken from #97452 Assisted-by: Claude Opus 4.6 via GitHub Copilot
# Conflicts: # .gitmodules # ci/jobs/build_clickhouse.py # contrib/libllvmlibc-cmake/CMakeLists.txt
LLVM Coverage Report
Changed lines: Changed C/C++ lines covered by tests: 42/52 (80.77%) | Lost baseline coverage: none · Uncovered code |
# Conflicts: # base/base/getL2CacheSize.cpp # base/base/getL2CacheSize.h # base/base/getPageSize.cpp # ci/jobs/compatibility_check.py # ci/jobs/fast_test.py # src/Common/ColumnsHashing.h # src/Interpreters/Aggregator.cpp # src/Interpreters/HashJoin/HashJoin.cpp
This comment was marked as outdated.
This comment was marked as outdated.
Build `memcmp`, `memcpy`, `memmove`, `memset`, `bcmp` and `memmem` from LLVM-libc into the existing `libllvmlibc` target (which already supplies the math functions), replacing the previous hand-rolled `memcmp`. The wrapper TU `contrib/libllvmlibc-cmake/string/mem_functions.cpp` includes the LLVM-libc implementation files and marks each exported C symbol `weak` via the `LLVM_LIBC_FUNCTION_ATTR_<name>` hook, so sanitizer interceptors can override them and so the existing strong `memcpy` from `base/glibc-compatibility/memcpy` keeps precedence. `memmem` is a strong symbol. On x86_64, `x86_64_mem_functions.cpp` provides `memcpy`/`memmove`/`memset` with runtime AVX-512 dispatch (a lazily-resolved function pointer, the same shape as a glibc IFUNC) plus an AVX-512/AVX2/scalar `memmem`. On aarch64, `aarch64_mem_functions.cpp` adds a NEON `memmove` and `memmem`. The library is built with `-ffreestanding -fno-builtin` so clang's loop-idiom recognition does not rewrite the inner loops into calls to the very functions being defined. SIMD tiers are selected from `X86_ARCH_LEVEL` (SSE2 / v3 / v4 with `-mprefer-vector-width=512`). Ported from the `use-musl` branch (PR ClickHouse#97452). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Revert CI job changes that switched release/fast-test builds to musl and rewrote the compatibility check: that is later "musl as main build" work. - Revert LLVM-libc mem functions (split to PR #107566); keep musl's own mem*/bcmp, restoring the x86_64 asm memcpy/memmove/memset. - Remove the llvm-tblgen -march pin and protoc DEPENDS additions that are not needed for this change. - Restyle contrib/musl-cmake and cmake/musl.cmake closer to the usual contrib cmake style: drop the unused musl-Scrt1 target, banner comments and redundant arch detection; MUSL_ARCH now comes from the toolchain files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # CMakeLists.txt
The AI reviewer correctly noted that the startup files must be linked as crt1, crti, [program objects], crtn: crti/crtn provide the prologue and epilogue of the .init/.fini sections and must wrap every other contribution to them. Reordering source lists cannot express this: CMake's Ninja generator always emits $<TARGET_OBJECTS:...> before the target's own object files, so crtn.o preceded the program objects regardless of its position in add_executable or target_sources. Wire the startup files through the link line instead: the objects are copied to stable paths in contrib/musl-cmake, crt1.o and crti.o are passed via CMAKE_EXE_LINKER_FLAGS (placed before all object files) and crtn.o at the end of DEFAULT_LIBS (placed after all libraries). This covers every executable uniformly, so the per-target special cases in clickhouse_add_executable, contrib and test CMake files are removed. The order is not hypothetical: OpenSSL contributes a .init fragment calling OPENSSL_cpuid_setup, which previously landed after crtn's epilogue (unreachable, so it never ran at startup); a binary whose .init lacks the crtn epilogue altogether segfaults in __libc_start_init, because musl calls _init. With this change, .init of the clickhouse binary disassembles to the complete push/call/pop/ret sequence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Disabling the trap unconditionally weakened the harmful-function guard on glibc builds, where mbsrtowcs is not needed by libc internals. Only musl uses it internally (in getaddrinfo), so limit the exemption to USE_MUSL as suggested by the AI reviewer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The glibc userspace headers from the sysroot were reachable in every compilation through clang's builtin sysroot search path, so any header musl does not provide (or does not win in the include order) silently fell back to glibc declarations without a link-time signal; the execinfo.h stub was one manually-shadowed example. The AI reviewer suggested narrowing the fallback to the kernel headers only. Compile everything with -nostdlibinc and pass the musl include directories explicitly in the toolchain flags. The kernel (uapi) headers, the only part of the sysroot that must stay reachable, are whitelisted through a directory of symlinks created at configure time in cmake/musl.cmake. A header musl does not provide is now a compile error: for example, #include <gnu/libc-version.h> fails with "file not found", while execinfo.h resolves to the stub and linux/asm headers to the whitelist. The two target_include_directories(global-libs ...) blocks removed here were inert: only INTERFACE_LINK_LIBRARIES of global-libs is forwarded to global-group, so these directories never appeared on any compile line; the fallback came from the builtin sysroot path, which -nostdlibinc disables. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The compiler-rt build resets CMAKE_<LANG>_FLAGS to a whitelist, which kept the musl -isystem header paths but dropped -nostdlibinc, so the runtime support objects — linked into every musl executable — could still silently fall back to the sysroot's glibc userspace headers. Keep -nostdlibinc through the strip step, as noted by the AI reviewer, so compiler-rt honors the same "musl headers only" contract as the rest of the build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| set(MUSL_GENERATED_DIR "${ClickHouse_BINARY_DIR}/contrib/musl-cmake/include") | ||
| set(MUSL_INCLUDE_DIR "${ClickHouse_SOURCE_DIR}/contrib/musl/include") | ||
|
|
||
| set(MUSL_C_INCLUDES "-isystem ${MUSL_GENERATED_DIR} -isystem ${MUSL_ARCH_DIR} -isystem ${MUSL_GENERIC_DIR} -isystem ${MUSL_INCLUDE_DIR}") |
There was a problem hiding this comment.
MUSL_C_INCLUDES no longer reconstructs the full header set that the musl toolchain started with. The regex above strips every -isystem ...musl... entry out of CMAKE_*_FLAGS, which removes both contrib/musl-cmake/include (the new execinfo.h stub) and contrib/musl-cmake/kernel-headers.
That means Cargo cc/cxx builds under USE_MUSL compile against a narrower header surface than the rest of the tree. A source that includes <execinfo.h> already fails here even though this PR intentionally adds that header for musl builds. Please re-add the stub and kernel-header directories when rebuilding MUSL_C_INCLUDES.
There was a problem hiding this comment.
This else () hard-codes RelWithDebInfo for every non-DEBUG musl build. As written, -DCMAKE_BUILD_TYPE=Release and MinSizeRel still feed Cargo cc/cxx shims the CMAKE_*_FLAGS_RELWITHDEBINFO profile.
That breaks the build-type contract specifically for musl artifacts and can leave release builds with the wrong optimisation/debug settings. Please mirror the non-musl per-build-type cases here, or select CMAKE_<LANG>_FLAGS_${CMAKE_BUILD_TYPE_UC} dynamically instead of forcing RelWithDebInfo.

Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
Build
muslfrom sources instead of using vendored binaries.