Installation instructions for HAProxy

=====================================

This is a development version, so it is expected to break from time to time,

to add and remove features without prior notification and it should not be used

in production, unless you're an experienced user and are willing to follow

weekly updates. If you are not used to build from sources or if you are not

used to follow updates then it is recommended that instead you use the packages

provided by your software vendor or Linux distribution. Most of them are taking

this task seriously and are doing a good job at backporting important fixes.

If for any reason you would prefer a different version than the one packaged

for your system, you want to be certain to have all the fixes or to get some

commercial support, other choices are available at http://www.haproxy.com/.

Areas covered in this document

==============================

1) Quick build & install

2) Basic principles

3) Build environment

4) Dependencies

5) Advanced build options

6) How to install HAProxy

1) Quick build & install

========================

If you've already built HAProxy and are just looking for a quick reminder, here

are a few build examples :

- recent Linux system with all options, make and install :

$ make clean

$ make -j $(nproc) TARGET=linux-glibc \

USE_OPENSSL=1 USE_QUIC=1 USE_QUIC_OPENSSL_COMPAT=1 \

USE_LUA=1 USE_PCRE2=1

$ sudo make install

- FreeBSD + OpenSSL, build with all options :

$ gmake -j $(sysctl -n hw.ncpu) TARGET=freebsd \

USE_OPENSSL=1 USE_QUIC=1 USE_QUIC_OPENSSL_COMPAT=1 \

USE_LUA=1 USE_PCRE2=1

- OpenBSD + LibreSSL, build with all options :

$ gmake -j $(sysctl -n hw.ncpu) TARGET=openbsd \

USE_OPENSSL=1 USE_QUIC=1 USE_LUA=1 USE_PCRE2=1

- embedded Linux, build using a cross-compiler :

$ make -j $(nproc) TARGET=linux-glibc USE_OPENSSL=1 USE_PCRE2=1 \

CC=/opt/cross/gcc730-arm/bin/gcc CFLAGS="-mthumb" ADDLIB=-latomic

- Build with static PCRE on Solaris / UltraSPARC :

$ make -j $(/usr/sbin/psrinfo -p) TARGET=solaris \

CPU_CFLAGS="-mcpu=v9" USE_STATIC_PCRE2=1

For more advanced build options or if a command above reports an error, please

read the following sections.

2) Basic principles

===================

HAProxy uses a single GNU Makefile which supports options on the command line,

so that there is no need to hack a "configure" file to work on your system. The

makefile totally supports parallel build using "make -j <jobs>" where <jobs>

matches the number of usable processors, which on some platforms is returned by

the "nproc" utility. The explanations below may occasionally refer to some

options, usually in the form "name=value", which have to be passed to the

command line. This means that the option has to be passed after the "make"

command. For example :

$ make -j $(nproc) TARGET=generic USE_GZIP=1

One required option is TARGET, it must be set to a target platform name, which

provides a number of presets. The list of known platforms is displayed when no

target is specified. It is not strictly required to use the exact target, you

can use a relatively similar one and adjust specific variables by hand.

Most configuration variables are in fact booleans. Some options are detected and

enabled by default if available on the target platform. This is the case for all

those named "USE_<feature>". These booleans are enabled by "USE_<feature>=1"

and are disabled by "USE_<feature>=" (with no value) or "USE_<feature>=0". An

exhaustive list of the supported USE_* features is located at the top of the

main Makefile. The last occurrence of such an option on the command line

overrides any previous one. Example :

$ make TARGET=generic USE_THREAD=

In case of error or missing TARGET, a help screen is displayed. It is also

possible to display a list of all known options using "make help".

Some optional components which may depend on third-party libraries, are used

with popular tools which are not necessarily standard implementations, or are

maintained at slower pace than the core of the project, are located in the

"addons/" directory. These ones may disappear in a future version if the

product they depend on disappears or if their maintainers do not assign enough

resources to maintain them any more. For this reason they are not built by

default, but some USE_* options are usually provided for them, and their build

is routinely tested anyway.

3) Build environment

====================

HAProxy requires a working GCC or Clang toolchain and GNU make :

- GNU make >= 3.80. Note that neither Solaris nor OpenBSD's make work with

the GNU Makefile. If you get many syntax errors when running "make", you

may want to retry with "gmake" which is the name commonly used for GNU make

on BSD systems.

- GCC >= 4.7 (up to 15 tested). Older versions are no longer supported due to

the latest mt_list update which only uses c11-like atomics. Newer versions

may sometimes break due to compiler regressions or behaviour changes. The

version shipped with your operating system is very likely to work with no

trouble. Clang >= 3.0 is also known to work as an alternative solution, and

versions up to 19 were successfully tested. Recent versions may emit a bit

more warnings that are worth reporting as they may reveal real bugs. TCC

(https://repo.or.cz/tinycc.git) is also usable for developers but will not

support threading and was found at least once to produce bad code in some

rare corner cases (since fixed). But it builds extremely quickly (typically

half a second for the whole project) and is very convenient to run quick

tests during API changes or code refactoring.

- GNU ld (binutils package), with no particular version. Other linkers might

work but were not tested. The default one from your operating system will

normally work.

On debian or Ubuntu systems and their derivatives, you may get all these tools

at once by issuing the two following commands :

$ sudo apt-get update

$ sudo apt-get install build-essential

On Fedora, CentOS, RHEL and derivatives, you may get the equivalent packages

with the following command :

$ sudo yum groupinstall "Development Tools"

Please refer to your operating system's documentation for other systems.

It is also possible to build HAProxy for another system or platform using a

cross-compiler but in this case you probably already have installed these

tools.

Building HAProxy may require between 60 and 80 MB of free space in the

directory where the sources have been extracted, depending on the debugging

options involved.

4) Dependencies

===============

HAProxy in its basic form does not depend on anything beyond a working libc.

However a number of options are enabled by default, or are highly recommended,

and these options will typically involve some external components or libraries,

depending on the targeted platform.

Optional dependencies may be split into several categories :

- memory allocation

- regular expressions

- multi-threading

- password encryption

- cryptography

- compression

- lua

- device detection

- miscellaneous

4.1) Memory allocation

----------------------

By default, HAProxy uses the standard malloc() call provided by the libc. It

may also be built to use jemalloc, which is fast and thread-safe. In order to

use it, please add "-ljemalloc" to the ADDLIB variable. You may possibly also

need to append "-lpthread" and/or "-ldl" depending on the operating system.

4.2) Regular expressions

------------------------

HAProxy may make use regular expressions (regex) to match certain patterns. The

regex engine is provided by default in the libc. On some operating systems, it

might happen that the original regex library provided by the libc is too slow,

too limited or even bogus. For example, on older Solaris versions up to 8, the

default regex used not to properly extract group references, without reporting

compilation errors. Also, some early versions of the GNU libc used to include a

regex engine which could be slow or even crash on certain patterns.

If you plan on importing a particularly heavy configuration involving a lot of

regex, you may benefit from using some alternative regex implementations such as

PCRE. HAProxy natively supports PCRE and PCRE2 (recommended), both in standard

and JIT flavors (Just In Time). The following options are available depending on

the library version provided on your system :

- "USE_PCRE=1" : enable PCRE version 1, dynamic linking

- "USE_STATIC_PCRE=1" : enable PCRE version 1, static linking

- "USE_PCRE_JIT=1" : enable PCRE version 1 in JIT mode

- "USE_PCRE2=1" : enable PCRE version 2, dynamic linking

- "USE_STATIC_PCRE2=1" : enable PCRE version 2, static linking

- "USE_PCRE2_JIT=1" : enable PCRE version 2 in JIT mode

Both of these libraries may be downloaded from https://www.pcre.org/.

By default, the include and library paths are figured from the "pcre-config"

and "pcre2-config" utilities. If these ones are not installed or inaccurate

(for example when cross-compiling), it is possible to force the path to include

files using "PCRE_INC" and "PCRE2_INC" respectively, and the path to library

files using "PCRE_LIB" and "PCRE2_LIB" respectively. For example :

$ make TARGET=generic \

USE_PCRE2_JIT=1 PCRE2_INC=/opt/cross/include PCRE2_LIB=/opt/cross/lib

4.3) Multi-threading

--------------------

On some systems for which positive feedback was reported, multi-threading will

be enabled by default. When multi-threading is used, the libpthread library

(POSIX threading) will be used. If the target system doesn't contain such a

library, it is possible to forcefully disable multi-threading by adding

"USE_THREAD=" on the command line.

4.4) Password encryption

------------------------

Many systems provide password encryption functions used for authentication. On

some systems these functions are part of the libc. On others, they're part of a

separate library called "libcrypt". The default targets are pre-configured

based on which system needs the library. It is possible to forcefully disable

the linkage against libcrypt by adding "USE_LIBCRYPT=" on the command line, or

to forcefully enable it using "USE_LIBCRYPT=1".

4.5) Cryptography

-----------------

For SSL/TLS, it is necessary to use a cryptography library. HAProxy currently

supports the OpenSSL library, and is known to build and work with branches

1.0.0, 1.0.1, 1.0.2, 1.1.0, 1.1.1, and 3.0 to 3.6. It is recommended to use

at least OpenSSL 1.1.1 to have support for all SSL keywords and configuration

in HAProxy. OpenSSL follows a long-term support cycle similar to HAProxy's,

and each of the branches above receives its own fixes, without forcing you to

upgrade to another branch. There is no excuse for staying vulnerable by not

applying a fix available for your version. There is always a small risk of

regression when jumping from one branch to another one, especially when it's

very new, so it's preferable to observe for a while if you use a different

version than your system's defaults. Specifically, it has been well established

that OpenSSL 3.0 can be 2 to 20 times slower than earlier versions on

multiprocessor systems due to design issues that cannot be fixed without a

major redesign, so in this case upgrading should be carefully thought about

(please see https://github.com/openssl/openssl/issues/20286 and

https://github.com/openssl/openssl/issues/17627). If a migration to 3.x is

mandated by support reasons, at least 3.1 recovers a small fraction of this

important loss.

Three OpenSSL derivatives called LibreSSL, QUICTLS, and AWS-LC are

reported to work as well. While there are some efforts from the community to

ensure they work well, OpenSSL remains the primary target and this means that

in case of conflicting choices, OpenSSL support will be favored over other

options. Note that QUIC is not fully supported when haproxy is built with

OpenSSL < 3.5.2 version. In this case, QUICTLS or AWS-LC are the preferred

alternatives. As of writing this, the QuicTLS project follows OpenSSL very

closely and provides update simultaneously, but being a volunteer-driven

project, its long-term future does not look certain enough to convince

operating systems to package it, so it needs to be build locally. Recent

versions of AWS-LC (>= 1.22 and the FIPS branches) are pretty complete and

generally more performant than other OpenSSL derivatives, but may behave

slightly differently, particularly when dealing with outdated setups. See

the section about QUIC in this document.

A fifth option is wolfSSL (https://github.com/wolfSSL/wolfssl). It is the only

supported alternative stack not based on OpenSSL, yet which implements almost

all of its API and natively supports QUIC. At the time of writing, the vast

majority of SSL features are well supported by wolfSSL though not everything is

exposed in haproxy yet, advanced users might notice tiny differences that the

wolfSSL and HAProxy teams are working on together to address in the wolfSSL

code base. Features like ecdsa/rsa dual stack, crt-list and client auth might

not work as expected. As of November 2023, wolfSSL support is considered

experimental. This stack is not affected by OpenSSL's design issue regarding

multi-processor systems and is viewed by the HAProxy team as the most promising

mid-term solution for general deployments and QUIC deployments.

In order to enable SSL/TLS support, simply pass "USE_OPENSSL=1" on the command

line and the default library present on your system will be used :

$ make TARGET=generic USE_OPENSSL=1

If you want to use a different version from the one provided by your system

(which is not recommended due to the risk of missing security fixes), it is

possible to indicate the path to the SSL include files using SSL_INC, and the

SSL library files using SSL_LIB. Example :

$ make TARGET=generic \

USE_OPENSSL=1 SSL_INC=/opt/ssl-1.1.1/include SSL_LIB=/opt/ssl-1.1.1/lib

To use HAProxy with WolfSSL, WolfSSL must be built with haproxy support, at

least WolfSSL 5.6.6 is needed, but a development version might be needed for

some of the features:

$ cd ~/build/wolfssl

$ ./configure --enable-haproxy --enable-quic --prefix=/opt/wolfssl-5.6.6/

$ make -j $(nproc)

$ make install

Please also note that wolfSSL supports many platform-specific features that may

affect performance, and that for production uses it might be a good idea to

check them using "./configure --help". Please refer to the lib's documentation.

When running wolfSSL in chroot, either mount /dev/[u]random devices into the

chroot:

$ mkdir -p /path/to/chrootdir/dev/

$ mknod -m 444 /path/to/chrootdir/dev/random c 1 8

$ mknod -m 444 /path/to/chrootdir/dev/urandom c 1 9

Or, if your OS supports it, enable the getrandom() syscall by appending the

following argument to the wolfSSL configure command:

EXTRA_CFLAGS=-DWOLFSSL_GETRANDOM=1

Building HAProxy with wolfSSL requires to specify the API variant on the "make"

command line, for example:

$ cd ~/build/haproxy

$ make -j $(nproc) TARGET=generic USE_OPENSSL_WOLFSSL=1 USE_QUIC=1 \

SSL_INC=/opt/wolfssl-5.6.6/include SSL_LIB=/opt/wolfssl-5.6.6/lib

To use HAProxy with AWS-LC you must have version v1.22.0 or newer of AWS-LC

built and installed locally.

$ cd ~/build/aws-lc

$ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/opt/aws-lc

$ make -j $(nproc)

$ make install

Building HAProxy with AWS-LC requires you to enable AWS-LC support, and specify

the path it was installed to when running make for HAPRoxy.

$ cd ~/build/haproxy

$ make -j $(nproc) TARGET=generic USE_OPENSSL_AWSLC=1 \

SSL_INC=/opt/aws-lc/include SSL_LIB=/opt/aws-lc/lib

In order to link OpenSSL statically against HAProxy, first download OpenSSL

from https://www.openssl.org/ then build it with the "no-shared" keyword and

install it to a local directory, so your system is not affected :

$ export STATICLIBSSL=/tmp/staticlibssl

$ ./config --prefix=$STATICLIBSSL no-shared

$ make && make install_sw

Then when building haproxy, pass that path via SSL_INC and SSL_LIB :

$ make TARGET=generic \

USE_OPENSSL=1 SSL_INC=$STATICLIBSSL/include SSL_LIB=$STATICLIBSSL/lib

When building with OpenSSL on some systems, you may also need to enable support

for the "libz" library, which is visible if the linker complains about function

"deflateInit()" not being found. In this case, simply append "ADDLIB=-lz" to

the command line.

It is worth mentioning that asynchronous cryptography engines are supported on

OpenSSL 1.1.0 and above. Such engines are used to access hardware cryptography

acceleration that might be present on your system. Due to API changes that

appeared with OpenSSL 3.0 and cause lots of build warnings, engines are not

enabled by default anymore in HAProxy 2.6. It is required to pass USE_ENGINE=1

if they are desired.

If for any reason you are forced to use OpenSSL 3.x and the performance is not

acceptable at all, you may want to try replacing the pthread locks that OpenSSL

uses with HAProxy's much lighter locks that are able to emulate them:

$ make TARGET=generic \

USE_OPENSSL=1 USE_PTHREAD_EMULATION=1

On large multi-processor systems, this may result in a performance increase of

50 to 100% on OpenSSL 3.0 depending on the level of contention, but this will

of course not recover everything. It should not be used by distro packagers as

it is a bit less observable.

4.6) Compression

----------------

HAProxy can compress HTTP responses before delivering them to clients, in order

to save network bandwidth. Two compression options are available. The first one

relies on the libslz library (http://libslz.org) that is embedded in haproxy.

It is enabled by default as it is very fast and does not keep a copy of the

contents in memory. It is possible to disable it, for example for very small

systems, by passing "USE_SLZ=" to the "make" command.

Please note that SLZ will benefit from some CPU-specific instructions like the

availability of the CRC32 extension on some ARM processors. Thus it can further

improve its performance to build with:

- "CPU_CFLAGS=-march=native" on the target system or

- "CPU_CFLAGS=-march=armv81" on modern systems such as Graviton2 or A55/A75

and beyond)

- "CPU_CFLAGS=-march=a72" (e.g. for RPi4, or AWS Graviton)

- "CPU_CFLAGS=-march=a53" (e.g. for RPi3)

- "CPU_CFLAGS=-march=armv8-auto" automatic detection with minor runtime

penalty)

A second option involves the widely known zlib library, which is very likely

installed on your system. In order to use zlib, simply pass "USE_ZLIB=1" to the

"make" command line, which will also automatically disable SLZ. If the library

is not installed in your default system's path, it is possible to specify the

path to the include files using ZLIB_INC, and the path to the library files

using ZLIB_LIB :

$ make TARGET=generic \

USE_ZLIB=1 ZLIB_INC=/opt/zlib-1.2.11/include ZLIB_LIB=/opt/zlib-1.2.11/lib

Zlib is commonly found on most systems, otherwise updates can be retrieved from

http://www.zlib.net/. It is easy and fast to build, and new versions sometimes

provide better performance so it might be worth using an up-to-date one.

Zlib compresses a bit better than libslz but at the expense of more CPU usage

(about 3.5 times more minimum), and a huge memory usage (~260 kB per compressed

stream). The only valid reason for uzing Zlib instead of SLZ here usually is to

deal with a very limited internet bandwidth while CPU and RAM are abundant so

that the last few percent of compression ratio are worth the invested hardware.

4.7) Lua

--------

Lua is an embedded programming language supported by HAProxy to provide more

advanced scripting capabilities. Only versions 5.3 and above are supported.

In order to enable Lua support, please specify "USE_LUA=1" on the command line.

Some systems provide this library under various names to avoid conflicts with

previous versions. By default, HAProxy looks for "lua5.4", "lua54", "lua5.3",

"lua53", "lua". If your system uses a different naming, you may need to set the

library name in the "LUA_LIB_NAME" variable.

If Lua is not provided on your system, it can be very simply built locally. It

can be downloaded from https://www.lua.org/, extracted and built, for example :

$ cd /opt/lua-5.4.6

$ make linux

The path to the include files and library files may be set using "LUA_INC" and

"LUA_LIB" respectively. For example :

$ make TARGET=generic \

USE_LUA=1 LUA_INC=/opt/lua-5.4.6/src LUA_LIB=/opt/lua-5.4.6/src

4.8) Device detection

---------------------

HAProxy supports several device detection modules relying on third party

products. Some of them may provide free code, others free libs, others free

evaluation licenses. Please read about their respective details in the

following files :

doc/DeviceAtlas-device-detection.txt for DeviceAtlas

doc/51Degrees-device-detection.txt for 51Degrees

doc/WURFL-device-detection.txt for Scientiamobile WURFL

4.9) Miscellaneous

------------------

Some systems have specificities. Usually these specificities are known and/or

detected and properly set for you. If you need to adjust the behaviour, here

are the extra libraries that may be referenced at build time :

- USE_RT=1 build with librt, which is sometimes needed on some systems

when using threads. It is set by default on Linux platforms,

and may be disabled using "USE_RT=" if your system doesn't

have one. You may have to set it as well if you face an error

indicating that clock_gettime() was not found.

- USE_DL=1 build with libdl, which is usually needed for Lua and OpenSSL

on Linux. It is automatically detected and may be disabled

using "USE_DL=", though it should never harm.

4.10) Common errors

-------------------

Some build errors may happen depending on the options combinations or the

selected target. When facing build errors, if you know that your system is a

bit special or particularly old, start from TARGET=generic, it is easier to

start from there and fix the remaining issues than trying to degrade another

target. Common issues may include:

- clock_gettime() not found

=> your system needs USE_RT=1

- many __sync_<something> errors in many files

=> your gcc is too old, build without threads.

- many openssl errors

=> your OpenSSL version really is too old, do not enable OpenSSL

- quic_conn-t.h: field 'level' has incomplete type

=> you tried to build QUIC with the legacy OpenSSL library, which does

not support QUIC. Either disable QUIC with "USE_QUIC=" or use any

other supported compatible library.

- many "dereferencing pointer 'sa.985' does break strict-aliasing rules"

=> these warnings happen on old compilers (typically gcc before 7.x),

and may safely be ignored; newer ones are better on these.

4.11) QUIC

----------

QUIC is the new transport layer protocol and is required for HTTP/3. This

protocol stack is currently supported as an experimental feature in haproxy on

the frontend side. In order to enable it, use "USE_QUIC=1 USE_OPENSSL=1".

Note that QUIC is not always fully supported by the OpenSSL library depending on

its version. Indeed QUIC 0-RTT cannot be supported by OpenSSL for versions before

3.5 contrary to others libraries with full QUIC support. The preferred option is

to use QUICTLS. This is a fork of OpenSSL with a QUIC-compatible API. Its

repository is available at this location:

https://github.com/quictls/openssl

You can use the following instruction to build a functional QUICTLS.

$ ./config --libdir=lib [--prefix=/opt/quictls]

$ make

$ make install

On a development environment, use SSL_INC and SSL_LIB when building haproxy to

point to the correct cryptographic library. It may be useful to specify QUICTLS

location via rpath for haproxy execution. Example :

$ make -j $(nproc) TARGET=generic \

USE_QUIC=1 \

USE_OPENSSL=1 SSL_INC=/opt/quictls/include SSL_LIB=/opt/quictls/lib \

LDFLAGS="-Wl,-rpath,/opt/quictls/lib"

Alternately, building against wolfSSL is supported as well, for example this

way assuming that wolfSSL was installed in /opt/wolfssl-5.6.0 as shown in 4.5:

$ make -j $(nproc) TARGET=generic \

USE_QUIC=1 \

USE_OPENSSL_WOLFSSL=1 \

SSL_INC=/opt/wolfssl-5.6.0/include SSL_LIB=/opt/wolfssl-5.6.0/lib

LDFLAGS="-Wl,-rpath,/opt/wolfssl-5.6.0/lib"

As last resort, haproxy may be compiled against OpenSSL as follows from 3.5

version with 0-RTT support:

$ make TARGET=generic USE_OPENSSL=1 USE_QUIC=1

or as follows for all OpenSSL versions but without O-RTT support:

$ make TARGET=generic USE_OPENSSL=1 USE_QUIC=1 USE_QUIC_OPENSSL_COMPAT=1

In addition to this requirements, the QUIC listener bindings must be explicitly

enabled with a specific QUIC tuning parameter. (see "limited-quic" global

parameter of haproxy Configuration Manual).

5) How to build HAProxy

=======================

This section assumes that you have already read section 2 (basic principles)

and section 3 (build environment). It often refers to section 4 (dependencies).

It goes into more details with the main options.

5.1) Configuring the TARGET

---------------------------

To build haproxy, you have to choose your target OS amongst the following ones

and assign it to the TARGET variable :

- linux-glibc for Linux kernel 4.17 and above

- linux-glibc-legacy for Linux kernel 2.6.28 and above without new features

- linux-musl for Linux kernel 4.17 and above with musl libc

- solaris for Solaris 10 and above

- freebsd for FreeBSD 10 and above

- dragonfly for DragonFlyBSD 4.3 and above

- netbsd for NetBSD 8 and above

- osx for Mac OS/X

- openbsd for OpenBSD 6.3 and above

- aix51 for AIX 5.1

- aix52 for AIX 5.2

- aix72-gcc for AIX 7.2 (using gcc)

- cygwin for Cygwin

- haiku for Haiku

- generic for any other OS or version.

- custom to manually adjust every setting

Example:

$ make -j $(nproc) TARGET=linux-glibc

AIX 5.3 is known to work with the generic target. However, for the binary to

also run on 5.2 or earlier, you need to build with DEFINE="-D_MSGQSUPPORT",

otherwise __fd_select() will be used while not being present in the libc, but

this is easily addressed using the "aix52" target. If you get build errors

because of strange symbols or section mismatches, simply remove -g from

ARCH_FLAGS.

Building on AIX 7.2 works fine using the "aix72-gcc" TARGET. It adds two

special CFLAGS to prevent the loading of AIX's xmem.h and var.h. This is done

by defining the corresponding include-guards _H_XMEM and _H_VAR. Without

excluding those header-files the build fails because of redefinition errors.

Furthermore, the atomic library is added to the LDFLAGS to allow for

multithreading via USE_THREAD.

You can easily define your own target with the GNU Makefile. Unknown targets

are processed with no default option except USE_POLL=default. So you can very

well use that property to define your own set of options. USE_POLL and USE_SLZ

can even be disabled by setting them to an empty string or a zero. For

example :

$ gmake TARGET=tiny USE_POLL="" USE_SLZ=0 TARGET_CFLAGS=-fomit-frame-pointer

5.2) Adding extra CFLAGS for compiling

--------------------------------------

A generic CFLAGS variable may be set to append any option to pass to the C

compiler. These flags are passed last so the variable may be used to override

other options such as warnings, optimization levels, include paths etc.

A default optimization level of -O2 is set by variable OPT_CFLAGS which may be

overridden if desired. It's used early in the list of CFLAGS so that any other

set of CFLAGS providing a different value may easily override it.

Some platforms may benefit from some CPU-specific options that will enable

certain instruction sets, word size or endianness for example. One of them is

the common "-march=native" that indicates to modern compilers that they need to

optimize for the machine the compiler is running on. Such options may be either

passed in the CPU_CFLAGS or in the CFLAGS variable, either will work though

one may be more convenient for certain methods of packaging and the other one

for other methods. Among the many possible options, the following ones are

known for having successfully been used:

- "-march=native" for a native build

- "-march=armv8-a+crc" for older ARM Cortex A53/A72/A73 (such as RPi 3B/4B)

- "-march=armv8.1-a" for modern ARM Cortex A55/A76, Graviton2+, RPi 5

- "-march=armv8-a+crc -moutline-atomics" to support older ARM with better

support of modern cores with gcc-10+

- "-mavx", "-mavx2", "-mavx512", to enable certain x86 SIMD instruction sets

- "-march=i586" to support almost all 32-bit x86 systems

- "-march=i686" to support only the latest 32-bit x86 systems

- "-march=i386" to support even the oldest 32-bit x86 systems

- "-mlittle-endian -march=armv5te" for some little-endian ARMv5 systems

- "-mcpu=v9 -mtune=ultrasparc -m64" for a 64-bit Solaris SPARC build

- "-march=1004kc -mtune=1004kc" for some multi-core 32-bit MIPS 1004Kc

- "-march=24kc -mtune=24kc" for some single-core 32-bit MIPS 24Kc

If you are building for a different system than the one you're building on,

this is called "cross-compiling". HAProxy supports cross-compilation pretty

well and tries to ease it by letting you adjust paths to all libraries (please

read section 4 on dependencies for more details). When cross-compiling, you

just need to pass the path to your compiler in the "CC" variable, and the path

to the linker in the "LD" variable. Most of the time, setting the CC variable

is enough since LD points to it by default.

By default the build process runs in quiet mode and hide the details of the

commands that are executed. This allows to more easily catch build warnings

and see what is happening. However it is not convenient at all to observe what

flags are passed to the compiler nor what compiler is involved. Simply append

"V=1" to the "make" command line to switch to verbose mode and display the

details again. It is recommended to use this option when cross-compiling to

verify that the paths are correct and that /usr/include is never involved.

If you need to pass some defines to the preprocessor or compiler, you may pass

them all in the DEFINE variable. Example:

$ make TARGET=generic DEFINE="-DDEBUG_DONT_SHARE_POOLS"

The ADDINC variable may be used to add some extra include paths; this is

sometimes needed when cross-compiling. Similarly the ADDLIB variable may be

used to specify extra paths to library files. Example :

$ make TARGET=generic ADDINC=-I/opt/cross/include ADDLIB=-L/opt/cross/lib64

5.3) Adding extra LDFLAGS for linking

-------------------------------------

If a particular target requires specific link-time flags, these can be passed

via the LDFLAGS variable. This variable is passed to the linker immediately

after ARCH_FLAGS. One of the common use cases is to add some run time search

paths for a dynamic library that's not part of the default system search path:

$ make -j $(nproc) TARGET=generic USE_OPENSSL_AWSLC=1 USE_QUIC=1 \

SSL_INC=/opt/aws-lc/include SSL_LIB=/opt/aws-lc/lib \

LDFLAGS="-Wl,-rpath,/opt/aws-lc/lib"

Some options require to be consistent between the compilation stage and the

linking stage. This is the case for options which enable debugging (e.g. "-g"),

profiling ("-pg"), link-time optimization ("-flto"), endianness ("-EB", "-EL"),

bit width ("-m32", "-m64"), or code analyzers ("-fsanitize=address"). These

options can be passed via the ARCH_FLAGS variable, which will be used at both

stages during the build process, thus avoiding the risk of inconsistencies. By

default, ARCH_FLAGS only contains "-g" to enable the generation of debug

symbols. For example, in order to build a 32-bit binary on an x86_64 Linux

system with SSL support without support for compression but when OpenSSL

requires ZLIB anyway :

$ make TARGET=linux-glibc ARCH_FLAGS="-m32 -g" USE_OPENSSL=1 ADDLIB=-lz

and building with the address sanitizer (ASAN) simply requires:

$ make TARGET=linux-glibc ARCH_FLAGS="-fsanitize=address -g"

5.4) Other common OS-specific options

-------------------------------------

Recent systems can resolve IPv6 host names using getaddrinfo(). This primitive

is not present in all libcs and does not work in all of them either. Support in

glibc was broken before 2.3. Some embedded libs may not properly work either,

thus, support is disabled by default, meaning that some host names which only

resolve as IPv6 addresses will not resolve and configs might emit an error

during parsing. If you know that your OS libc has reliable support for

getaddrinfo(), you can add USE_GETADDRINFO=1 on the make command line to enable

it. This is the recommended option for most Linux distro packagers since it's

working fine on all recent mainstream distros. It is automatically enabled on

Solaris 8 and above, as it's known to work.

If your system supports PCRE (Perl Compatible Regular Expressions), then you

really should build with libpcre which is between 2 and 10 times faster than

other libc implementations. Regex are used for header processing (deletion,

rewriting, allow, deny). Please see section 4 about dependencies to figure

how to build with PCRE support.

It is possible to add native support for SSL, by passing "USE_OPENSSL=1" on the

make command line. The libssl and libcrypto will automatically be linked with

HAProxy. Some systems also require libz, so if the build fails due to missing

symbols such as deflateInit(), then try again with "ADDLIB=-lz". Please check

section 4 about dependencies for more information on how to build with OpenSSL.

HAProxy can compress HTTP responses to save bandwidth. Please see section 4

about dependencies to see the available libraries and associated options.

If you need to pass other defines, includes, libraries, etc... then please

check the Makefile to see which ones will be available in your case, and

use/override the USE_* variables from the Makefile.

5.5) Adjusting the build error / warning behavior

-------------------------------------------------

If the ERR variable is set to any non-empty value other than "0", then -Werror

will be added to the compiler so that any build warning will trigger an error.

This is the recommended way to build when developing, and it is expected that

contributed patches were tested with ERR=1. Similarly, for developers, another

variable, FAILFAST enables -Wfatal-errors when set to non-empty except 0, and

makes the compiler stop at the first error instead of scrolling pages. It's

essentially a matter of taste.

Packagers who want to achieve the cleanest warning-free builds may be

interested in knowing that all enabled warnings are normally placed into

the WARN_CFLAGS variable. The variable contains a list of pre-established

warnings and a list of some that are dynamically detected on the compiler.

If the build environment or toolchain doesn't even support some of the basic

ones, it is then possible to just redefine them by passing the main ones in

WARN_CFLAGS (e.g. at the very least -W -Wall). Similarly, it may sometimes

be desirable not to disable certain warnings when porting to new platforms

or during code audits, or simply because the toolchain doesn't support some

of the most basic -Wno options. In this case, the list of automatic -Wno

variables is specified by variable NOWARN_CFLAGS, which is passed after

WARN_CFLAGS (i.e. it can undo some of the WARN_CFLAGS settings). Be careful

with it, as clearing this list can yield many warnings depending on the

compiler and options.

The DEP variable is automatically set to the list of include files and also

designates a file that contains the last build options used. It is used during

the build process to compute dependencies and decide whether or not to rebuild

everything (we do rebuild everything when .h files are touched or when build

options change). Sometimes when performing fast build iterations on inline

functions it may be desirable to avoid a full rebuild. Forcing this variable

to be empty will be sufficient to achieve this. This variable must never be

forced to produce final binaries, and must not be used during bisect sessions,

as it will often lead to the wrong commit.

Examples:

# silence strict-aliasing warnings with old gcc-5.5:

$ make -j$(nproc) TARGET=linux-glibc CC=gcc-55 CFLAGS=-fno-strict-aliasing

# disable all warning options:

$ make -j$(nproc) TARGET=linux-glibc CC=mycc WARN_CFLAGS= NOWARN_CFLAGS=

# enable -Werror and -Wfatal-errors to immediately stop on error

$ make -j$(nproc) TARGET=linux-glibc ERR=1 FAILFAST=1

# try to restart the build where it was after hacking an include file, to

# check if that was sufficient or not:

$ make -j$(nproc) TARGET=linux-glibc ERR=1 DEP=

5.6) Enabling a DEBUG build

---------------------------

The DEBUG variable is used to extend the CFLAGS and is preset to a list of

build-time options that are known for providing significant reliability

improvements and a barely perceptible performance cost. Unless instructed to do

so by some project developers, or trying to save the last ounce of performance,

these options should not be changed. Among the usable ones are:

- -DDEBUG_STRICT: enable some runtime assertions at key places in the code.

The goal is to emit a warning or stop the program if certain expected

conditions are not met, and whose violation will result in a misbehaving

process due to memory corruption or other significant trouble, possibly

caused by an attempt to exploit a bug in the program or a library it relies

on. The option knows 3 values: 0 (disable all such assertions, not

recommended), 1 (enable all inexpensive assertions, the default), and

2 (enable all assertions even in fast paths). Setting the option with no

value corresponds to 1, which is the recommended value for production.

- -DDEBUG_STRICT_ACTION: indicates how to react to a check violation. There

are 3 types of checks: BUG (condition that is known to have serious

consequences), WARN (warning about a highly suspicious condition which the

process may recover from, but whose unknown cause may also have serious

consequences), CHECK (verification whether a condition that developers now

consider impossible still happens). The variable takes a value from 0 to 3,

that adjusts the behavior on these 3 violations:

BUG WARN CHECK

0 warn warn warn

1 stop warn warn

2 stop stop warn

3 stop stop stop

The default value is 1, which is the best balance for production in that it

will do its best to prevent a known bogus process from running away, but

will let it run if it believes it can recover. Users running the process in

sensitive environments (finance etc) may prefer to run at level 2 to make

sure to stop any detected anomaly before it may have an impact. Level 3

should only be used at the request of developers. In any case, any emitted

warning should be reported to developers.

- -DDEBUG_MEMORY_POOLS: this enables by default extra controls around memory

allocation that will help detect coding errors such as double-frees and

freeing a bad memory location. It will also detect earlier risks of memory

overflows, which may have security implications. The cost is extremely low

(less than 1% increase in memory footprint). This is equivalent to adding

"-dMtag" on the command line. This option is enabled in the default build

options and may be disabled with -DDEBUG_MEMORY_POOLS=0.

- -DDEBUG_DONT_SHARE_POOLS: this will keep separate pools for same-sized

objects of different types. Using this increases the memory usage a little

bit but further reduces the risk of memory management related bugs and will

lead to more accurate traces in case of error. It is equivalent to adding

"-dMno-merge" on the command line. It is not enabled in the default build

options.

- -DDEBUG_POOL_INTEGRITY: this will enable runtime detection and stopping of

a class of bugs known as "use after free", which consists in modifying a

memory area after freeing it while it was reused for something else. This

option is quite powerful but such bugs are fortunately extremely rare, and

it will cause a measurable performance degradation (a few percent). This is

equivalent to adding "-dMcold-first,integrity" on the command line. This

option is not enabled by default but users running development versions on

moderate performance sites in order to participate to reliability testing

are encouraged to use it, in combination with -DDEBUG_DONT_SHARE_POOLS and

-DDEBUG_MEMORY_POOLS, as this could catch dangerous regressions.

As such, "-DDEBUG_STRICT -DDEBUG_MEMORY_POOLS" is implicit and recommended for

production. For security sensitive environments, it is recommended to use

"-DDEBUG_STRICT_ACTION=2 -DDEBUG_DONT_SHARE_POOLS". When testing new versions

or trying to nail a bug down, use "-DDEBUG_STRICT=2 -DDEBUG_STRICT_ACTION=2 \

-DDEBUG_DONT_SHARE_POOLS -DDEBUG_POOL_INTEGRITY". Finally in order to minimize

memory usage by disabling these integrity features, it is also possible to use

"-DDEBUG_STRICT=0 -DDEBUG_MEMORY_POOLS=0".

5.7) Summary of the Makefile's main variables

---------------------------------------------

The following variables are commonly used:

- TARGET platform name, empty by default, see help

- CC path to the C compiler, defaults to "cc"

- LD path to the linker, defaults to "$CC"

- CFLAGS CFLAGS to append at the end, empty by default

- LDFLAGS LDFLAGS to append at the end, empty by default

- ARCH_FLAGS flags common to CC and LD (-fsanitize, etc). Defaults to "-g"

- OPT_CFLAGS C compiler optimization level. Defaults to "-O2"

- WARN_CFLAGS list of autodetected C compiler warnings to enable

- NOWARN_CFLAGS list of autodetected C compiler warnings to disable

- ADDINC include directives to append at the end, empty by default

- ADDLIB lib directives to append at the end, empty by default

- DEFINE extra macros definitions for compiler, empty by default

- DEBUG extra DEBUG options for compiler, empty by default

- ERR enables -Werror if non-zero, empty by default

- FAILFAST enables -Wfatal-error if non-zero, empty by default

6) How to install HAProxy

=========================

To install haproxy, you can either copy the single resulting binary to the

place you want, or run :

$ sudo make install

If you're packaging it for another system, you can specify its root directory

in the usual DESTDIR variable.

-- end