You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[3.4.x] localpeer without bind results in worker segfault #3516
We figured out in production that a faulty configuration file could result in a SEGFAULT when reloading haproxy.
We tested many case and ended to the conclusion that if we use the keyword localpeer in global section while not binding any address in the peers section, then if we reload, haproxy will end to a segfault for the former worker. We did not catch any error/warning messages in the logs before the segfault happens.
Expected Behavior
The HAProxy should not be able to boot if the configuration makes worker ending to a segfault. (or it should ignore the requested behavior and log it as a warning)
Steps to Reproduce the Behavior
Copy this haproxy.cfg:
global
localpeer local
daemon
maxconn 256
peers local# Persist sticky table over reloads
server local
defaults
timeout connect 3s
timeout client 30s
timeout server 30s
mode http
option httplog
log stderr local2
frontend http
bind*:8080
# add your rules here
default_backend dummy
backend dummy
http-request return status 200 content-type text/plain string "hello"
Do a reload, we see the log Former worker (7) exited with code 139 (Segmentation fault) (if you redo reloads, it ends with the same segfault)
Then change the haproxy.cfg to:
global
localpeer local
daemon
maxconn 256
peers local# Persist sticky table over reloads
server localbind 127.0.0.1:10240 <- I added this
defaults
timeout connect 3s
timeout client 30s
timeout server 30s
mode http
option httplog
log stderr local2
frontend http
bind*:8080
# add your rules here
default_backend dummy
backend dummy
http-request return status 200 content-type text/plain string "hello"
Reloads go right: [WARNING] (1) : Former worker (8) exited with code 0 (Exit)
Do you have any idea what may have caused this?
I tested many versions since 3.0.0 and I reproduce the issue only with HAProxy >= 3.4.0
Do you have an idea how to solve the issue?
No.
What is your configuration?
global
localpeer local
daemon maxconn256
peers local# Persist sticky table over reloads server local
defaults
timeout connect3s timeout client30s timeout server30s mode http option httplog log stderr local2
frontend http bind *:8080# add your rules here default_backend dummy
backend dummy http-request return status 200 content-type text/plain string "hello"
Output of haproxy -vv
root@fd30a187495a:/# haproxy -vv
HAProxy version 3.4.0-64a335366 2026/06/03 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2031.
Known bugs: http://www.haproxy.org/bugs/bugs-3.4.0.html
Running on: Linux 6.8.0-50-generic #51-Ubuntu SMP PREEMPT_DYNAMIC Sat Nov 9 18:03:35 UTC 2024 aarch64
Build options :
TARGET = linux-glibc
CC = cc
CFLAGS = -O2 -g -fwrapv -fvect-cost-model=very-cheap
OPTIONS = USE_PTHREAD_EMULATION=1 USE_LINUX_TPROXY=1 USE_GETADDRINFO=1 USE_OPENSSL_AWSLC=1 USE_LUA=1 USE_SLZ=1 USE_TFO=1 USE_QUIC=1 USE_PROMEX=1 USE_PCRE2=1 USE_PCRE2_JIT=1
DEBUG =
Feature list : -51DEGREES +ACCEPT4 +ACME +BACKTRACE -CLOSEFROM +CPU_AFFINITY +CRYPT_H -DEVICEATLAS +DL -ECH -ENGINE +EPOLL -EVPORTS +GETADDRINFO +HAVE_TCP_MD5SIG -KQUEUE +KTLS -LIBATOMIC +LIBCRYPT +LINUX_CAP +LINUX_SPLICE +LINUX_TPROXY +LUA +MATH -MEMORY_PROFILING +NETFILTER +NS -OBSOLETE_LINKER +OPENSSL +OPENSSL_AWSLC -OPENSSL_WOLFSSL -OT -PCRE +PCRE2 +PCRE2_JIT -PCRE_JIT +POLL +PRCTL -PROCCTL +PROMEX +PTHREAD_EMULATION +QUIC -QUIC_OPENSSL_COMPAT +RT +SHM_OPEN +SLZ +SSL -STATIC_PCRE -STATIC_PCRE2 +TFO +THREAD +THREAD_DUMP +TPROXY +TRACE -WURFL -ZLIB
Detected feature list : +HAVE_WORKING_TCP_MD5SIG
Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with multi-threading support (MAX_TGROUPS=32, MAX_THREADS=1024, default=2).
Built with SSL library version : OpenSSL 1.1.1 (compatible; AWS-LC 4.0.0)
Running on SSL library version : AWS-LC 4.0.0
SSL library supports TLS extensions : yes
SSL library supports SNI : yes
SSL library FIPS mode : no
SSL library default verify directory : /etc/ssl/certs
SSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
QUIC: connection sock-per-conn mode support : yes
QUIC: GSO emission support : yes
Built with Lua version : Lua 5.4.7
Built with the Prometheus exporter as a service
Built with network namespace support.
Built with libslz for stateless compression.
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.46 2025-08-27
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 14.2.0
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
qmux : mode=HTTP side=FE|BE mux=QMUX flags=HTX|NO_UPG
quic : mode=HTTP side=FE|BE mux=QUIC flags=HTX|NO_UPG|FRAMED
h2 : mode=HTTP side=FE|BE mux=H2 flags=HTX|HOL_RISK|NO_UPG
<default> : mode=HTTP side=FE|BE mux=H1 flags=HTX
h1 : mode=HTTP side=FE|BE mux=H1 flags=HTX|NO_UPG
fcgi : mode=HTTP side=BE mux=FCGI flags=HTX|HOL_RISK|NO_UPG
<default> : mode=SPOP side=BE mux=SPOP flags=HOL_RISK|NO_UPG
spop : mode=SPOP side=BE mux=SPOP flags=HOL_RISK|NO_UPG
<default> : mode=TCP side=FE|BE mux=PASS flags=
none : mode=TCP side=FE|BE mux=PASS flags=NO_UPG
Available services : prometheus-exporter
Available filters :
[BWLIM] bwlim-in
[BWLIM] bwlim-out
[CACHE] cache
[COMP] comp-req
[COMP] comp-res
[COMP] compression
[FCGI] fcgi-app
[SPOE] spoe
[TRACE] trace
Last Outputs and Backtraces
[NOTICE] (1) : Initializing new worker (8)
[NOTICE] (8) : haproxy version is 3.4.0-64a335366
[NOTICE] (8) : path to executable is /usr/local/sbin/haproxy
[WARNING] (8) : [haproxy.main()] HAProxy was started as root without any 'chroot' directive. A chroot limits filesystem access of an intruder to a single, preferably empty, directory. It is strongly recommended to enable this feature whenever possible (it's always possible when starting as root), via 'chroot auto' in the global section. If you think you have good reasons for running outside a chroot, explicitly configure 'chroot /' to silence this warning.
[WARNING] (8) : [haproxy.main()] HAProxy was started as the root user and does not make use of 'user' nor 'uid' global options to drop the privileges. This is generally considered as a bad practice security-wise. If running as root is intentional, please make it explicit using 'uid 0' or 'user root', and also please consider using the 'chroot' directive to isolate the process into a totally empty and read-only directory if possible. Also, since your operating system supports it, always prefer relying on capabilities with unprivileged users than running with full privileges (look for 'setcap' in the configurationmanual).
[NOTICE] (1) : Loading success.
[NOTICE] (1) : Reloading HAProxy
[NOTICE] (1) : Initializing new worker (17)
[NOTICE] (17) : haproxy version is 3.4.0-64a335366
[NOTICE] (17) : path to executable is /usr/local/sbin/haproxy
[WARNING] (17) : [haproxy.main()] HAProxy was started as root without any 'chroot' directive. A chroot limits filesystem access of an intruder to a single, preferably empty, directory. It is strongly recommended to enable this feature whenever possible (it's always possible when starting as root), via 'chroot auto' in the global section. If you think you have good reasons for running outside a chroot, explicitly configure 'chroot /' to silence this warning.
[WARNING] (17) : [haproxy.main()] HAProxy was started as the root user and does not make use of 'user' nor 'uid' global options to drop the privileges. This is generally considered as a bad practice security-wise. If running as root is intentional, please make it explicit using 'uid 0' or 'user root', and also please consider using the 'chroot' directive to isolate the process into a totally empty and read-only directory if possible. Also, since your operating system supports it, always prefer relying on capabilities with unprivileged users than running with full privileges (look for 'setcap' in the configurationmanual).
[NOTICE] (1) : Loading success.
[WARNING] (8) : Proxy http stopped (cumulated conns: FE: 0, BE: 0).
<148>Sep 29 11:47:45 haproxy[8]: Proxy http stopped (cumulated conns: FE: 0, BE: 0).
[WARNING] (8) : Proxy dummy stopped (cumulated conns: FE: 0, BE: 0).
<148>Sep 29 11:47:45 haproxy[8]: Proxy dummy stopped (cumulated conns: FE: 0, BE: 0).
[NOTICE] (1) : haproxy version is 3.4.0-64a335366
[NOTICE] (1) : path to executable is /usr/local/sbin/haproxy
[WARNING] (1) : Former worker (8) exited with code 139 (Segmentation fault)
changed the title [-][3.4.x] localpeer without bind results in segfault[/-][+][3.4.x] localpeer without bind results in worker segfault[/+]on Sep 29, 2026
thanks for the detailed report, I'm able to reproduce given the provided config file. And indeed you are right if the configuration is invalid it should trigger an error and not lead to the process crashing. I'll do a few tests to check whether this is a regression or not and then try to find a fix.
It really started crashing since commit 879c78c. Prior to that it was simply "ignored", the configuration wasn't harmful. Either we decide this is an error starting from 3.4, or we make the code more robust to prevent the crash.
I do not know the philosophy of HAProxy on these kind of questions but IMO if you decide to make the code more robust (cf.: no error), it should not bait the user making think that everything works as expected in the configuration file while its not really behaving as expected
If you cannot make it like this, I think that an error message is better
I am thinking of the ulimit-n parameter (if my memory is good) which make crashing the haproxy at the startup time if its not correct while it could just set file descriptor limits to the max possible and show a warning message.
Interesting. Regarding the purely peer aspect, I agree that such invalid configuration should be rejected. This is already stated as not correct in the documentation so I do not consider this a breaking change.
Nevertheless, there may be an issue with connect_server() for other cases. I'm not sure if it's possible to really have a connection without its ctrl member set here, but at least do_connect_server() which is called later checks for a possible NULL value. Maybe it's necessary to check for a NULL conn->ctrl in conn_select_mux_be() and return NULL in this caes.
I think you are right, it doesn't crash if server local is removed. This means the config is obviously incomplete because using server without an address means the peer new syntax (server + bind) is assumed, and in this case a bind line MUST be provided. I'll see if we can detect that and report an error in this case.
Here is a patch that implements the behavior described just above. @a-denoyelle told me he has concerns about the fact that we introduce a new warning that may not be justified. I was afraid of emitting a fatal error since such config used to work by accident and caused haproxy to simply make the peer section ineffective, and perhaps existing config could rely on this behavior to disable a peers section without commenting it, I don't know.
@wtarreau@haproxyFred what are your thoughts about this? Should we ignore and warn as we already do when the local peer declaration is missing in a peers section? Or raise a fatal error instead, preventing haproxy from starting at all?
Activity
[-][3.4.x] localpeer without bind results in segfault[/-][+][3.4.x] localpeer without bind results in worker segfault[/+]Darlelet commentedon Sep 29, 2026
Hi,
thanks for the detailed report, I'm able to reproduce given the provided config file. And indeed you are right if the configuration is invalid it should trigger an error and not lead to the process crashing. I'll do a few tests to check whether this is a regression or not and then try to find a fix.
Darlelet commentedon Sep 29, 2026
It really started crashing since commit 879c78c. Prior to that it was simply "ignored", the configuration wasn't harmful. Either we decide this is an error starting from 3.4, or we make the code more robust to prevent the crash.
alexandret-ik commentedon Sep 29, 2026
I do not know the philosophy of HAProxy on these kind of questions but IMO if you decide to make the code more robust (cf.: no error), it should not bait the user making think that everything works as expected in the configuration file while its not really behaving as expected
If you cannot make it like this, I think that an error message is better
I am thinking of the
ulimit-nparameter (if my memory is good) which make crashing the haproxy at the startup time if its not correct while it could just set file descriptor limits to the max possible and show a warning message.a-denoyelle commentedon Sep 29, 2026
Interesting. Regarding the purely peer aspect, I agree that such invalid configuration should be rejected. This is already stated as not correct in the documentation so I do not consider this a breaking change.
Nevertheless, there may be an issue with
connect_server()for other cases. I'm not sure if it's possible to really have a connection without itsctrlmember set here, but at leastdo_connect_server()which is called later checks for a possible NULL value. Maybe it's necessary to check for a NULLconn->ctrlinconn_select_mux_be()and return NULL in this caes.Darlelet commentedon Oct 8, 2026
I think you are right, it doesn't crash if
server localis removed. This means the config is obviously incomplete because usingserverwithout an address means the peer new syntax (server + bind) is assumed, and in this case a bind line MUST be provided. I'll see if we can detect that and report an error in this case.Darlelet commentedon Oct 8, 2026
Hmm thinking about it, best is probably a warning + ignoring the section as we already do for missing local peer declaration.
Darlelet commentedon Oct 9, 2026
Here is a patch that implements the behavior described just above. @a-denoyelle told me he has concerns about the fact that we introduce a new warning that may not be justified. I was afraid of emitting a fatal error since such config used to work by accident and caused haproxy to simply make the peer section ineffective, and perhaps existing config could rely on this behavior to disable a peers section without commenting it, I don't know.
@wtarreau @haproxyFred what are your thoughts about this? Should we ignore and warn as we already do when the local peer declaration is missing in a peers section? Or raise a fatal error instead, preventing haproxy from starting at all?
0001-BUG-MINOR-cfgparse-peers-warn-and-prevent-undefined-.patch
BUG/MINOR: cfgparse/peers: warn and prevent undefined behavior when p…