[3.4.x] localpeer without bind results in worker segfault · Issue #3516 · haproxy/haproxy · GitHub
Skip to content

[3.4.x] localpeer without bind results in worker segfault #3516

Open

Description

@alexandret-ik

Detailed Description of the Problem

Hello,

We figured out in production that a faulty configuration file could result in a SEGFAULT when reloading haproxy.

We tested many case and ended to the conclusion that if we use the keyword localpeer in global section while not binding any address in the peers section, then if we reload, haproxy will end to a segfault for the former worker. We did not catch any error/warning messages in the logs before the segfault happens.

Expected Behavior

The HAProxy should not be able to boot if the configuration makes worker ending to a segfault. (or it should ignore the requested behavior and log it as a warning)

Steps to Reproduce the Behavior

Copy this haproxy.cfg:

global
  localpeer local
  daemon
  maxconn 256

peers local
  # Persist sticky table over reloads
  server local

defaults
  timeout connect 3s
  timeout client 30s
  timeout server 30s
  mode http
  option httplog
  log stderr local2

frontend http
  bind *:8080

  # add your rules here

  default_backend dummy

backend dummy
  http-request return status 200 content-type text/plain string "hello"

Do a reload, we see the log Former worker (7) exited with code 139 (Segmentation fault) (if you redo reloads, it ends with the same segfault)

Then change the haproxy.cfg to:

global
  localpeer local
  daemon
  maxconn 256

peers local
  # Persist sticky table over reloads
  server local
  bind 127.0.0.1:10240 <- I added this

defaults
  timeout connect 3s
  timeout client 30s
  timeout server 30s
  mode http
  option httplog
  log stderr local2

frontend http
  bind *:8080

  # add your rules here

  default_backend dummy

backend dummy
  http-request return status 200 content-type text/plain string "hello"

Reloads go right: [WARNING] (1) : Former worker (8) exited with code 0 (Exit)

Do you have any idea what may have caused this?

I tested many versions since 3.0.0 and I reproduce the issue only with HAProxy >= 3.4.0

Do you have an idea how to solve the issue?

No.

What is your configuration?

global
  localpeer local
  daemon
  maxconn 256

peers local
  # Persist sticky table over reloads
  server local

defaults
  timeout connect 3s
  timeout client 30s
  timeout server 30s
  mode http
  option httplog
  log stderr local2

frontend http
  bind *:8080

  # add your rules here

  default_backend dummy

backend dummy
  http-request return status 200 content-type text/plain string "hello"

Output of haproxy -vv

root@fd30a187495a:/# haproxy -vv
HAProxy version 3.4.0-64a335366 2026/06/03 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2031.
Known bugs: http://www.haproxy.org/bugs/bugs-3.4.0.html
Running on: Linux 6.8.0-50-generic #51-Ubuntu SMP PREEMPT_DYNAMIC Sat Nov  9 18:03:35 UTC 2024 aarch64
Build options :
  TARGET  = linux-glibc
  CC      = cc
  CFLAGS  = -O2 -g -fwrapv -fvect-cost-model=very-cheap
  OPTIONS = USE_PTHREAD_EMULATION=1 USE_LINUX_TPROXY=1 USE_GETADDRINFO=1 USE_OPENSSL_AWSLC=1 USE_LUA=1 USE_SLZ=1 USE_TFO=1 USE_QUIC=1 USE_PROMEX=1 USE_PCRE2=1 USE_PCRE2_JIT=1
  DEBUG   =

Feature list : -51DEGREES +ACCEPT4 +ACME +BACKTRACE -CLOSEFROM +CPU_AFFINITY +CRYPT_H -DEVICEATLAS +DL -ECH -ENGINE +EPOLL -EVPORTS +GETADDRINFO +HAVE_TCP_MD5SIG -KQUEUE +KTLS -LIBATOMIC +LIBCRYPT +LINUX_CAP +LINUX_SPLICE +LINUX_TPROXY +LUA +MATH -MEMORY_PROFILING +NETFILTER +NS -OBSOLETE_LINKER +OPENSSL +OPENSSL_AWSLC -OPENSSL_WOLFSSL -OT -PCRE +PCRE2 +PCRE2_JIT -PCRE_JIT +POLL +PRCTL -PROCCTL +PROMEX +PTHREAD_EMULATION +QUIC -QUIC_OPENSSL_COMPAT +RT +SHM_OPEN +SLZ +SSL -STATIC_PCRE -STATIC_PCRE2 +TFO +THREAD +THREAD_DUMP +TPROXY +TRACE -WURFL -ZLIB
Detected feature list : +HAVE_WORKING_TCP_MD5SIG

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_TGROUPS=32, MAX_THREADS=1024, default=2).
Built with SSL library version : OpenSSL 1.1.1 (compatible; AWS-LC 4.0.0)
Running on SSL library version : AWS-LC 4.0.0
SSL library supports TLS extensions : yes
SSL library supports SNI : yes
SSL library FIPS mode : no
SSL library default verify directory : /etc/ssl/certs
SSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
QUIC: connection sock-per-conn mode support : yes
QUIC: GSO emission support : yes
Built with Lua version : Lua 5.4.7
Built with the Prometheus exporter as a service
Built with network namespace support.
Built with libslz for stateless compression.
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.46 2025-08-27
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 14.2.0

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
       qmux : mode=HTTP  side=FE|BE  mux=QMUX  flags=HTX|NO_UPG
       quic : mode=HTTP  side=FE|BE  mux=QUIC  flags=HTX|NO_UPG|FRAMED
         h2 : mode=HTTP  side=FE|BE  mux=H2    flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=HTTP  side=FE|BE  mux=H1    flags=HTX
         h1 : mode=HTTP  side=FE|BE  mux=H1    flags=HTX|NO_UPG
       fcgi : mode=HTTP  side=BE     mux=FCGI  flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=SPOP  side=BE     mux=SPOP  flags=HOL_RISK|NO_UPG
       spop : mode=SPOP  side=BE     mux=SPOP  flags=HOL_RISK|NO_UPG
  <default> : mode=TCP   side=FE|BE  mux=PASS  flags=
       none : mode=TCP   side=FE|BE  mux=PASS  flags=NO_UPG

Available services : prometheus-exporter
Available filters :
        [BWLIM] bwlim-in
        [BWLIM] bwlim-out
        [CACHE] cache
        [COMP] comp-req
        [COMP] comp-res
        [COMP] compression
        [FCGI] fcgi-app
        [SPOE] spoe
        [TRACE] trace

Last Outputs and Backtraces

[NOTICE]   (1) : Initializing new worker (8)
[NOTICE]   (8) : haproxy version is 3.4.0-64a335366
[NOTICE]   (8) : path to executable is /usr/local/sbin/haproxy
[WARNING]  (8) : [haproxy.main()] HAProxy was started as root without any 'chroot' directive. A chroot limits filesystem access of an intruder to a single, preferably empty, directory. It is strongly recommended to enable this feature whenever possible (it's always possible when starting as root), via 'chroot auto' in the global section. If you think you have good reasons for running outside a chroot, explicitly configure 'chroot /' to silence this warning.
[WARNING]  (8) : [haproxy.main()] HAProxy was started as the root user and does not make use of 'user' nor 'uid' global options to drop the privileges. This is generally considered as a bad practice security-wise. If running as root is intentional, please make it explicit using 'uid 0' or 'user root', and also please consider using the 'chroot' directive to isolate the process into a totally empty and read-only directory if possible. Also, since your operating system supports it, always prefer relying on capabilities with unprivileged users than running with full privileges (look for 'setcap' in the configurationmanual).
[NOTICE]   (1) : Loading success.
[NOTICE]   (1) : Reloading HAProxy
[NOTICE]   (1) : Initializing new worker (17)
[NOTICE]   (17) : haproxy version is 3.4.0-64a335366
[NOTICE]   (17) : path to executable is /usr/local/sbin/haproxy
[WARNING]  (17) : [haproxy.main()] HAProxy was started as root without any 'chroot' directive. A chroot limits filesystem access of an intruder to a single, preferably empty, directory. It is strongly recommended to enable this feature whenever possible (it's always possible when starting as root), via 'chroot auto' in the global section. If you think you have good reasons for running outside a chroot, explicitly configure 'chroot /' to silence this warning.
[WARNING]  (17) : [haproxy.main()] HAProxy was started as the root user and does not make use of 'user' nor 'uid' global options to drop the privileges. This is generally considered as a bad practice security-wise. If running as root is intentional, please make it explicit using 'uid 0' or 'user root', and also please consider using the 'chroot' directive to isolate the process into a totally empty and read-only directory if possible. Also, since your operating system supports it, always prefer relying on capabilities with unprivileged users than running with full privileges (look for 'setcap' in the configurationmanual).
[NOTICE]   (1) : Loading success.
[WARNING]  (8) : Proxy http stopped (cumulated conns: FE: 0, BE: 0).
<148>Sep 29 11:47:45 haproxy[8]: Proxy http stopped (cumulated conns: FE: 0, BE: 0).
[WARNING]  (8) : Proxy dummy stopped (cumulated conns: FE: 0, BE: 0).
<148>Sep 29 11:47:45 haproxy[8]: Proxy dummy stopped (cumulated conns: FE: 0, BE: 0).
[NOTICE]   (1) : haproxy version is 3.4.0-64a335366
[NOTICE]   (1) : path to executable is /usr/local/sbin/haproxy
[WARNING]  (1) : Former worker (8) exited with code 139 (Segmentation fault)

Additional Information

For info, I am reproducing the issue in a docker environment with this image: https://hub.docker.com/r/haproxytech/haproxy-debian

Activity

  1. changed the title [-][3.4.x] localpeer without bind results in segfault[/-] [+][3.4.x] localpeer without bind results in worker segfault[/+] on Sep 29, 2026
  2. Darlelet commented on Sep 29, 2026

    @Darlelet
    Contributor

    Hi,

    thanks for the detailed report, I'm able to reproduce given the provided config file. And indeed you are right if the configuration is invalid it should trigger an error and not lead to the process crashing. I'll do a few tests to check whether this is a regression or not and then try to find a fix.

  3. added
    status: reviewedThis issue was reviewed. A fix is required.
    and removed on Sep 29, 2026
  4. Darlelet commented on Sep 29, 2026

    @Darlelet
    Contributor

    It really started crashing since commit 879c78c. Prior to that it was simply "ignored", the configuration wasn't harmful. Either we decide this is an error starting from 3.4, or we make the code more robust to prevent the crash.

  5. added
    devThis issue affects the HAProxy development branch.
    3.4This issue affects the HAProxy 3.4 stable branch.
    on Sep 29, 2026
  6. alexandret-ik commented on Sep 29, 2026

    @alexandret-ik
    Author

    I do not know the philosophy of HAProxy on these kind of questions but IMO if you decide to make the code more robust (cf.: no error), it should not bait the user making think that everything works as expected in the configuration file while its not really behaving as expected

    If you cannot make it like this, I think that an error message is better

    I am thinking of the ulimit-n parameter (if my memory is good) which make crashing the haproxy at the startup time if its not correct while it could just set file descriptor limits to the max possible and show a warning message.

  7. a-denoyelle commented on Sep 29, 2026

    @a-denoyelle
    Contributor

    Interesting. Regarding the purely peer aspect, I agree that such invalid configuration should be rejected. This is already stated as not correct in the documentation so I do not consider this a breaking change.

    Nevertheless, there may be an issue with connect_server() for other cases. I'm not sure if it's possible to really have a connection without its ctrl member set here, but at least do_connect_server() which is called later checks for a possible NULL value. Maybe it's necessary to check for a NULL conn->ctrl in conn_select_mux_be() and return NULL in this caes.

  8. Darlelet commented on Oct 8, 2026

    @Darlelet
    Contributor

    I think you are right, it doesn't crash if server local is removed. This means the config is obviously incomplete because using server without an address means the peer new syntax (server + bind) is assumed, and in this case a bind line MUST be provided. I'll see if we can detect that and report an error in this case.

  9. Darlelet commented on Oct 8, 2026

    @Darlelet
    Contributor

    Hmm thinking about it, best is probably a warning + ignoring the section as we already do for missing local peer declaration.

  10. Darlelet commented on Oct 9, 2026

    @Darlelet
    Contributor

    Here is a patch that implements the behavior described just above. @a-denoyelle told me he has concerns about the fact that we introduce a new warning that may not be justified. I was afraid of emitting a fatal error since such config used to work by accident and caused haproxy to simply make the peer section ineffective, and perhaps existing config could rely on this behavior to disable a peers section without commenting it, I don't know.

    @wtarreau @haproxyFred what are your thoughts about this? Should we ignore and warn as we already do when the local peer declaration is missing in a peers section? Or raise a fatal error instead, preventing haproxy from starting at all?

    0001-BUG-MINOR-cfgparse-peers-warn-and-prevent-undefined-.patch

  11. added
    status: fixedThis issue is a now-fixed bug.
    and removed
    devThis issue affects the HAProxy development branch.
    status: reviewedThis issue was reviewed. A fix is required.
    on Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.4This issue affects the HAProxy 3.4 stable branch.status: fixedThis issue is a now-fixed bug.type: bugThis issue describes a bug.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions