EventMachine Segmentation Faults With Ruby Built With Jemalloc on Ubuntu · Issue #965 · eventmachine/eventmachine · GitHub
Skip to content

EventMachine Segmentation Faults With Ruby Built With Jemalloc on Ubuntu #965

Open
Open
EventMachine Segmentation Faults With Ruby Built With Jemalloc on Ubuntu#965

Description

@eviljoel
eviljoel
opened on Mar 15, 2022

When I run "::EventMachine.run({return})" (EventMachine 1.2.7) on Ubuntu 20.04 (x86_64) with Ruby 2.7.4 compiled with Jemalloc 5.2.1, EventMachine segmentation faults. Ruby must also be compiled with the --enable-shared configuration option. I can also reproduce the issue on Ubuntu 18.04 and Ubuntu 21.10. I have not be able to reproduce the error on CentOS 7 or Almalinux 8.

Here is a minimal Dockerfile to reproduce the issue:

FROM ubuntu:focal
RUN apt --assume-yes update
RUN apt --assume-yes install build-essential curl libssl-dev libz-dev
RUN curl -LO https://cache.ruby-lang.org/pub/ruby/2.7/ruby-2.7.4.tar.xz
RUN echo "2a80824e0ad6100826b69b9890bf55cfc4cf2b61a1e1330fccbcb30c46cef8d7 ruby-2.7.4.tar.xz" | sha256sum -c -
RUN tar -xvf ruby-2.7.4.tar.xz
RUN curl -LO https://github.com/jemalloc/jemalloc/releases/download/5.2.1/jemalloc-5.2.1.tar.bz2
RUN echo "34330e5ce276099e2e8950d9335db5a875689a4c6a56751ef3b1d8c537f887f6 jemalloc-5.2.1.tar.bz2" | sha256sum -c -
RUN tar -xvf jemalloc-5.2.1.tar.bz2
RUN cd jemalloc-5.2.1 && ./configure
RUN cd jemalloc-5.2.1 && make
RUN cd jemalloc-5.2.1 && make install
RUN cd ruby-2.7.4 && ./configure --with-jemalloc --enable-shared --with-opt-dir=/usr/local
RUN cd ruby-2.7.4 && make
RUN cd ruby-2.7.4 && make install
RUN /usr/local/bin/gem install eventmachine
RUN /usr/local/bin/ruby -r eventmachine -e '::EventMachine.run {return}'

I hacked around the EventMachine extension's C++ source to try to isolate the cause. I removed the EventMachine_t destructor, removed all code from the EventMachine_t constructor, and changed evma_initialize_library to simply contain the following:

EventMachine_t* EventMachine = new EventMachine_t ();
delete EventMachine;

This simple code causes EventMachine to crash. The program segmentation faults on the 'delete' statement.

When I tried to isolate the problem further by removing almost all other symbols and methods from the program, the segmentation fault no longer occurs. This leads me to believe this is an EventMachine problem and not a Jemalloc or Ruby problem. I could probably narrow the problem down further, but I'm having a hard time justifying to my employer the amount of research I've already put into this.

Here is the segmentation fault:

/usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/eventmachine.rb:203: [BUG] Segmentation fault at 0x000000000012cdd0
ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0005 p:---- s:0022 e:000021 CFUNC  :release_machine
c:0004 p:0027 s:0018 e:000017 RESCUE /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/eventmachine.rb:203
c:0003 p:0306 s:0014 e:000013 METHOD /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/eventmachine.rb:206
c:0002 p:0015 s:0006 E:000d40 EVAL   -e:1 [FINISH]
c:0001 p:0000 s:0003 E:0015b0 (none) [FINISH]

-- Ruby level backtrace information ----------------------------------------
-e:1:in `<main>'
/usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/eventmachine.rb:206:in `run'
/usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/eventmachine.rb:203:in `ensure in run'
/usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/eventmachine.rb:203:in `release_machine'

-- Machine register context ------------------------------------------------
 RIP: 0x00007f29c43507aa RBP: 0x0000000000000001 RSP: 0x00007ffc27d86b00
 RAX: 0x000000000012cdd0 RBX: 0x0000000000000008 RCX: 0x00005576659ba3b0
 RDX: 0x0000000000000070 RDI: 0x00007f29c3fa8768 RSI: 0x00007f29c3fa87d8
  R8: 0x0000000000000001  R9: 0x0000000000000001 R10: 0x00007f29c44872d8
 R11: 0x0000000000000000 R12: 0x00007f29c3fa86a8 R13: 0x00005576659ba3b0
 R14: 0x0000557640000000 R15: 0xffffffffffffe000 EFL: 0x0000000000010202

-- C level backtrace information -------------------------------------------
/usr/local/lib/libruby.so.2.7(rb_print_backtrace+0x19) [0x7f29c4a5a059] vm_dump.c:755
/usr/local/lib/libruby.so.2.7(rb_vm_bugreport+0x21c) [0x7f29c4a5a28c] vm_dump.c:941
[0x7f29c48d407b]
/usr/local/lib/libruby.so.2.7(sigsegv+0x4d) [0x7f29c49d773d] signal.c:946
/lib/x86_64-linux-gnu/libc.so.6(0x7f29c46720c0) [0x7f29c46720c0]
/usr/local/lib/libjemalloc.so.2(je_sdallocx_default+0x3da) [0x7f29c43507aa] include/jemalloc/internal/rtree.h:341
/usr/local/lib/libjemalloc.so.2(operator delete+0x12) [0x7f29c43bedb2] src/jemalloc_cpp.cpp:131
/usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/rubyeventmachine.so(evma_release_library+0x26) [0x7f29c3e17a96] cmain.cpp:72
/usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/rubyeventmachine.so(evma_release_library) (null):0
/usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/rubyeventmachine.so(t_release_machine+0xd) [0x7f29c3e2257d] rubymain.cpp:876
[0x7f29c4a357e4]
[0x7f29c4a49dab]
/usr/local/lib/libruby.so.2.7(vm_sendish+0x76) [0x7f29c4a30946] vm_insnhelper.c:4023
[0x7f29c4a4159a]
[0x7f29c4a478f2]
/usr/local/lib/libruby.so.2.7(rb_ec_exec_node+0xc1) [0x7f29c48d82d1] eval.c:278
/usr/local/lib/libruby.so.2.7(ruby_run_node+0x4e) [0x7f29c48dd10e] eval.c:336
/usr/local/bin/ruby(main+0x5f) [0x55766438c17f] ./main.c:50

-- Other runtime information -----------------------------------------------

* Loaded script: -e

* Loaded features:

    0 enumerator.so
    1 thread.rb
    2 rational.so
    3 complex.so
    4 ruby2_keywords.rb
    5 /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/encdb.so
    6 /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/trans/transdb.so
    7 /usr/local/lib/ruby/2.7.0/x86_64-linux/rbconfig.rb
    8 /usr/local/lib/ruby/2.7.0/rubygems/compatibility.rb
    9 /usr/local/lib/ruby/2.7.0/rubygems/defaults.rb
   10 /usr/local/lib/ruby/2.7.0/rubygems/deprecate.rb
   11 /usr/local/lib/ruby/2.7.0/rubygems/errors.rb
   12 /usr/local/lib/ruby/2.7.0/rubygems/version.rb
   13 /usr/local/lib/ruby/2.7.0/rubygems/requirement.rb
   14 /usr/local/lib/ruby/2.7.0/rubygems/platform.rb
   15 /usr/local/lib/ruby/2.7.0/rubygems/basic_specification.rb
   16 /usr/local/lib/ruby/2.7.0/rubygems/stub_specification.rb
   17 /usr/local/lib/ruby/2.7.0/rubygems/util.rb
   18 /usr/local/lib/ruby/2.7.0/rubygems/text.rb
   19 /usr/local/lib/ruby/2.7.0/rubygems/user_interaction.rb
   20 /usr/local/lib/ruby/2.7.0/rubygems/specification_policy.rb
   21 /usr/local/lib/ruby/2.7.0/rubygems/util/list.rb
   22 /usr/local/lib/ruby/2.7.0/rubygems/specification.rb
   23 /usr/local/lib/ruby/2.7.0/rubygems/exceptions.rb
   24 /usr/local/lib/ruby/2.7.0/rubygems/bundler_version_finder.rb
   25 /usr/local/lib/ruby/2.7.0/rubygems/dependency.rb
   26 /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_gem.rb
   27 /usr/local/lib/ruby/2.7.0/x86_64-linux/monitor.so
   28 /usr/local/lib/ruby/2.7.0/monitor.rb
   29 /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb
   30 /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_warn.rb
   31 /usr/local/lib/ruby/2.7.0/rubygems.rb
   32 /usr/local/lib/ruby/2.7.0/rubygems/path_support.rb
   33 /usr/local/lib/ruby/2.7.0/did_you_mean/version.rb
   34 /usr/local/lib/ruby/2.7.0/did_you_mean/core_ext/name_error.rb
   35 /usr/local/lib/ruby/2.7.0/did_you_mean/levenshtein.rb
   36 /usr/local/lib/ruby/2.7.0/did_you_mean/jaro_winkler.rb
   37 /usr/local/lib/ruby/2.7.0/did_you_mean/spell_checker.rb
   38 /usr/local/lib/ruby/2.7.0/did_you_mean/spell_checkers/name_error_checkers/class_name_checker.rb
   39 /usr/local/lib/ruby/2.7.0/did_you_mean/spell_checkers/name_error_checkers/variable_name_checker.rb
   40 /usr/local/lib/ruby/2.7.0/did_you_mean/spell_checkers/name_error_checkers.rb
   41 /usr/local/lib/ruby/2.7.0/did_you_mean/spell_checkers/method_name_checker.rb
   42 /usr/local/lib/ruby/2.7.0/did_you_mean/spell_checkers/key_error_checker.rb
   43 /usr/local/lib/ruby/2.7.0/did_you_mean/spell_checkers/null_checker.rb
   44 /usr/local/lib/ruby/2.7.0/did_you_mean/formatters/plain_formatter.rb
   45 /usr/local/lib/ruby/2.7.0/did_you_mean/tree_spell_checker.rb
   46 /usr/local/lib/ruby/2.7.0/did_you_mean.rb
   47 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/rubyeventmachine.so
   48 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/version.rb
   49 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/pool.rb
   50 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/deferrable.rb
   51 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/future.rb
   52 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/streamer.rb
   53 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/spawnable.rb
   54 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/processes.rb
   55 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/iterator.rb
   56 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/buftok.rb
   57 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/timers.rb
   58 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/protocols.rb
   59 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/connection.rb
   60 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/callback.rb
   61 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/queue.rb
   62 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/channel.rb
   63 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/file_watch.rb
   64 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/process_watch.rb
   65 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/tick_loop.rb
   66 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/resolver.rb
   67 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/completion.rb
   68 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/em/threaded_resource.rb
   69 /usr/local/lib/ruby/2.7.0/shellwords.rb
   70 /usr/local/lib/ruby/2.7.0/x86_64-linux/socket.so
   71 /usr/local/lib/ruby/2.7.0/x86_64-linux/io/wait.so
   72 /usr/local/lib/ruby/2.7.0/socket.rb
   73 /usr/local/lib/ruby/2.7.0/timeout.rb
   74 /usr/local/lib/ruby/2.7.0/securerandom.rb
   75 /usr/local/lib/ruby/2.7.0/resolv.rb
   76 /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/eventmachine.rb

* Process memory map:

55766438b000-55766438c000 r--p 00000000 00:5f 18110254                   /usr/local/bin/ruby
55766438c000-55766438d000 r-xp 00001000 00:5f 18110254                   /usr/local/bin/ruby
55766438d000-55766438e000 r--p 00002000 00:5f 18110254                   /usr/local/bin/ruby
55766438e000-55766438f000 r--p 00002000 00:5f 18110254                   /usr/local/bin/ruby
55766438f000-557664390000 rw-p 00003000 00:5f 18110254                   /usr/local/bin/ruby
55766582d000-557665dec000 rw-p 00000000 00:00 0                          [heap]
7f29be4fa000-7f29beb78000 r--s 00000000 00:5f 17630603                   /usr/local/lib/libjemalloc.so.2
7f29beb78000-7f29bed68000 r--s 00000000 00:5f 17324466                   /usr/lib/x86_64-linux-gnu/libc-2.31.so
7f29bed68000-7f29bf5c9000 rw-p 00000000 00:00 0
7f29bf5c9000-7f29bfe0c000 r--s 00000000 00:5f 18110288                   /usr/local/lib/libruby.so.2.7.4
7f29bfe0c000-7f29bfe12000 r--s 00000000 00:5f 18110254                   /usr/local/bin/ruby
7f29bfe12000-7f29bfe13000 r--p 00000000 00:5f 18111425                   /usr/local/lib/ruby/2.7.0/x86_64-linux/io/wait.so
7f29bfe13000-7f29bfe14000 r-xp 00001000 00:5f 18111425                   /usr/local/lib/ruby/2.7.0/x86_64-linux/io/wait.so
7f29bfe14000-7f29bfe15000 r--p 00002000 00:5f 18111425                   /usr/local/lib/ruby/2.7.0/x86_64-linux/io/wait.so
7f29bfe15000-7f29bfe16000 r--p 00002000 00:5f 18111425                   /usr/local/lib/ruby/2.7.0/x86_64-linux/io/wait.so
7f29bfe16000-7f29bfe17000 rw-p 00003000 00:5f 18111425                   /usr/local/lib/ruby/2.7.0/x86_64-linux/io/wait.so
7f29bfe17000-7f29bfe18000 r--p 00000000 00:5f 17324451                   /usr/lib/x86_64-linux-gnu/libanl-2.31.so
7f29bfe18000-7f29bfe1a000 r-xp 00001000 00:5f 17324451                   /usr/lib/x86_64-linux-gnu/libanl-2.31.so
7f29bfe1a000-7f29bfe1b000 r--p 00003000 00:5f 17324451                   /usr/lib/x86_64-linux-gnu/libanl-2.31.so
7f29bfe1b000-7f29bfe1c000 r--p 00003000 00:5f 17324451                   /usr/lib/x86_64-linux-gnu/libanl-2.31.so
7f29bfe1c000-7f29bfe1d000 rw-p 00004000 00:5f 17324451                   /usr/lib/x86_64-linux-gnu/libanl-2.31.so
7f29bfe1d000-7f29bfe24000 r--p 00000000 00:5f 18111440                   /usr/local/lib/ruby/2.7.0/x86_64-linux/socket.so
7f29bfe24000-7f29bfe44000 r-xp 00007000 00:5f 18111440                   /usr/local/lib/ruby/2.7.0/x86_64-linux/socket.so
7f29bfe44000-7f29bfe4c000 r--p 00027000 00:5f 18111440                   /usr/local/lib/ruby/2.7.0/x86_64-linux/socket.so
7f29bfe4c000-7f29bfe4d000 r--p 0002e000 00:5f 18111440                   /usr/local/lib/ruby/2.7.0/x86_64-linux/socket.so
7f29bfe4d000-7f29bfe4e000 rw-p 0002f000 00:5f 18111440                   /usr/local/lib/ruby/2.7.0/x86_64-linux/socket.so
7f29bfe4e000-7f29bfe6a000 r--p 00000000 00:5f 17647718                   /usr/lib/x86_64-linux-gnu/libssl.so.1.1
7f29bfe6a000-7f29bfeb9000 r-xp 0001c000 00:5f 17647718                   /usr/lib/x86_64-linux-gnu/libssl.so.1.1
7f29bfeb9000-7f29bfed3000 r--p 0006b000 00:5f 17647718                   /usr/lib/x86_64-linux-gnu/libssl.so.1.1
7f29bfed3000-7f29bfed4000 ---p 00085000 00:5f 17647718                   /usr/lib/x86_64-linux-gnu/libssl.so.1.1
7f29bfed4000-7f29bfedd000 r--p 00085000 00:5f 17647718                   /usr/lib/x86_64-linux-gnu/libssl.so.1.1
7f29bfedd000-7f29bfee1000 rw-p 0008e000 00:5f 17647718                   /usr/lib/x86_64-linux-gnu/libssl.so.1.1
7f29bfee1000-7f29bff59000 r--p 00000000 00:5f 17647613                   /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
7f29bff59000-7f29c00f4000 r-xp 00078000 00:5f 17647613                   /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
7f29c00f4000-7f29c0185000 r--p 00213000 00:5f 17647613                   /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
7f29c0185000-7f29c01b1000 r--p 002a3000 00:5f 17647613                   /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
7f29c01b1000-7f29c01b3000 rw-p 002cf000 00:5f 17647613                   /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
7f29c01b3000-7f29c01b7000 rw-p 00000000 00:00 0
7f29c01b7000-7f29c01b8000 ---p 00000000 00:00 0
7f29c01b8000-7f29c0259000 rw-p 00000000 00:00 0
7f29c0259000-7f29c025a000 ---p 00000000 00:00 0
7f29c025a000-7f29c02fb000 rw-p 00000000 00:00 0
7f29c02fb000-7f29c02fc000 ---p 00000000 00:00 0
7f29c02fc000-7f29c039d000 rw-p 00000000 00:00 0
7f29c039d000-7f29c039e000 ---p 00000000 00:00 0
7f29c039e000-7f29c043f000 rw-p 00000000 00:00 0
7f29c043f000-7f29c0440000 ---p 00000000 00:00 0
7f29c0440000-7f29c04e1000 rw-p 00000000 00:00 0
7f29c04e1000-7f29c04e2000 ---p 00000000 00:00 0
7f29c04e2000-7f29c0583000 rw-p 00000000 00:00 0
7f29c0583000-7f29c0584000 ---p 00000000 00:00 0
7f29c0584000-7f29c0625000 rw-p 00000000 00:00 0
7f29c0625000-7f29c0626000 ---p 00000000 00:00 0
7f29c0626000-7f29c06c7000 rw-p 00000000 00:00 0
7f29c06c7000-7f29c06c8000 ---p 00000000 00:00 0
7f29c06c8000-7f29c0769000 rw-p 00000000 00:00 0
7f29c0769000-7f29c076a000 ---p 00000000 00:00 0
7f29c076a000-7f29c080b000 rw-p 00000000 00:00 0
7f29c080b000-7f29c080c000 ---p 00000000 00:00 0
7f29c080c000-7f29c08ad000 rw-p 00000000 00:00 0
7f29c08ad000-7f29c08ae000 ---p 00000000 00:00 0
7f29c08ae000-7f29c094f000 rw-p 00000000 00:00 0
7f29c094f000-7f29c0950000 ---p 00000000 00:00 0
7f29c0950000-7f29c09f1000 rw-p 00000000 00:00 0
7f29c09f1000-7f29c09f2000 ---p 00000000 00:00 0
7f29c09f2000-7f29c0a93000 rw-p 00000000 00:00 0
7f29c0a93000-7f29c0a94000 ---p 00000000 00:00 0
7f29c0a94000-7f29c0b35000 rw-p 00000000 00:00 0
7f29c0b35000-7f29c0b36000 ---p 00000000 00:00 0
7f29c0b36000-7f29c0bd7000 rw-p 00000000 00:00 0
7f29c0bd7000-7f29c0bd8000 ---p 00000000 00:00 0
7f29c0bd8000-7f29c0c79000 rw-p 00000000 00:00 0
7f29c0c79000-7f29c0c7a000 ---p 00000000 00:00 0
7f29c0c7a000-7f29c0d1b000 rw-p 00000000 00:00 0
7f29c0d1b000-7f29c0d1c000 ---p 00000000 00:00 0
7f29c0d1c000-7f29c0dbd000 rw-p 00000000 00:00 0
7f29c0dbd000-7f29c0dbe000 ---p 00000000 00:00 0
7f29c0dbe000-7f29c0e5f000 rw-p 00000000 00:00 0
7f29c0e5f000-7f29c0e60000 ---p 00000000 00:00 0
7f29c0e60000-7f29c0f01000 rw-p 00000000 00:00 0
7f29c0f01000-7f29c0f02000 ---p 00000000 00:00 0
7f29c0f02000-7f29c0fa3000 rw-p 00000000 00:00 0
7f29c0fa3000-7f29c0fa4000 ---p 00000000 00:00 0
7f29c0fa4000-7f29c1045000 rw-p 00000000 00:00 0
7f29c1045000-7f29c1046000 ---p 00000000 00:00 0
7f29c1046000-7f29c10e7000 rw-p 00000000 00:00 0
7f29c10e7000-7f29c10e8000 ---p 00000000 00:00 0
7f29c10e8000-7f29c1189000 rw-p 00000000 00:00 0
7f29c1189000-7f29c118a000 ---p 00000000 00:00 0
7f29c118a000-7f29c122b000 rw-p 00000000 00:00 0
7f29c122b000-7f29c122c000 ---p 00000000 00:00 0
7f29c122c000-7f29c12cd000 rw-p 00000000 00:00 0
7f29c12cd000-7f29c12ce000 ---p 00000000 00:00 0
7f29c12ce000-7f29c136f000 rw-p 00000000 00:00 0
7f29c136f000-7f29c1370000 ---p 00000000 00:00 0
7f29c1370000-7f29c1411000 rw-p 00000000 00:00 0
7f29c1411000-7f29c1412000 ---p 00000000 00:00 0
7f29c1412000-7f29c14b3000 rw-p 00000000 00:00 0
7f29c14b3000-7f29c14b4000 ---p 00000000 00:00 0
7f29c14b4000-7f29c1555000 rw-p 00000000 00:00 0
7f29c1555000-7f29c1556000 ---p 00000000 00:00 0
7f29c1556000-7f29c3600000 rw-p 00000000 00:00 0
7f29c3600000-7f29c3e00000 rw-p 00000000 00:00 0
7f29c3e03000-7f29c3e12000 r--p 00000000 00:5f 17632876                   /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/rubyeventmachine.so
7f29c3e12000-7f29c3e27000 r-xp 0000f000 00:5f 17632876                   /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/rubyeventmachine.so
7f29c3e27000-7f29c3e30000 r--p 00024000 00:5f 17632876                   /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/rubyeventmachine.so
7f29c3e30000-7f29c3e31000 ---p 0002d000 00:5f 17632876                   /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/rubyeventmachine.so
7f29c3e31000-7f29c3e32000 r--p 0002d000 00:5f 17632876                   /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/rubyeventmachine.so
7f29c3e32000-7f29c3e34000 rw-p 0002e000 00:5f 17632876                   /usr/local/lib/ruby/gems/2.7.0/gems/eventmachine-1.2.7/lib/rubyeventmachine.so
7f29c3e34000-7f29c3e35000 r--p 00000000 00:5f 18111428                   /usr/local/lib/ruby/2.7.0/x86_64-linux/monitor.so
7f29c3e35000-7f29c3e36000 r-xp 00001000 00:5f 18111428                   /usr/local/lib/ruby/2.7.0/x86_64-linux/monitor.so
7f29c3e36000-7f29c3e37000 r--p 00002000 00:5f 18111428                   /usr/local/lib/ruby/2.7.0/x86_64-linux/monitor.so
7f29c3e37000-7f29c3e38000 r--p 00002000 00:5f 18111428                   /usr/local/lib/ruby/2.7.0/x86_64-linux/monitor.so
7f29c3e38000-7f29c3e39000 rw-p 00003000 00:5f 18111428                   /usr/local/lib/ruby/2.7.0/x86_64-linux/monitor.so
7f29c3e39000-7f29c3e3a000 r--p 00000000 00:5f 18111405                   /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/trans/transdb.so
7f29c3e3a000-7f29c3e3c000 r-xp 00001000 00:5f 18111405                   /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/trans/transdb.so
7f29c3e3c000-7f29c3e3d000 r--p 00003000 00:5f 18111405                   /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/trans/transdb.so
7f29c3e3d000-7f29c3e3e000 r--p 00003000 00:5f 18111405                   /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/trans/transdb.so
7f29c3e3e000-7f29c3e3f000 rw-p 00004000 00:5f 18111405                   /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/trans/transdb.so
7f29c3e3f000-7f29c3e40000 r--p 00000000 00:5f 18111362                   /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/encdb.so
7f29c3e40000-7f29c3e41000 r-xp 00001000 00:5f 18111362                   /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/encdb.so
7f29c3e41000-7f29c3e42000 r--p 00002000 00:5f 18111362                   /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/encdb.so
7f29c3e42000-7f29c3e43000 r--p 00002000 00:5f 18111362                   /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/encdb.so
7f29c3e43000-7f29c3e44000 rw-p 00003000 00:5f 18111362                   /usr/local/lib/ruby/2.7.0/x86_64-linux/enc/encdb.so
7f29c3e44000-7f29c3fae000 rw-p 00000000 00:00 0
7f29c3fae000-7f29c3fb1000 r--p 00000000 00:5f 17324491                   /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
7f29c3fb1000-7f29c3fc3000 r-xp 00003000 00:5f 17324491                   /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
7f29c3fc3000-7f29c3fc7000 r--p 00015000 00:5f 17324491                   /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
7f29c3fc7000-7f29c3fc8000 r--p 00018000 00:5f 17324491                   /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
7f29c3fc8000-7f29c3fc9000 rw-p 00019000 00:5f 17324491                   /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
7f29c3fc9000-7f29c405f000 r--p 00000000 00:5f 17324591                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
7f29c405f000-7f29c4150000 r-xp 00096000 00:5f 17324591                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
7f29c4150000-7f29c4199000 r--p 00187000 00:5f 17324591                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
7f29c4199000-7f29c419a000 ---p 001d0000 00:5f 17324591                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
7f29c419a000-7f29c41a5000 r--p 001d0000 00:5f 17324591                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
7f29c41a5000-7f29c41a8000 rw-p 001db000 00:5f 17324591                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
7f29c41a8000-7f29c41ab000 rw-p 00000000 00:00 0
7f29c41ab000-7f29c41b8000 r--p 00000000 00:5f 17324521                   /usr/lib/x86_64-linux-gnu/libm-2.31.so
7f29c41b8000-7f29c425f000 r-xp 0000d000 00:5f 17324521                   /usr/lib/x86_64-linux-gnu/libm-2.31.so
7f29c425f000-7f29c42f8000 r--p 000b4000 00:5f 17324521                   /usr/lib/x86_64-linux-gnu/libm-2.31.so
7f29c42f8000-7f29c42f9000 r--p 0014c000 00:5f 17324521                   /usr/lib/x86_64-linux-gnu/libm-2.31.so
7f29c42f9000-7f29c42fa000 rw-p 0014d000 00:5f 17324521                   /usr/lib/x86_64-linux-gnu/libm-2.31.so
7f29c42fa000-7f29c42fc000 r--p 00000000 00:5f 17324473                   /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0
7f29c42fc000-7f29c4311000 r-xp 00002000 00:5f 17324473                   /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0
7f29c4311000-7f29c432b000 r--p 00017000 00:5f 17324473                   /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0
7f29c432b000-7f29c432c000 r--p 00030000 00:5f 17324473                   /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0
7f29c432c000-7f29c432d000 rw-p 00031000 00:5f 17324473                   /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0
7f29c432d000-7f29c4335000 rw-p 00000000 00:00 0
7f29c4335000-7f29c4336000 r--p 00000000 00:5f 17324477                   /usr/lib/x86_64-linux-gnu/libdl-2.31.so
7f29c4336000-7f29c4338000 r-xp 00001000 00:5f 17324477                   /usr/lib/x86_64-linux-gnu/libdl-2.31.so
7f29c4338000-7f29c4339000 r--p 00003000 00:5f 17324477                   /usr/lib/x86_64-linux-gnu/libdl-2.31.so
7f29c4339000-7f29c433a000 r--p 00003000 00:5f 17324477                   /usr/lib/x86_64-linux-gnu/libdl-2.31.so
7f29c433a000-7f29c433b000 rw-p 00004000 00:5f 17324477                   /usr/lib/x86_64-linux-gnu/libdl-2.31.so
7f29c433b000-7f29c433d000 rw-p 00000000 00:00 0
7f29c433d000-7f29c4343000 r--p 00000000 00:5f 17630603                   /usr/local/lib/libjemalloc.so.2
7f29c4343000-7f29c43bf000 r-xp 00006000 00:5f 17630603                   /usr/local/lib/libjemalloc.so.2
7f29c43bf000-7f29c43cd000 r--p 00082000 00:5f 17630603                   /usr/local/lib/libjemalloc.so.2
7f29c43cd000-7f29c43ce000 ---p 00090000 00:5f 17630603                   /usr/local/lib/libjemalloc.so.2
7f29c43ce000-7f29c43d3000 r--p 00090000 00:5f 17630603                   /usr/local/lib/libjemalloc.so.2
7f29c43d3000-7f29c43d4000 rw-p 00095000 00:5f 17630603                   /usr/local/lib/libjemalloc.so.2
7f29c43d4000-7f29c45e6000 rw-p 00000000 00:00 0
7f29c45e6000-7f29c45e8000 r--p 00000000 00:5f 17324579                   /usr/lib/x86_64-linux-gnu/librt-2.31.so
7f29c45e8000-7f29c45ec000 r-xp 00002000 00:5f 17324579                   /usr/lib/x86_64-linux-gnu/librt-2.31.so
7f29c45ec000-7f29c45ee000 r--p 00006000 00:5f 17324579                   /usr/lib/x86_64-linux-gnu/librt-2.31.so
7f29c45ee000-7f29c45ef000 r--p 00007000 00:5f 17324579                   /usr/lib/x86_64-linux-gnu/librt-2.31.so
7f29c45ef000-7f29c45f0000 rw-p 00008000 00:5f 17324579                   /usr/lib/x86_64-linux-gnu/librt-2.31.so
7f29c45f0000-7f29c45f6000 r--p 00000000 00:5f 17324575                   /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
7f29c45f6000-7f29c4607000 r-xp 00006000 00:5f 17324575                   /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
7f29c4607000-7f29c460d000 r--p 00017000 00:5f 17324575                   /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
7f29c460d000-7f29c460e000 r--p 0001c000 00:5f 17324575                   /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
7f29c460e000-7f29c460f000 rw-p 0001d000 00:5f 17324575                   /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
7f29c460f000-7f29c4613000 rw-p 00000000 00:00 0
7f29c4613000-7f29c4615000 r--p 00000000 00:5f 17324611                   /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f29c4615000-7f29c4626000 r-xp 00002000 00:5f 17324611                   /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f29c4626000-7f29c462c000 r--p 00013000 00:5f 17324611                   /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f29c462c000-7f29c462d000 ---p 00019000 00:5f 17324611                   /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f29c462d000-7f29c462e000 r--p 00019000 00:5f 17324611                   /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f29c462e000-7f29c462f000 rw-p 0001a000 00:5f 17324611                   /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f29c462f000-7f29c4651000 r--p 00000000 00:5f 17324466                   /usr/lib/x86_64-linux-gnu/libc-2.31.so
7f29c4651000-7f29c47c9000 r-xp 00022000 00:5f 17324466                   /usr/lib/x86_64-linux-gnu/libc-2.31.so
7f29c47c9000-7f29c4817000 r--p 0019a000 00:5f 17324466                   /usr/lib/x86_64-linux-gnu/libc-2.31.so
7f29c4817000-7f29c481b000 r--p 001e7000 00:5f 17324466                   /usr/lib/x86_64-linux-gnu/libc-2.31.so
7f29c481b000-7f29c481d000 rw-p 001eb000 00:5f 17324466                   /usr/lib/x86_64-linux-gnu/libc-2.31.so
7f29c481d000-7f29c4821000 rw-p 00000000 00:00 0
7f29c4824000-7f29c4851000 r--p 00000000 00:5f 18110288                   /usr/local/lib/libruby.so.2.7.4
7f29c4851000-7f29c4a64000 r-xp 0002d000 00:5f 18110288                   /usr/local/lib/libruby.so.2.7.4
7f29c4a64000-7f29c4b5b000 r--p 00240000 00:5f 18110288                   /usr/local/lib/libruby.so.2.7.4
7f29c4b5b000-7f29c4b61000 r--p 00336000 00:5f 18110288                   /usr/local/lib/libruby.so.2.7.4
7f29c4b61000-7f29c4b64000 rw-p 0033c000 00:5f 18110288                   /usr/local/lib/libruby.so.2.7.4
7f29c4b64000-7f29c4b81000 rw-p 00000000 00:00 0
7f29c4b81000-7f29c4b82000 r--p 00000000 00:5f 17324444                   /usr/lib/x86_64-linux-gnu/ld-2.31.so
7f29c4b82000-7f29c4ba5000 r-xp 00001000 00:5f 17324444                   /usr/lib/x86_64-linux-gnu/ld-2.31.so
7f29c4ba5000-7f29c4bad000 r--p 00024000 00:5f 17324444                   /usr/lib/x86_64-linux-gnu/ld-2.31.so
7f29c4bae000-7f29c4baf000 r--p 0002c000 00:5f 17324444                   /usr/lib/x86_64-linux-gnu/ld-2.31.so
7f29c4baf000-7f29c4bb0000 rw-p 0002d000 00:5f 17324444                   /usr/lib/x86_64-linux-gnu/ld-2.31.so
7f29c4bb0000-7f29c4bb1000 rw-p 00000000 00:00 0
7ffc27589000-7ffc27d88000 rw-p 00000000 00:00 0                          [stack]
7ffc27db6000-7ffc27db9000 r--p 00000000 00:00 0                          [vvar]
7ffc27db9000-7ffc27dba000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]


Aborted (core dumped)

Please let me know if you think this is not an EventMachine issue and where I sould re-file this bug.

Activity

  1. sodabrew commented on Mar 15, 2022

    @sodabrew
    sodabrew
    on Mar 15, 2022
    Member

    Amazing analysis so far. Could you try with the current master branch to see if this is not already resolved by other bug fixes?

  2. eviljoel commented on Mar 15, 2022

    @eviljoel
    eviljoel
    on Mar 15, 2022
    Author

    I forgot to mention that I did try it with the 'master' branch. It also segmentation faults on 'delete'.

  3. eviljoel commented on Mar 16, 2022

    @eviljoel
    eviljoel
    on Mar 16, 2022
    Author

    OK, this turned out to be a bigger issue for me than I expected. I was able to justify spending more time to narrow down the problem.

    This is the simplest case that produces the error. Replace the contents of rubymain.cpp with the following:

    #include <ruby.h>

class EventMachine_t
{
        public:
                EventMachine_t () {}
                uint64_t array[4097];
};

extern "C" void Init_rubyeventmachine()
{
        delete new EventMachine_t ();
}

    It seems like you cannot create a C object greater than 256 32 KiB on Ubuntu. This strongly suggests that the root cause of this error is not incorrect EventMachine coding and the true underlying issue lies somewhere else.

    I'm getting this error with your code because of this line:

    struct epoll_event epoll_events [MaxEvents];

    I'm going to leave this issue open in case you want to code around this scenario.

    @sodabrew, maybe you can help me figure out where to file this underlying issue. I tried to create a simple C++ program that links to jemalloc to see if I could reproduce this issue outside of Ruby. That attempt was unsuccessful. This leads me to believe this is a jemalloc + ubuntu + ruby issue. Do you think I should file this as a bug with Ruby?

  4. sodabrew commented on Mar 16, 2022

    @sodabrew
    sodabrew
    on Mar 16, 2022
    Member

    Off the cuff, that smells like a stack size problem, as usually an array allocated by subscripts is on the stack rather than heap. I don't recall if the top of my head if C++ follows that same rule as C.

  5. eviljoel commented on Mar 17, 2022

    @eviljoel
    eviljoel
    on Mar 17, 2022
    Author

    Yeah, I came across that in my research too. However, everything I found indicates that the stack size is the same for both CentOS and Ubuntu. I printed the stack size both inside and outside of the Ruby extension. It is 4096 KiB in all cases. Also, I asked a C++ programmer friend about this, and he says the object ends up on the heap.

    For people who find this issue while searching, it works when I configure jemalloc with --with-lg-page=16, but I really shouldn't have to do that. Both Ubuntu and CentOS use a 4 KiB page size.

  6. eviljoel commented on Mar 18, 2022

    @eviljoel
    eviljoel
    on Mar 18, 2022
    Author

    I figured this out. Turns out the fix has nothing to do with the page size. The increased page size "worked" because it was large enough to accommodate the large EventMachine object.

    Apparently Ruby is linking to the wrong malloc at build time due to a Debian patch to 'ld'. This was actually fixed in Ruby 3.1.0 here: ruby/ruby@737e443 . Six days ago the fix was back ported to the Ruby 3.0 branch. (No fixed 3.0 release yet.) I've requested a back port to Ruby 2.7 here: https://bugs.ruby-lang.org/issues/18650

    For people who are encountering the same issue, you can work around the issue by appending LDFLAGS="-Wl,--no-as-needed" to the end of your configure script like so:

    ./configure --with-jemalloc --enable-shared --with-opt-dir=/usr/local LDFLAGS="-Wl,--no-as-needed"

    Note that --no-as-needed is not available on all platforms.

    Still leaving open in case you want to work around this upstream issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions