iframe-proxy

nevans · 2021-10-28T21:51:23Z

This isn't complete by itself, because EM hasn't properly configured an
X509_STORE to go with its SSL_CTX and SSL, nor does it verify the
peer's identity (verify_hostname in stdlib). Unlike stdlib, the
"store_context" is merely a read-only data structure, containing a copy
of the most useful data from the X509_STORE_CTX.

Also, the default implementation for EM::Connection#ssl_verify_peer
has been updated to log errors and returns preverify_ok unchanged.

nevans · 2021-10-28T21:54:51Z

sodabrew · 2021-10-28T22:35:17Z

Thank you!

nevans · 2021-10-29T14:12:26Z

FYI: I just force-pushed a tiny fix to tests. Replaced a . with a &. to keep a few of the other test_ssl_*.rb passing.

MSP-Greg · 2021-10-29T14:17:35Z

@nevans I think &. was introduced in Ruby 2.3, and CI (and the EM gem) have Ruby 2.2 as the minimum version? I.think...

nevans · 2021-10-29T20:14:20Z

@MSP-Greg yeah, you're right. I hadn't checked to see how far back EM is maintaining support! &. is just a little bit of sugar, not worth upgrading over. I'll push an update and run tests in different rubies locally.

Although the last time tried, I wasn't even able to compile some of the older versions, specifically because of OpenSSL library versions (IIRC). I see the gemspec is set to 2.0.0, maybe that should be updated to 2.2 to match CI?

And I just now looked into the gh action repo to see if I could just grab one of those images and put it into docker, and I saw this:

Note that Ruby ≤ 2.4 and the OpenSSL version it needs (1.0.2) are both end-of-life,
which means Ruby ≤ 2.4 is unmaintained and considered insecure.

-- ruby/setup-ruby README.md

IMO, that might be a good reason to drop support for older rubies! 😉

MSP-Greg · 2021-10-29T21:20:48Z

the gemspec is set to 2.0.0, maybe that should be updated to 2.2 to match CI?

The main people involved in setup-ruby are also involved with Ruby master, so there might be a bias there. The Actions code here was committed before Ruby 2.1 and 2.0 were available in Actions (2.1 was added Jul 16, 2020, 2.0 was later).

I'm not the best person to ask about 'old Ruby support', but if it currently works, I've always been inclined to leave it included.

I'll see if it works in my fork... I'm aware of the OpenSSL EOL issue, and I do wonder why people don't want to/can't update their apps. I suppose it does support TLS 1.2, sort of...

nevans · 2021-11-10T19:19:21Z

I'd like to make some other changes to the API, to give access to error (Integer), error_string, depth, maybe even error = code. In short: a wrapper for the X509_STORE_CTX. ;)

IMO, it's a real pain in the neck to debug issues if you don't have all of that. I'll update this PR hopefully some time later this week or weekend with my proposed API.

In the meantime, I have an bigger experimental branch that I'm interested in feedback on. I'll post a draft PR later today (I want to do a few rebase cleanups to make it easier to read first)

nevans · 2021-11-10T23:46:46Z

Okay, I'll update this PR to work with different versions of ruby, but please also see #957

nevans · 2021-11-13T04:40:21Z

It's passing in 2.4 - 3.0 now, but 2.0-2.3 don't pass in master (for me). I rebased it on top of #958 and #959.

nevans · 2021-11-13T18:51:41Z

FYI: that last commit updated the API to be closer to how I'd prefer it. What do you think? (still waiting for the various tests to run).

nevans · 2021-11-13T19:03:45Z

I haven't added tests for the new StoreContext object, yet. I'll update the existing ssl_verify tests to look at that too.

nevans · 2021-11-13T21:56:12Z

haha, I swear I'm not pushing until it passes for me locally. I suppose I shouldn't be surprised that the behavior here is slightly different, but I am a bit puzzled. I'm running OpenSSL 1.1.1l 24 Aug 2021 on Ubuntu 21.10, using ruby versions via rbenv install...

I think the behavior that's different is that, when you override errors, sometimes openssl calls the verify callback once per error and then a final time per certificate with no error, and other times it only calls once per error. These test failures are all displaying "once per error, w/o a non-error" behavior, but locally I get that final non-error (as shown in the tests).

If you know anywhere that it's documented why this happens, or some way to regularize the behavior, that'd be great. Otherwise, I'll just relax the tests to not care about that final non-error callback.

nevans · 2021-11-15T20:31:25Z

FYI: this is passing all jobs on my branch now: https://github.com/nevans/eventmachine/actions/runs/1457505841

sodabrew · 2024-09-05T17:21:42Z

+	return rb_str_new(NULL, (long)size);
+}
+
+// adapted from stdlib openssl's ossl_str_new


Would it be possible to use those method directly (if they are part of the public API?)

For the most part, yes, and I would love to just use the methods directly. Even better, I'd love to just use OpenSSL::X509::StoreContext directly, and skip all of the re-implementation.

The problem (as I understand it) is that there's no mechanism for a ruby gem to publish C header files that can safely be used (not even for bundled or default gems, like openssl). And even then, we'd also need the openssl gem to commit to a public API for any C funcs that we used... many of the functions I want access to are static and not exported. There's at least one b.r-l.org conversation about this issue: https://bugs.ruby-lang.org/issues/17760, but it didn't really come to any resolution, AFAICT.

That said, it should be possible to find the gem's install dir and use ext/openssl/ossl.h. And the most important functions that we might want to access are exported. I just don't have any idea how brittle that would be... and when it breaks, how hard to debug will it be?

What you've done makes sense given that these are internal methods that aren't being intentionally exported with a stable API. Someone put the idea in my head ages ago to move all of the TLS work up into Ruby space and implement it using the ruby/openssl standard library gem, and just rip out all of the C++ TLS bits. That's not something I've had any time in years to think much about, but would seem to be the right move.

Adding a few more links I found:

https://bugs.ruby-lang.org/issues/17760

https://bugs.ruby-lang.org/issues/17761 (Fixed by: ruby/ruby@5cdf99f6)

https://bugs.ruby-lang.org/issues/15156

Have a 'includedir' specifier in Gem::Specification. ruby/rubygems#2408

added include_dir attribute to Gem::Specificification ruby/rubygems#2416

Safe to link against gem extension? ruby/openssl#475
I created this issue on the openssl gem back when I first worked on this current eventmachine PR.

sodabrew · 2024-09-18T23:12:00Z

Let me know if you're still working on this PR? I think it looks good

sodabrew · 2024-09-18T23:34:54Z

    elsif opcode == SslVerify
      c = @conns[conn_binding] or raise ConnectionNotBound, "received SslVerify for unknown signature: #{conn_binding}"
-      result = c.ssl_verify_peer(data) or c.close_connection
+      result = (c.original_method(:ssl_verify_peer).arity == 1) ?


Suggest to add a comment here that arity == 1 is for backwards compatibility

nevans · 2024-09-19T22:16:50Z

Let me know if you're still working on this PR? I think it looks good

Thanks! Not quite yet. I should have it ready some time tomorrow.

nevans · 2024-09-20T16:34:27Z

For what it's worth, I did find quite a few examples of gems that link against other gem extensions:

The ruby-gnome project actually seems to have a lot of infrastructure based around this:
- mkmf-gnome has several methods which support linking against other gems
- See e.g: add_depend_package(gem_name, ...), which is used by 17 different gems in the ruby-gnome repo, and many other gems outside it
I found many gems linking against narray, its successor numo-narray, and nmatrix. For example:
Linked against the mysql2 gem:
- mysql2-replication
- mysql2-arrow
libxslt-ruby links to libxml-ruby
debase-ruby_core_source looks like it primarily exists so other gems can link against ruby's internal headers:
xnd-ruby gems:
- gumath links against xnd and ndtypes
- xnd links against ndtypes
nokogumbo linked against the nokogiri gem
(nokogumbo was merged into nokogiri several years ago, but it's still a useful example of the pattern)
rugged-redis links against the rugged and hiredis gems
(not updated in 7 years, but still a useful example of the pattern)
rugged-mysql links against the rugged gem
(not updated in 10 years, but still a useful example of the pattern)
ruby-libappindicator links against the glib2 and gtk2 gems
(not updated in 13 years, but still a useful example of the pattern)

But I have questions and concerns about stability. Is this a safe thing to do? Under what circumstances can they break? Some of these gems have millions of downloads, so the pattern does seem to have some resilience.

Is there a way for a gem to signal that all of the exported methods in a particular header file will be ABI backward compatible within a version range (besides just asking in their discussion forums or issue trackers or adding it to their README or other docs)?

nevans · 2024-09-20T16:57:29Z

For backward compatibility, this will continue sending only the current certificate when ssl_verify_peer takes only a single argument. Otherwise, the arguments will match stdlib's `verify_callback`. The certificate PEM is still easily obtained through `x509_store_ctx.current_cert.to_pem`. Also, the default implementation for `EM::Connection#ssl_verify_peer` has been updated to log errors and return `preverify_ok` unchanged. To support this, a new EM::SSL::X509::StoreContext class has been added, with similar semantics to stdlib's OpenSSL::SSL::X509::StoreContext. Unlike stdlib, this X509 store context is not a wrapper around X509_STORE_CTX. It simply contains copies of the most useful data from X509_STORE_CTX. The biggest difference from stdlib: * It is frozen, and has none of the attr writer methods. * It does not have `#chain` (array of X509::Certificate objects). * It cannot return the `#current_crl`. * It cannot perform the certificate verfication using `#verify`. Most of the new code was copied directly from the `openssl` gem, and modified only where needed, in order to work with eventmachine. The functions are placed in files named after their files of origin. This is still missing significant features, which will be fixed in subsequent PRs: * This still defaults to _not_ verify peers. * This still requires extra code to verify the peer's identity. * This still can't configure the X509_STORE.

nevans force-pushed the openssl-preverify_ok-passthrough branch from fbe9b7d to c6e63f6 Compare October 28, 2021 21:59

nevans force-pushed the openssl-preverify_ok-passthrough branch from c6e63f6 to 39e1a5d Compare October 29, 2021 14:10

nevans force-pushed the openssl-preverify_ok-passthrough branch from 39e1a5d to 77a311e Compare November 13, 2021 04:24

nevans changed the title ~~Adds preverify_ok to verify_peer~~ Adds preverify_ok, error int and string, and depth to verify_peer Nov 13, 2021

nevans changed the title ~~Adds preverify_ok, error int and string, and depth to verify_peer~~ Adds preverify_ok, error, and depth to verify_peer Nov 13, 2021

nevans force-pushed the openssl-preverify_ok-passthrough branch from 9afcc72 to 9979411 Compare November 13, 2021 22:34

nevans force-pushed the openssl-preverify_ok-passthrough branch from 9979411 to 5f5864f Compare November 22, 2021 23:13

nevans force-pushed the openssl-preverify_ok-passthrough branch 4 times, most recently from 210551c to ea180ae Compare August 28, 2024 22:19

nevans force-pushed the openssl-preverify_ok-passthrough branch 2 times, most recently from b5ef36c to bf6efcd Compare September 5, 2024 14:27

sodabrew reviewed Sep 5, 2024

View reviewed changes

nevans force-pushed the openssl-preverify_ok-passthrough branch 3 times, most recently from 4c7ccb4 to d9a2abe Compare September 6, 2024 14:18

nevans changed the title ~~Adds preverify_ok, error, and depth to verify_peer~~ Adds preverify_ok, and X509_STORE_CTX to verify_peer Sep 6, 2024

nevans changed the title ~~Adds preverify_ok, and X509_STORE_CTX to verify_peer~~ Adds preverify_ok and X509_STORE_CTX to verify_peer Sep 6, 2024

nevans changed the title ~~Adds preverify_ok and X509_STORE_CTX to verify_peer~~ Add preverify_ok and X509_STORE_CTX to verify_peer Sep 6, 2024

nevans force-pushed the openssl-preverify_ok-passthrough branch 3 times, most recently from e631c0e to 93384c8 Compare September 11, 2024 20:23

nevans force-pushed the openssl-preverify_ok-passthrough branch 2 times, most recently from 09a8624 to 282c39c Compare September 18, 2024 22:37

sodabrew reviewed Sep 18, 2024

View reviewed changes

nevans marked this pull request as draft September 19, 2024 22:17

nevans mentioned this pull request Sep 20, 2024

Safe to link against gem extension? ruby/openssl#475

Open

nevans force-pushed the openssl-preverify_ok-passthrough branch 2 times, most recently from ea43d2f to 75c0112 Compare February 18, 2025 19:05

nevans force-pushed the openssl-preverify_ok-passthrough branch from 75c0112 to 7fcb81f Compare February 18, 2025 19:16

Sunbelt Computer Software

PL/B Language Development and Support

Uh oh!

Conversation

nevans commented Oct 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nevans commented Oct 28, 2021

Uh oh!

sodabrew commented Oct 28, 2021

Uh oh!

nevans commented Oct 29, 2021

Uh oh!

MSP-Greg commented Oct 29, 2021

Uh oh!

nevans commented Oct 29, 2021

Uh oh!

MSP-Greg commented Oct 29, 2021

Uh oh!

nevans commented Nov 10, 2021

Uh oh!

nevans commented Nov 10, 2021

Uh oh!

nevans commented Nov 13, 2021

Uh oh!

nevans commented Nov 13, 2021

Uh oh!

nevans commented Nov 13, 2021

Uh oh!

nevans commented Nov 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nevans commented Nov 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sodabrew Sep 5, 2024

Choose a reason for hiding this comment

Uh oh!

nevans Sep 5, 2024

Choose a reason for hiding this comment

Uh oh!

sodabrew Sep 5, 2024

Choose a reason for hiding this comment

Uh oh!

nevans Sep 5, 2024

Choose a reason for hiding this comment

Uh oh!

sodabrew commented Sep 18, 2024

Uh oh!

sodabrew Sep 18, 2024

Choose a reason for hiding this comment

Uh oh!

nevans commented Sep 19, 2024

Uh oh!

nevans commented Sep 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nevans commented Sep 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nevans commented Oct 28, 2021 •

edited

Loading

nevans commented Nov 13, 2021 •

edited

Loading

nevans commented Nov 15, 2021 •

edited

Loading

nevans commented Sep 20, 2024 •

edited

Loading