iframe-proxy

mkleczek · 2026-03-10T14:21:34Z

DISCLAIMER:
This commit was authored entirely by a human without the assistance of LLMs.

Stacked on top of #4702 as it is not enough to start a new instance, it is also necessary not to fail in-flight requests on the old instance.

steve-chavez · 2026-03-10T16:27:13Z

mkleczek · 2026-03-10T16:29:32Z

This is failing all tests, I'd suggest to put these type of PRs as draft.

Hmm... worked before latest push. Will switch to draft and fix.

mkleczek · 2026-03-10T16:53:19Z

This is failing all tests, I'd suggest to put these type of PRs as draft.

Hmm... worked before latest push. Will switch to draft and fix.

@steve-chavez - it looks like the issue is that on my machine PostgREST startup is fast enough so that it loads the schema cache before it accepts any requests. Here in CI the new instance fails with 503 because it didn't yet load the schema cache.

The question: is there any particular reason why we return 503, instead of simply not start listening on the socket until schema cache is loaded? The way we have it right now means we can't really support zero-downtime upgrades because once we start the new instance but before it loads the schema cache, some clients will get 503.

steve-chavez · 2026-03-10T20:41:52Z

The question: is there any particular reason why we return 503, instead of simply not start listening on the socket until schema cache is loaded?

We were aiming to have requests wait instead of 503, this waiting does happen during schema cache reload but not on startup; we discussed this on #4129. Would it be better to not listen on the socket? How would clients behave in this case?

develop7 · 2026-03-10T21:36:38Z

They would get a "connection refused" error, which means nobody there and is imo more confusing that any 5xx error. UX-wise I would prefer some waiting to a presumably hard fail any day.

mkleczek · 2026-03-11T13:28:11Z

We were aiming to have requests wait instead of 503, this waiting does happen during schema cache reload but not on startup; we discussed this on #4129. Would it be better to not listen on the socket? How would clients behave in this case?

They would get a "connection refused" error, which means nobody there and is imo more confusing that any 5xx error. UX-wise I would prefer some waiting to a presumably hard fail any day.

I am not convinced, see below.

This is a complex topic so let's dig into it a little more. The startup sequence right now is:

Postgrest is not running.
Clients get connection refused
Postgrest starts listening on a socket
Clients get 503
Postgrest loaded schema cache
Normal traffic

The alternatives are:
a - blocking during schema cache loading

Postgrest is not running.
Clients get connection refused
Postgrest starts listening on a socket
Clients are blocked and potentially timeout getting some network error
Postgrest loaded schema cache
Normal traffic

b - listening on a socket only after schema cache loaded

Postgrest is not running.
Clients get connection refused
Postgrest loaded schema cache and starts listening
Normal traffic

So from the point of view of the clients (they don't know when Postgrest was started), we have 3 alternatives:

connection refused -> 503 -> normal
connection refused -> blocked/time out -> normal
connection refused -> normal

I am not sure what value clients get from the first two options comparing to the third one. Diagnostics and readiness checks should be done using admin server anyway.

In case of SO_REUSEPORT the situation is even worse if we start listening early. We have the following situation: instance 1 is running, instance 2 is started. From the point of view of clients:

Today: normal traffic -> some clients get 503 (both instances are listening only one is ready) -> normal traffic
Blocking: normal traffic -> some clients blocked/time out (both instances are listening one is blocking) -> normal traffic
Not listening: just normal traffic (there are no disruptions at all because all requests are handled by instance 1 until instance 2 is ready and starts listening).

So the first two options cause disruptions whereas the third one is fully zero-downtime and transparent to the clients.

My take on it would be:

Start admin server as early as possible.
Load schema cache.
Start listening on main socket.

This would require splitting binding from listening on the main socket (ie. we need to bind without listening first so that we can pass the socket to the admin server).

@steve-chavez @develop7 thoughts?

mkleczek · 2026-03-11T13:51:21Z

Dependent on resolving (or having a workaround to) yesodweb/wai#853

steve-chavez · 2026-03-11T19:04:24Z

So the first two options cause disruptions whereas the third one is fully zero-downtime and transparent to the clients.
My take on it would be:

Agree, sounds much better.

steve-chavez · 2026-03-19T17:47:45Z

Start admin server as early as possible.
Load schema cache.
Start listening on main socket.

@mkleczek Q: why is the above dependent on yesodweb/wai#853? It looks like it can be done without it?

mkleczek · 2026-03-20T14:52:29Z

Start admin server as early as possible.
Load schema cache.
Start listening on main socket.

@mkleczek Q: why is the above dependent on yesodweb/wai#853? It looks like it can be done without it?

@steve-chavez - changing startup sequence is independent and can be done now (will raise a PR, it should be easy thing to do).

But this PR and prerequisite #4702 needs warp to be fixed.

wolfgangwalther

Only looked at the second commit here.

wolfgangwalther · 2026-04-19T10:51:16Z

 - Add config `db-timezone-enabled` for optional querying of timezones by @taimoorzaeem in #4751
 - Log schema cache queries timings on `log-level=debug` by @steve-chavez in #4805
 - Shutdown should wait for in flight requests by @mkleczek in #4702
+- Use SO_REUSEPORT on platforms supporting it by @mkleczek in #4703 #4694


I started looking at this PR by looking at the changelog. As a user I wonder: "so what?". I have no idea what that means. The changelog should be user facing, not developer facing.

wolfgangwalther · 2026-04-19T10:52:55Z

-        assert (
+        assert any(
            "Failed to load the schema cache using db-schemas=public and db-extra-search-path=x"
-            in output[7]
+            in line
+            for line in output
        )


There seem to be some changes in here that make our io tests more robust to changes. These look uncontroversial, so if we split them separately, we can merge them much quicker.

wolfgangwalther · 2026-04-19T11:02:47Z

+  try @SomeException (bindPortGenEx reusePortOpts NS.Stream port hostPreference)
+  >>= either (const $ bindPortGenEx [] NS.Stream port hostPreference) pure
+  >>= listenSocket
+  where
+    reusePortOpts = [(NS.ReusePort, 1)]


Entirely optional nit: I find it easier to read when inlining reusePortOpts - will put the [ ... ] and [] right on top of each other, making it easy to see the "diff" there.

develop7 · 2026-04-20T13:31:49Z

-  pure (sock, adminSock)
+initServerSocket :: AppConfig -> IO NS.Socket
+initServerSocket AppConfig{..} = case configServerUnixSocket of
+  -- I'm not using `streaming-commons`' bindPath function here because it's not defined for Windows,


https://github.com/fpco/streaming-commons/blob/master/ChangeLog.md#0230 says it's in there since at least 0.2.3.0 (by fpco/streaming-commons#80); what lib (and Windows) version do you have in mind?

https://github.com/fpco/streaming-commons/blob/master/ChangeLog.md#0230 says it's in there since at least 0.2.3.0 (by fpco/streaming-commons#80); what lib (and Windows) version do you have in mind?

The comment is copied from the exiting original code AFAIR - there is no change in streaming-commons usage.

mkleczek requested review from steve-chavez and wolfgangwalther March 10, 2026 14:27

mkleczek force-pushed the so-reuseport branch 3 times, most recently from c252511 to 27c16d7 Compare March 10, 2026 15:43

mkleczek marked this pull request as draft March 10, 2026 16:31

mkleczek force-pushed the so-reuseport branch from 27c16d7 to 070d7a1 Compare March 10, 2026 20:00

mkleczek force-pushed the so-reuseport branch 2 times, most recently from 64bb6ca to 2cb6503 Compare March 11, 2026 18:49

mkleczek force-pushed the so-reuseport branch from 2cb6503 to 83a7a91 Compare March 11, 2026 19:32

mkleczek mentioned this pull request Mar 11, 2026

refactor: provide AppState.waitForSchemaCacheLoaded function #4709

Merged

mkleczek force-pushed the so-reuseport branch 7 times, most recently from 7d7375a to bd4c7ec Compare March 12, 2026 10:20

mkleczek mentioned this pull request Mar 12, 2026

refactor: move socket creation and management to App module #4713

Merged

mkleczek force-pushed the so-reuseport branch from bd4c7ec to 2092d2a Compare March 12, 2026 15:13

mkleczek mentioned this pull request Mar 12, 2026

test(io): add test_so_reuseport_zero_downtime_handover #4715

Merged

mkleczek force-pushed the so-reuseport branch from 2092d2a to 52cb192 Compare March 12, 2026 15:20

mkleczek force-pushed the so-reuseport branch 5 times, most recently from acdac3c to babc120 Compare March 12, 2026 22:18

mkleczek force-pushed the so-reuseport branch 5 times, most recently from 56a3d2f to 0212dc5 Compare April 7, 2026 05:43

mkleczek force-pushed the so-reuseport branch 3 times, most recently from de2835b to 101e938 Compare April 15, 2026 08:13

mkleczek force-pushed the so-reuseport branch 3 times, most recently from d185fcd to 7fc4acb Compare April 18, 2026 05:31

wolfgangwalther reviewed Apr 19, 2026

View reviewed changes

mkleczek force-pushed the so-reuseport branch 2 times, most recently from 94919bc to 9b5cef7 Compare April 20, 2026 12:52

develop7 reviewed Apr 20, 2026

View reviewed changes

mkleczek force-pushed the so-reuseport branch 6 times, most recently from 341f183 to 00d3e4e Compare April 21, 2026 04:54

add: use SO_REUSEPORT on platform supporting it

a00329c

mkleczek force-pushed the so-reuseport branch from 00d3e4e to a00329c Compare April 21, 2026 12:21

Sunbelt Computer Software

PL/B Language Development and Support

Uh oh!

Conversation

mkleczek commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steve-chavez commented Mar 10, 2026

Uh oh!

mkleczek commented Mar 10, 2026

Uh oh!

mkleczek commented Mar 10, 2026

Uh oh!

steve-chavez commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

develop7 commented Mar 10, 2026

Uh oh!

mkleczek commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mkleczek commented Mar 11, 2026

Uh oh!

steve-chavez commented Mar 11, 2026

Uh oh!

steve-chavez commented Mar 19, 2026

Uh oh!

mkleczek commented Mar 20, 2026

Uh oh!

wolfgangwalther left a comment

Choose a reason for hiding this comment

Uh oh!

wolfgangwalther Apr 19, 2026

Choose a reason for hiding this comment

Uh oh!

wolfgangwalther Apr 19, 2026

Choose a reason for hiding this comment

Uh oh!

wolfgangwalther Apr 19, 2026

Choose a reason for hiding this comment

Uh oh!

develop7 Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

mkleczek Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

mkleczek commented Mar 10, 2026 •

edited

Loading

steve-chavez commented Mar 10, 2026 •

edited

Loading

mkleczek commented Mar 11, 2026 •

edited

Loading