Stacked on #13356, which touches the same block in finish_response(). Review that one first; this branch contains its two commits. Happy to rebase onto master if #13356 lands or is dropped.

What do these changes do?

finish_response() replays the bytes buffered behind an upgrade request the handler declined (#12796). It feeds them to the parser without the two things data_received() does around the same call, and both are reachable from the network.

A parse error escapes. data_received() wraps feed_data() and turns HttpProcessingError into a queued 400. finish_response() does not, and it replays before resp.prepare() runs — so a malformed request pipelined behind the declined upgrade discards the response already built for the upgrade, force-closes the connection, and logs a routine client error as an unhandled server exception:

GET /upgrade  (Upgrade: websocket, handler declines)
POST /bad     (Transfer-Encoding: chunked + Content-Length: 5)

before:  response bytes b''            connection closed, "Unhandled exception" logged
after:   HTTP/1.1 200 ... then 400     both answered, connection intact

The identical malformed request on the ordinary path already gets a clean 400.

The upgraded flag is discarded. When the replayed bytes contain a second upgrade, the parser switches to upgraded while self._upgraded stays False. The next data_received() therefore takes the parser branch and assigns _message_tail instead of appending to it, dropping whatever was buffered there:

The C parser's corruption is worth spelling out, because it is the sharper symptom. nb is declared at _http_parser.pyx:631 and assigned only inside the HPE_PAUSED_UPGRADE branch at :675, but it is read at :711:

if self._upgraded:
    return messages, True, data[nb:]

Entering feed_data() already upgraded, without llhttp returning HPE_PAUSED_UPGRADE on that call, therefore slices data with an uninitialised size_t. In the reproducer it lands on 1, and the next request line arrives as b"ET /later HTTP/1.1" — the leading G eaten. Keeping the flag stops the protocol from ever handing the parser that state, so this is a hardening of an invariant the C parser already assumes rather than only a bookkeeping fix.

Are there changes in behavior for the user?

A malformed request pipelined behind a declined upgrade is answered with 400 instead of killing the connection and losing the preceding response. Requests buffered behind a second declined upgrade are dispatched instead of dropped. No API change.

Is it a substantial burden for the maintainers to support this?

No. It makes the replay path mirror data_received(), which is where the same three concerns are already handled.

Related issue number

None. Both defects predate #13356 — verified by running each reproducer against 288cf469b and against this branch's parent and getting identical results — and both are on the path added in #12796.

Checklist

• I think the code is well written
• Unit tests for the changes exist
• Documentation reflects the changes — N/A, no user-facing API change
• If you provide code modification, please add yourself to CONTRIBUTORS.txt — already listed
• Add a new news fragment into the CHANGES/ folder

$ pytest tests/test_web_websocket_functional.py tests/test_web_protocol.py \
         tests/test_web_functional.py tests/test_http_parser.py tests/test_web_server.py
1072 passed, 6 deselected, 3 xfailed        # C extensions
656 passed, 35 skipped, 4 deselected        # AIOHTTP_NO_EXTENSIONS=1

FAILED tests/test_web_websocket_functional.py::test_bad_pipelined_request_after_failed_websocket_upgrade
FAILED tests/test_web_websocket_functional.py::test_second_upgrade_pipelined_after_failed_websocket_upgrade