Cover both HTTP parsers in tests that only exercised one by LuShadowX · Pull Request #13271 · aio-libs/aiohttp · GitHub
Skip to content

Cover both HTTP parsers in tests that only exercised one - #13271

Closed
LuShadowX wants to merge 3 commits into
aio-libs:masterfrom
LuShadowX:parser-parity-tests
Closed

LuShadowX wants to merge 3 commits into
aio-libs:masterfrom
LuShadowX:parser-parity-tests

Conversation

@LuShadowX

@LuShadowX LuShadowX commented Jul 27, 2026 •

Copy link
Copy Markdown
Contributor

What do these changes do?

Follow-up to the suggestion in #13246: some parser tests only ever fed input to one implementation, so nobody knew whether the other one agreed.

I checked the three request-parser tests that construct a parser class directly and ran their input through both implementations:

input Python parser C parser
control char in a header value (Set-Cookie: abc\x01def) InvalidHeader BadHttpMessage
bare LF as a header terminator (Host: a\nX-Evil: b) InvalidHeader BadHttpMessage
header line that isn't valid UTF-8 (\xff) InvalidHeader BadHttpMessage
lowercase method (get /test HTTP/1.1) normalised to GET BadHttpMethod

The first three agree — both reject, only the error detail differs (InvalidHeader is a BadHttpMessage), so they are now parametrised over both parsers. The existing C-only tests keep their error-snippet assertions, which really are C-specific.

The method case genuinely differs, and only the Python side was covered, so the C side now has an explicit test next to it. That difference is intentional (llhttp rejects lowercase upstream) but it was invisible in the test suite.

No production code is touched.

Are there changes in behavior for the user?

No, tests only.

Is it a substantial burden for the maintainers to support this?

No. It removes single-implementation blind spots rather than adding anything to maintain, and it makes any future divergence between the two parsers fail a test instead of going unnoticed.

Related issue number

Refs #13246.

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes — N/A, tests only
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt — already listed
  • Add a new news fragment into the CHANGES/ folder

Drafted with Claude Code; reviewed by @LuShadowX.

Three of the parser tests fed input to a single implementation, so
whether the other one behaved the same was untested. Both parsers reject
a control character in a header value, a bare LF used as a header
terminator, and a header line that isn't valid UTF-8, only with
different error detail. Add the missing lowercase-method case for the C
parser too, which rejects what the Python parser normalises.
@pyba-chronographer pyba-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label Jul 27, 2026
@LuShadowX

LuShadowX commented Jul 27, 2026 •

Copy link
Copy Markdown
Contributor Author
Test run (Cython extensions installed, C parser available)
$ PYTHONPATH='.' pytest tests/test_http_parser.py -q -p no:randomly
844 passed, 6 deselected, 3 xfailed in 7.36s      # 837 passed before this branch

$ PYTHONPATH='.' AIOHTTP_NO_EXTENSIONS=1 pytest tests/test_http_parser.py -q -p no:randomly
430 passed, 35 skipped, 4 deselected in 4.29s     # the new C-only test skips

The three remaining xfails are pre-existing (test_parse_unusual_request_line,
test_http_request_parser_utf8_request_line, test_parse_uri_utf8, all
[c-parser]). I looked at them and they are real behaviour differences over
request-line and URI handling, not stale skips, so I left them alone rather than
change parser behaviour in a test-only PR.

@codecov

codecov Bot commented Jul 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.98%. Comparing base (4ae5d49) to head (184eac6).
⚠️ Report is 14 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #13271   +/-   ##
=======================================
  Coverage   98.98%   98.98%           
=======================================
  Files         131      132    +1     
  Lines       48993    49020   +27     
  Branches     2550     2552    +2     
=======================================
+ Hits        48496    48523   +27     
  Misses        373      373           
  Partials      124      124           
Flag Coverage Δ
Autobahn 22.13% <30.00%> (+<0.01%) ⬆️
CI-GHA 98.90% <100.00%> (+<0.01%) ⬆️
OS-Linux 98.67% <100.00%> (+<0.01%) ⬆️
OS-Windows 97.03% <100.00%> (+<0.01%) ⬆️
OS-macOS 97.92% <100.00%> (-0.02%) ⬇️
Py-3.10 98.11% <100.00%> (-0.01%) ⬇️
Py-3.11 98.37% <100.00%> (+<0.01%) ⬆️
Py-3.12 98.46% <100.00%> (+<0.01%) ⬆️
Py-3.13 98.44% <100.00%> (+<0.01%) ⬆️
Py-3.14 98.45% <100.00%> (-0.01%) ⬇️
Py-3.14t 97.55% <100.00%> (+<0.01%) ⬆️
Py-pypy-3.11 97.38% <60.00%> (-0.01%) ⬇️
VM-macos 97.92% <100.00%> (-0.02%) ⬇️
VM-ubuntu 98.67% <100.00%> (+<0.01%) ⬆️
VM-windows 97.03% <100.00%> (+<0.01%) ⬆️
cython-coverage 37.97% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@codspeed

codspeed Bot commented Jul 27, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 84 untouched benchmarks
⏩ 83 skipped benchmarks1


Comparing LuShadowX:parser-parity-tests (184eac6) with master (b2b7b50)

Open in CodSpeed

Footnotes

  1. 83 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@LuShadowX
LuShadowX marked this pull request as ready for review July 27, 2026 16:52
@LuShadowX

Copy link
Copy Markdown
Contributor Author

@Dreamsorcerer this is the parity pass you suggested on #13246 — the three inputs both parsers reject are now parametrised over both, and the lowercase-method case that only they differ on has a C-side test next to the Python one. Tests only, no parser changes.

@Dreamsorcerer

Copy link
Copy Markdown
Member

I'm not sure why new tests are being added, rather than fixing existing ones.

Rework per review: rather than adding parametrised tests next to the
narrow ones, convert the narrow ones.

test_unpaired_surrogate_in_header_py and
test_py_parser_normalises_method_to_uppercase both hand-rolled a single
implementation. They now take the parser fixture and run against both,
keeping the implementation-specific expectations: the Python parser
raises InvalidHeader and canonicalises a lowercase method, llhttp raises
BadHttpMessage and BadHttpMethod respectively.

Distinguishing the two needs a module check rather than isinstance(),
because HttpRequestParserPy aliases the same class the annotations name,
which makes the negative branch statically unreachable.

The two genuinely new cases are dropped: control characters in a header
value are already covered for both parsers by
test_ctl_host_header_bad_characters, and the bare-LF case belongs with
the divergence noted on the pull request rather than in this change.
@LuShadowX

LuShadowX commented Jul 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Fair point — reworked in 184eac6 so the PR now only widens existing tests and adds none.

Converted:

  • test_unpaired_surrogate_in_header_py → test_unpaired_surrogate_in_header, on the parser fixture. Expectations stay implementation-specific: InvalidHeader for the Python parser, BadHttpMessage for llhttp.
  • test_py_parser_normalises_method_to_uppercase → test_lowercase_method, likewise. The Python parser canonicalises the token, llhttp raises BadHttpMethod.

Dropped both of the ones that were genuinely new rather than parity work: control characters in a header value are already covered for both parsers by test_ctl_host_header_bad_characters, and the bare-LF case is the divergence below.

One thing worth a decision separately. test_invalid_linebreak is C-only, and it has to stay that way, because the Python parser does not reject its input:

py  | GET /world HTTP/1.1\r\nHost: 127.0.0.1\n\r\n  | no exception, no messages
c   | GET /world HTTP/1.1\r\nHost: 127.0.0.1\n\r\n  | BadHttpMessage

A bare LF terminating a header line leaves the Python parser waiting for more data instead of failing. Move the LF into the middle of the header block — Host: a\nX-Evil: b\r\n\r\n — and it does raise InvalidHeader, so it is specifically the header-terminating position that slips through. RFC 9112 §2.2 wants a bare LF rejected there, and it is the kind of gap that matters for request smuggling, so I would rather raise it as its own issue than bury it in a test-parity PR. Want me to open one?

Verified locally with the C extension built: 839 passed, 3 xfailed on tests/test_http_parser.py, and 427 passed, 34 skipped under AIOHTTP_NO_EXTENSIONS=1. mypy and black clean on the file — the module check in _is_py_parser is there because HttpRequestParserPy aliases the class the annotations name, so isinstance() makes the negative branch statically unreachable.

@LuShadowX

Copy link
Copy Markdown
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants