Add limits for client cookie handling by bdraco · Pull Request #13930 · aio-libs/aiohttp · GitHub
Skip to content

Add limits for client cookie handling - #13930

Merged
bdraco merged 60 commits into
masterfrom
cookie_limits
Oct 4, 2026
Merged

bdraco merged 60 commits into
masterfrom
cookie_limits

Conversation

@bdraco

@bdraco bdraco commented Oct 3, 2026 •

Copy link
Copy Markdown
Member

What do these changes do?

Add limits to client cookie handling. Each Set-Cookie field now produces exactly one cookie, as browsers do, and CookieJar caps how many cookies it stores and how large a Cookie header it builds. Eviction follows Firefox, so expired and non secure cookies go first, least recently used first.

Are there changes in behavior for the user?

The jar applies these limits:

  • 50 Set-Cookie fields per response, 32 attributes per field;
  • 4,096 octets for a cookie's name and value, 1,024 octets per attribute value;
  • 180 cookies per domain and 3,300 in total;
  • 150 cookies and 8,190 octets per Cookie request header.

Extra name=value pairs after the first one in a Set-Cookie field are now ignored instead of becoming separate cookies; see the breaking fragment. Cookies passed with cookies= and an explicit Cookie header are not limited.

Is it a substantial burden for the maintainers to support this?

No, the limits are private constants and there is no new public API.

Related issue number

N/A

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt
    • The format is <Name> <Surname>.
    • Please keep alphabetical order, the file is sorted by names.
  • Add a new news fragment into the CHANGES/ folder
    • name it <issue_or_pr_num>.<type>.rst (e.g. 588.bugfix.rst)

    • if you don't have an issue number, change it to the pull request
      number after creating the PR

      • .bugfix: A bug fix for something the maintainers deemed an
        improper undesired behavior that got corrected to match
        pre-agreed expectations.
      • .feature: A new behavior, public APIs. That sort of stuff.
      • .deprecation: A declaration of future API removals and breaking
        changes in behavior.
      • .breaking: When something public is removed in a breaking way.
        Could be deprecated in an earlier release.
      • .doc: Notable updates to the documentation structure or build
        process.
      • .packaging: Notes for downstreams about unobvious side effects
        and tooling. Changes in the test invocation considerations and
        runtime assumptions.
      • .contrib: Stuff that affects the contributor experience. e.g.
        Running tests, building the docs, setting up the development
        environment.
      • .misc: Changes that are hard to assign to any of the above
        categories.
    • Make sure to use full sentences with correct case and punctuation,
      for example:

      Fixed issue with non-ascii contents in doctest text files
      -- by :user:`contributor-gh-handle`.

      Use the past tense or the present tense a non-imperative mood,
      referring to what's changed compared to the last released version
      of this project.

Local test run
pytest (full suite, xdist): 5833 passed, 25 skipped, 17 xfailed
cookie and client tests, C extensions: 863 passed, 4 skipped, 1 xfailed
cookie and client tests, AIOHTTP_NO_EXTENSIONS=1: 863 passed, 4 skipped, 1 xfailed

Drafted with Claude Opus 5.5; reviewed by @bdraco.

@agentscanapp

agentscanapp Bot commented Oct 3, 2026

Copy link
Copy Markdown

@pyba-chronographer pyba-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label Oct 3, 2026
@bdraco bdraco added backport-3.14 Trigger automatic backporting to the 3.14 release branch by Patchback robot backport-3.15 Trigger automatic backporting to the 3.15 release branch by Patchback robot and removed bot:chronographer:provided There is a change note present in this PR agentscan:mixed-signals labels Oct 3, 2026
@pyba-chronographer pyba-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label Oct 3, 2026
@codecov

codecov Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.09%. Comparing base (470daa7) to head (e41e211).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #13930      +/-   ##
==========================================
+ Coverage   99.06%   99.09%   +0.03%     
==========================================
  Files         135      135              
  Lines       52582    53491     +909     
  Branches     2739     2806      +67     
==========================================
+ Hits        52092    53009     +917     
+ Misses        369      363       -6     
+ Partials      121      119       -2     
Flag Coverage Δ
Autobahn 21.69% <14.35%> (-0.12%) ⬇️
CI-GHA 98.93% <100.00%> (+0.03%) ⬆️
OS-Linux 98.72% <100.00%> (+0.03%) ⬆️
OS-Windows 97.38% <100.00%> (+0.06%) ⬆️
OS-macOS 98.25% <100.00%> (+0.04%) ⬆️
Py-3.10 98.16% <100.00%> (+0.04%) ⬆️
Py-3.11 98.40% <100.00%> (+0.04%) ⬆️
Py-3.12 98.48% <100.00%> (+0.03%) ⬆️
Py-3.13 98.48% <100.00%> (+0.04%) ⬆️
Py-3.14 98.50% <99.81%> (+0.02%) ⬆️
Py-3.15 98.50% <99.81%> (+0.02%) ⬆️
Py-3.15t 97.91% <99.81%> (+0.04%) ⬆️
Py-pypy-3.12 96.57% <100.00%> (+0.07%) ⬆️
VM-macos 98.25% <100.00%> (+0.04%) ⬆️
VM-ubuntu 98.72% <100.00%> (+0.03%) ⬆️
VM-windows 97.38% <100.00%> (+0.06%) ⬆️
cython-coverage 83.59% <71.90%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Comment thread tests/test_cookiejar.py Fixed
@aiolibsbot

aiolibsbot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Previous review — superseded by a newer review below.

@aiolibsbot

Copy link
Copy Markdown
Contributor

@bdraco: > 1. load() no longer calls a subclass's update_cookies() and copies a plain CookieJar's state

Agreed. Finding dropped. Subclass load() path stays as is.

@aiolibsbot

aiolibsbot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Previous review — superseded by a newer review below.

@bdraco

bdraco commented Oct 3, 2026

Copy link
Copy Markdown
Member Author
  1. Flags past the 32-attribute cap are dropped without a log
    aiohttp/_cookie_helpers.py:298

these are unrealistic and browsers drop it without a log, decline

@bdraco

bdraco commented Oct 3, 2026

Copy link
Copy Markdown
Member Author
  1. _can_send accepts cookies too large to ever be sent
    aiohttp/cookiejar.py:733-738

decline, another check for a crazy edge case doesn't justify the performance hit just to log garbage

@aiolibsbot

Copy link
Copy Markdown
Contributor

@bdraco: > 1. Flags past the 32-attribute cap are dropped without a log

Accepted. Attribute-cap logging dropped. Matches browser behaviour.

@aiolibsbot

Copy link
Copy Markdown
Contributor

@bdraco: > 2. _can_send accepts cookies too large to ever be sent

Accepted. No size check in _can_send. Hot path stays lean.

@aiolibsbot

aiolibsbot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Previous review — superseded by a newer review below.

@bdraco
bdraco marked this pull request as ready for review October 3, 2026 19:40
@bdraco

bdraco commented Oct 3, 2026 •

Copy link
Copy Markdown
Member Author

performance is fixed now. I've cleaned it up as much as I think I can.

want to sleep on it and look at it with fresh eyes though

Its just bigger than I would like, but a limit, validation, and eviction system for cookies was already going to be big

@bdraco

bdraco commented Oct 4, 2026 •

Copy link
Copy Markdown
Member Author

A structurally better design would be keep containers in sync by storing things that cannot be allowed out of sync in the same container but its probably way to big to put in this PR and than we have to go get the performance right on that again, and I couldn't come up with a design that didn't make this PR 1000 lines longer and was performant. Too much mitigate risk, so going to put that on the shelf as its a larger refactor down the road when we want it

@bdraco

bdraco commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

I'm just going to add consistency tests instead as thats the right call to make sure the containers are never out of sync without massively expanding scope

@aiolibsbot

Copy link
Copy Markdown
Contributor

@bdraco: I'm just going to add consistency tests instead as thats the right call to make sure the containers ...

Consistency test landed in bcde497. Covers update, filter, clear, save/load. Mutation through jar.cookies[...] stays uncovered. Tracked as deferred finding.

@aiolibsbot

aiolibsbot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

PR Review — Add limits for client cookie handling

Merge-ready. The only new commit, bcde497, adds a randomized test that checks the jar's bookkeeping, as @bdraco planned. Two non-blocking suggestions remain.

What's good:

  • The new consistency test is strong. _assert_consistent checks every piece of bookkeeping against the cookies actually stored: _cookie_count, _domain_counts, _access_generations, host-only keys, _expirations and the morsel cache.
  • It runs 3,000 random operations with small limits. These cover header updates, update_cookies, filter_cookies, clear with a predicate, clear_domain, save/load and full clear. Because the limits are small, eviction, purging and the Cookie header limits are exercised constantly, and a fixed seed keeps the run reproducible.
  • The design is unchanged and still holds up: one cookie per Set-Cookie field, Firefox-style eviction with O(1) counters, the most specific cookies kept first when the header is too big, an atomic load(), and the threat-model section §5.16.

Findings and earlier dispositions:

  • Inner SimpleCookie buckets are still mutable, so the counters can drift. Downgraded to [Deferred] per @bdraco, who plans a later refactor and added the consistency tests instead.
  • load() stages into a base CookieJar, so subclass overrides are skipped. Low-impact suggestion.
  • Earlier logging requests (warning-level logs for evictions on load and for cookies left out of the header) are not raised again. This is in line with @bdraco's earlier decision to decline extra logging for edge cases, and the debug logs already exist.
  • The CodeQL alerts on tests/test_benchmarks_cookiejar.py:10 (unused import) and on jar in test_update_cookies_rejects_cookie_that_cannot_be_sent (possibly uninitialized) are still false positives.


Checklist

  • No hardcoded secrets
  • Input validation at boundaries (Set-Cookie parsing, update_cookies, load())
  • Unbounded collections bounded (jar, morsel cache, expiry heap)
  • Internal bookkeeping consistent with stored contents
  • Subclass/extension behaviour preserved
  • Docs, changelog and threat model reflect behaviour changes
  • New branches covered by tests
ℹ️ Triage summary

2 pre-existing finding(s) on unchanged code suppressed (freeze).


Silent Failure Analysis

🟡 **1. MEDIUM** — trusted caller input silently dropped
aiohttp/cookiejar.py:560-566

Risk: The public update_cookies() API takes trusted application input, so a cookie value with a control character or an unencodable surrogate used to fail loudly at the writer, but now it vanishes with only a debug-level log that production rarely shows, and the caller can't tell the cookie was never stored.

if check_sendable and self._limits_enabled and not self._can_send(cookie):
    internal_logger.debug(
        "Dropped cookie %r for %s: it can't be sent in a Cookie header",
        name,
        domain,
    )
    continue

Fix: For caller-supplied cookies (copy_morsels=True), raise ValueError/CookieError or at least log at warning level, and keep the quiet drop only for untrusted Set-Cookie input.

🟡 **2. MEDIUM** — partial load silently accepted
aiohttp/cookiejar.py:226-283

Risk: The load() docstring says a malformed file raises and leaves the jar unchanged, but only non-finite expiries and type errors raise; records that are unencodable, fail the domain check, or are dropped by eviction or the full-of-Secure rule disappear silently, so a corrupted or tampered jar file can come back as a partial jar with no signal.

loaded._update_cookies(
    {name: morsel},
    response_url,
    copy_morsels=False,
    expiration=expiration,
)

Fix: Count the records _update_cookies rejects during load() and either raise or log a warning with that count, and make the docstring say which malformations raise and which are skipped.

🟡 **3. MEDIUM** — subclass override silently bypassed
aiohttp/cookiejar.py:229-233

Risk: load() used to route records through self.update_cookies(), so a subclass's own filtering or validation ran on loaded cookies; now it fills a plain CookieJar and copies its __dict__ across, which silently skips any overridden update_cookies/_update_cookies hooks.

loaded = CookieJar(
    unsafe=self._unsafe,
    quote_cookie=self._quote_cookie,
    treat_as_secure_origin=self._treat_as_secure_origin,
)

Fix: Build the staging jar with type(self) (or call self.update_cookies into the staging state) so subclass hooks still apply, or document the behaviour change in the breaking changelog.

🟡 **4. MEDIUM** — silent truncation drops security attribute
aiohttp/_cookie_helpers.py:297-300

Risk: Once 32 attribute segments have been parsed, the rest of the field is dropped with no log; a Secure (or Domain/Path) that comes after them is lost, so the stored cookie can later go out over plain HTTP and nothing shows that parsing stopped.

for _ in range(_MAX_COOKIE_ATTRIBUTES):
    if index >= end:
        break
    if (attribute_match := _COOKIE_PATTERN.match(attributes, index)) is None:

Fix: Emit a debug log (counted in the existing per-response summary) when the attribute limit cuts parsing short; optionally keep scanning, without storing anything, just for the Secure flag so the cap can't widen where the cookie is sent.


Automated review by Kōan (Claude) HEAD=bcde497 1 min 31s

Comment thread THREAT_MODEL.md
Comment thread tests/test_cookiejar.py Outdated
@bdraco

bdraco commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

All good on the second Home Assistant instance

@bdraco

bdraco commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

Greptile's complaint about threat model being removed is not an issue as it was removed per maintainer request and will be added in a followup pr

@bdraco
bdraco merged commit b1f6c87 into master Oct 4, 2026
57 checks passed
@bdraco
bdraco deleted the cookie_limits branch October 4, 2026 19:24
@patchback

patchback Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Backport to 3.15: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply b1f6c87 on top of patchback/backports/3.15/b1f6c87b2bd3bd80f2b6c22ff46e11d00921e412/pr-13930

Backporting merged PR #13930 into master

  1. Ensure you have a local repo clone of your fork. Unless you cloned it
    from the upstream, this would be your origin remote.
  2. Make sure you have an upstream repo added as a remote too. In these
    instructions you'll refer to it by the name upstream. If you don't
    have it, here's how you can add it:
    git remote add upstream https://github.com/aio-libs/aiohttp.git
    
  3. Ensure you have the latest copy of upstream and prepare a branch
    that will hold the backported code:
    git fetch upstream
    git checkout -b patchback/backports/3.15/b1f6c87b2bd3bd80f2b6c22ff46e11d00921e412/pr-13930 upstream/3.15
    
  4. Now, cherry-pick PR Add limits for client cookie handling #13930 contents into that branch:
    git cherry-pick -x b1f6c87b2bd3bd80f2b6c22ff46e11d00921e412
    
    If it'll yell at you with something like fatal: Commit b1f6c87b2bd3bd80f2b6c22ff46e11d00921e412 is a merge but no -m option was given., add -m 1 as follows instead:
    git cherry-pick -m1 -x b1f6c87b2bd3bd80f2b6c22ff46e11d00921e412
    
  5. At this point, you'll encounter some merge conflicts. You must
    resolve them in order to preserve the patch from PR Add limits for client cookie handling #13930 as close to the
    original as possible.
  6. Once conflicts are resolved and git added, run:
    git cherry-pick --continue
    
  7. Push this branch to your fork on GitHub:
    git push origin patchback/backports/3.15/b1f6c87b2bd3bd80f2b6c22ff46e11d00921e412/pr-13930
    
  8. Create a PR, ensure that the CI is green. If it's not — update it so that
    the tests and any other checks pass. This is it!
    Now relax and wait for the maintainers to process your pull request
    when they have some cycles to do reviews. Don't worry — they'll tell you if
    any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

@patchback

patchback Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

bdraco added a commit that referenced this pull request Oct 4, 2026
…ling (#13940)

Co-authored-by: iamibi <8592115+iamibi@users.noreply.github.com>
Fixes
#123
bdraco added a commit that referenced this pull request Oct 4, 2026
…ling (#13942)

Co-authored-by: iamibi <8592115+iamibi@users.noreply.github.com>
Fixes
#123
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-3.14 Trigger automatic backporting to the 3.14 release branch by Patchback robot backport-3.15 Trigger automatic backporting to the 3.15 release branch by Patchback robot bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants