Cap concurrent Streamable HTTP sessions and expose the session limits… · modelcontextprotocol/python-sdk@c60175b · GitHub
Skip to content

Commit c60175b

Browse files
maxisbeymaxisbey
committed
Cap concurrent Streamable HTTP sessions and expose the session limits on the server factories
Add `max_sessions` (DEFAULT_MAX_SESSIONS = 10_000, `None` for no limit) to StreamableHTTPSessionManager: while that many stateful sessions are open, a request that would open another is answered 503 with a JSON-RPC error body and nothing is allocated; existing sessions are untouched and room frees up as they end or expire. This matches the Ruby SDK's defaults (the C# SDK uses the same 10 000 figure). `session_idle_timeout` and `max_sessions` are accepted by `Server.streamable_http_app()`, `MCPServer.streamable_http_app()`, `run_streamable_http_async()` and `run(transport="streamable-http")`, the same way `max_request_body_size` is, so applications can tune or disable them without reaching into `session_manager` after the fact. Docs: run/index.md options list, run/legacy-clients.md session cost, migration.md, troubleshooting.md.
1 parent 1e7fb45 commit c60175b

10 files changed

Lines changed: 160 additions & 27 deletions

File tree

‎docs/migration.md‎

Lines changed: 24 additions & 4 deletions

‎docs/run/index.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,12 @@ Each transport has its own keyword arguments, all on `run()`:
7070
* `max_request_body_size`: largest accepted request body in bytes. Defaults to 4 MiB; larger requests
7171
receive HTTP 413 before parsing or session creation. Raise it only when legitimate MCP messages
7272
exceed that size.
73+
* `session_idle_timeout`: how long, in seconds, a [legacy](legacy-clients.md) (session-based)
74+
client's session may sit with no request in flight before the server closes it. Defaults to 1800
75+
(30 minutes); `None` keeps sessions until the client deletes them. A client with an open `GET`
76+
stream or a request still being answered is never idle.
77+
* `max_sessions`: how many such sessions one app holds at once. Defaults to 10 000; while that many
78+
are open, a request that would open another gets HTTP 503. `None` removes the limit.
7379
* `event_store`, `retry_interval`, `transport_security`: resumability and DNS-rebinding protection. They can wait, until you deploy somewhere other than localhost; **[Deploy & scale](deploy.md)** covers `transport_security`.
7480

7581
!!! warning

‎docs/run/legacy-clients.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,13 @@ On one worker that is invisible. On two, it is the whole problem: a request that
5656
events to a client reconnecting to the *same* session), not a session store. It never makes a
5757
session reachable from another process.
5858

59+
The record is not kept forever. A client that ends its session (`DELETE`) frees it at once;
60+
a session that has had no request in flight for `session_idle_timeout` seconds (default 1800; an
61+
open `GET` stream or a request being answered counts as in flight) is closed, and its next request
62+
gets the same `404` a stray ID gets, so the client has to `initialize` again. Each worker process
63+
holds at most `max_sessions` of them (default 10 000) and answers `503` to a request that would
64+
open one more. Both are `run()` / `streamable_http_app()` options.
65+
5966
## The one knob: `stateless_http`
6067

6168
If stickiness is a cost you refuse to pay, there is exactly one thing you can change.

‎docs/troubleshooting.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -246,7 +246,7 @@ app = Starlette(routes=[Mount("/", app=mcp.streamable_http_app())], lifespan=lif
246246

247247
## `MCPError: Session not found`
248248

249-
The server does not recognise the `Mcp-Session-Id` your client sent, almost always because the server **restarted** (or you were routed to a different instance). Sessions live in that one process's memory.
249+
The server does not recognise the `Mcp-Session-Id` your client sent, because the server **restarted** (or you were routed to a different instance), or because the session **expired**: a legacy session with no request in flight for `session_idle_timeout` (30 minutes by default; an open `GET` stream or a request being answered counts as in flight) is closed, as is one the client ended with `DELETE`. Sessions live in that one process's memory.
250250

251251
There is no server bug to find. The HTTP response is a `404` whose body *is* JSON-RPC, so, unlike the `421` above, the python `Client` shows you this one verbatim:
252252

@@ -256,9 +256,9 @@ There is no server bug to find. The HTTP response is a `404` whose body *is* JSO
256256

257257
The fix is to reconnect: leave the `async with Client(...)` block and enter a new one, which negotiates a fresh session. For a long-lived client, that means catching `MCPError` around your calls and reconnecting on this message rather than retrying inside a dead session.
258258

259-
If it happens *without* a restart, you are running more than one worker without sticky sessions: each worker holds its own session table, so a request routed to the wrong one lands here. **[Deploy & scale](run/deploy.md)** and **[Serving legacy clients](run/legacy-clients.md)** own that story and its two fixes (sticky routing, or `stateless_http=True`).
259+
If it happens *without* a restart and without the client having gone quiet that long, you are running more than one worker without sticky sessions: each worker holds its own session table, so a request routed to the wrong one lands here. **[Deploy & scale](run/deploy.md)** and **[Serving legacy clients](run/legacy-clients.md)** own that story and its two fixes (sticky routing, or `stateless_http=True`).
260260

261-
For the server operator, the matching log line is `Rejected request with unknown or expired session ID: <id>`. It is logged at `INFO`, so it is invisible at the usual `WARNING` threshold. Seeing it in bursts right after a deploy is normal; every connected client is reconnecting.
261+
For the server operator, the matching log line is `Rejected request with unknown or expired session ID: <id>`. It is logged at `INFO`, so it is invisible at the usual `WARNING` threshold. Seeing it in bursts right after a deploy is normal; every connected client is reconnecting. When the session expired instead, that line is preceded by `Session <id> idle timeout`, also at `INFO`.
262262

263263
## `MCPError: Method not found`
264264

@@ -411,7 +411,7 @@ mcp = MCPServer("Weather", request_state_security=RequestStateSecurity(keys=[key
411411
* `Tool already exists:` in the server log is the only sign that two same-named tools collapsed into one.
412412
* One 421, three spellings: `Server returned an error response` (the python `Client`), `421 Misdirected Request` / `Invalid Host header` (everything else), `Invalid Host header: <host>` (the server log). Fix: `transport_security=TransportSecuritySettings(allowed_hosts=[...])`.
413413
* `Task group is not initialized` -> a mounted app whose host lifespan never entered `mcp.session_manager.run()`.
414-
* `Session not found` -> the server restarted; reconnect.
414+
* `Session not found` -> the server restarted or the session expired (`session_idle_timeout`); reconnect.
415415
* `Cannot send 'elicitation/create': ... no back-channel ...` -> `ctx.elicit()` needs a server-to-client channel: a `2026-07-28` connection never has one, `stateless_http=True` takes away the legacy one, and `json_response=True` takes away the request-scoped one. Use a resolver (a legacy client also needs a server that keeps the channel). Its neighbour `Method not found` is a request for a method the other side's protocol revision doesn't have.
416416
* `Client did not declare the form elicitation capability ...` and `Elicitation not supported` -> the client is missing `elicitation_callback=`.
417417
* `Invalid or expired requestState` never says why on the wire. The server log does; `unknown key` means share `RequestStateSecurity(keys=[...])` across workers.

‎src/mcp/server/lowlevel/server.py‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,12 @@ async def main():
6565
from mcp.server.models import InitializationOptions
6666
from mcp.server.runner import serve_dual_era_loop
6767
from mcp.server.streamable_http import EventStore
68-
from mcp.server.streamable_http_manager import StreamableHTTPASGIApp, StreamableHTTPSessionManager
68+
from mcp.server.streamable_http_manager import (
69+
DEFAULT_MAX_SESSIONS,
70+
DEFAULT_SESSION_IDLE_TIMEOUT,
71+
StreamableHTTPASGIApp,
72+
StreamableHTTPSessionManager,
73+
)
6974
from mcp.server.transport_security import DEFAULT_MAX_REQUEST_BODY_SIZE, TransportSecuritySettings
7075
from mcp.shared._stream_protocols import ReadStream, WriteStream
7176
from mcp.shared.exceptions import MCPDeprecationWarning
@@ -722,6 +727,8 @@ def streamable_http_app(
722727
event_store: EventStore | None = None,
723728
retry_interval: int | None = None,
724729
max_request_body_size: int = DEFAULT_MAX_REQUEST_BODY_SIZE,
730+
session_idle_timeout: float | None = DEFAULT_SESSION_IDLE_TIMEOUT,
731+
max_sessions: int | None = DEFAULT_MAX_SESSIONS,
725732
transport_security: TransportSecuritySettings | None = None,
726733
host: str = "127.0.0.1",
727734
auth: AuthSettings | None = None,
@@ -747,6 +754,8 @@ def streamable_http_app(
747754
stateless=stateless_http,
748755
security_settings=transport_security,
749756
max_request_body_size=max_request_body_size,
757+
session_idle_timeout=session_idle_timeout,
758+
max_sessions=max_sessions,
750759
)
751760
self._session_manager = session_manager
752761

‎src/mcp/server/mcpserver/server.py‎

Lines changed: 15 additions & 1 deletion

0 commit comments

Comments
 (0)