Summary

A single request to the public generation API with a client-controlled parameters.pd_kv_trans_params string permanently kills the engine's inference loop thread. The HTTP process stays alive and /health keeps returning 200 while every later request hangs, so both health checks and process supervisors miss the outage. Reproduced on a normal node and, through the pd_master forwarding the field to the decode leg, on PD deployments. Suggested severity High, CVSS 3.1 AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H (7.5), CWE-502.

Details

The native /generate handler in lightllm/server/api_lightllm.py passes request_dict["parameters"] verbatim to SamplingParams.init(**sample_params_dict). In lightllm/server/core/objs/sampling_params.py, init() stores the client value with self.pd_kv_trans_params.set(kwargs.get("pd_kv_trans_params", None)).

The comment right above the sibling PD fields in init() states those are PD master internal scheduling fields that must not be enabled or modified by external request parameters. Two of them, pd_high_priority_request and pd_node_resource_wait_timeout_seconds, are indeed hardcoded. But pd_kv_trans_params (and pd_master_node_id, suggested_dp_index on the same pattern) is still read from client kwargs.

PDKVTransParamObj.set() in lightllm/server/core/objs/pd_kv_trans_params.py only asserts the value is at most 8 KB and ctypes.memmoves it into shared memory, so a plain JSON string is accepted as-is.

For every request in every run mode, InferSamplingParams.__init__ in lightllm/server/router/model_infer/infer_batch.py runs:

if self.shm_param.pd_kv_trans_params.data_len > 0:
    self.pd_decode_node: PDDecodeNodeInfo = pickle.loads(self.shm_param.pd_kv_trans_params.get())

The unpickle runs inside infer_loop (lightllm/server/router/model_infer/mode_backend/chunked_prefill/impl.py), whose only handler logs and re-raises. The loop runs in a worker thread, so the exception kills the thread and leaves the process up.

On Linux the stored bytes cannot form a valid pickle: ctypes.memmove copies a Python str as UTF-32 (4 bytes per character), so any non-empty string arrives with NUL padding and pickle.loads raises UnpicklingError. The failure is deterministic, which makes it a reliable one-request denial of service rather than a code execution primitive on this platform.

In PD mode, fetch_pd_stream() in lightllm/server/httpserver_for_pd_master/manager.py pickles the same sampling_params object, client field untouched, and sends it to both the prefill and the decode leg. The prefill leg overwrites the field once it receives the decode node info (in lightllm/server/httpserver/manager.py, before the request is written to shm), the decode leg never overwrites it, so the decode engine hits the same unpickle while the prefill node is unaffected.

Verified at commit 1eb4810c78c6ea908b69c20c3c037b79979e1cd6 (v1.2.0+50). The same code is present on current main (checked 2026-10-15): the kwargs read in sampling_params.py and the unguarded pickle.loads in infer_batch.py are unchanged.

PoC

Tested on Linux, LightLLM at commit 1eb4810c78c6ea908b69c20c3c037b79979e1cd6 (v1.2.0+50), Qwen2.5-0.5B-Instruct, normal run mode on one GPU, server on port 8190, /health returning 200.

Baselines, both return HTTP 200:

curl -m 30 http://<node_ip>:8190/generate -H "Content-Type: application/json" \
  -d '{"inputs":"hi","parameters":{"max_new_tokens":8}}'

curl -m 30 http://<node_ip>:8190/generate -H "Content-Type: application/json" \
  -d '{"inputs":"hi","parameters":{"pd_kv_trans_params":"","max_new_tokens":8}}'

The empty string leaves data_len == 0 and skips the unpickle, which pins the trigger to the data_len > 0 branch.

The kill request:

curl -m 30 http://<node_ip>:8190/generate -H "Content-Type: application/json" \
  -d '{"inputs":"hi","parameters":{"pd_kv_trans_params":"AAAAdeadbeefBBBB","max_new_tokens":8}}'

Actual result: the curl times out. The node log shows _pickle.UnpicklingError: invalid load key, 'A'. raised from InferSamplingParams.__init__ in infer_batch.py, printed under Exception in thread Thread-N (infer_loop). The process is still running with the same PID, curl http://<node_ip>:8190/health still returns {"message":"Ok"}, and every later /generate, including plain requests without the field, hangs until the node is restarted. Reproduced 3/3 on cold starts.

PD variant, same request body sent to the pd_master client-facing port (run_mode pd_master, one prefill node and one decode node): the decode node logs the same UnpicklingError in its infer_loop thread and stops serving, the prefill node shows no error, and the master /health stays 200.

Impact

CWE-502: client-controlled bytes reach pickle.loads in the inference process through a documented request field.

Any client that can call /generate on the public API triggers this with one request. No credentials beyond what the deployment already requires, no internal network position, and the attack repeats at will. All run modes are affected; in PD deployments the entry is the pd_master port and the victim is the decode engine.

The demonstrated impact on Linux is availability only, but it is a permanent blind state: the inference thread is dead while /health stays 200 and the process stays alive, so orchestrators that route by health and supervisors that restart on process death both miss it. Recovery needs an explicit node restart. No confidentiality or integrity impact was demonstrated on this platform. On platforms where ctypes copies a Python str as UTF-16 (wchar_t of 2 bytes), the same bytes can encode a valid pickle and the same path would be remote code execution; that variant was not tested.

Suggested severity High, CVSS 3.1 AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H (7.5).