Symptom

After the GPU node of an elastic EP deployment was replaced (terraform scale-down/scale-up of the node group; head pod on the CPU node untouched), every native scale command to the master engine fails deterministically with HTTP 500. The engine log shows ERROR api_router.py:79 Scale failed: '023f790ec4893808048534ba5ee02cbf8087cc77164cefca7393271b' — the message is a Python KeyError whose key is a DEAD Ray node ID still present in the head's GCS.

Environment and trigger

• Engine image: the W07-W10 native engine (thxcode/gpustack-operator:dev-s2-vllm-ray-e9f1c63c0a98, vLLM v0.29.0 native elastic EP + Ray).
• The head's GCS accumulated 8 dead H100 worker nodes across two GPU-node generations (dead nodes are never forgotten while the head runs).
• With 4 alive GPU nodes registered, POST /scale_elastic_ep {"new_data_parallel_size": 4, "drain_timeout": 120} KeyErrors on one of the dead node IDs — reproduced live in W10 PASS-4 at 17:17:34Z; the same deployment scaled 2→4→2 cleanly in round-1 when the GCS held no dead nodes.

Notes for the fix

• The failing lookup is inside the elastic scale path below vllm/entrypoints/serve/elastic_ep/api_router.py:65 (client.scale_elastic_ep → vllm/v1/engine/async_llm.py:_scale_elastic_ep → engine_core.prepare_elastic_ep / commit_elastic_ep). The KeyError key is a Ray node_id hex.
• api_router.py:79 logs logger.error("Scale failed: %s", e) WITHOUT exc_info, so the traceback never reaches the log — please add it as part of the fix.
• Suggested direction: build the scale-time node/actor view from ALIVE Ray nodes only (or tolerate missing IDs), so dead GCS history cannot abort a scale.

Impact

This blocks the batch-cycle operating model for elastic EP: any GPU node replacement (preemption, scale-down/up, upgrade) leaves dead nodes in the head GCS, and every later scale command fails until the head pod restarts. The operator-side half (the op parking in AwaitingWidth with no refusal surfaced) is filed separately.