…858)

Fixes stream cleanup for chat and legacy completions. Previously,
breaking out of a stream could leave the model lock held and block
subsequent requests. Creating a stream without consuming it could also
hold the lock indefinitely.

The model lock is now acquired on the first iteration and released when
the stream finishes, throws, or is closed. Early closure also finalizes
the partial assistant reply and resets any request-specific seed.

Worker streams are tracked by a unique stream ID instead of the model
ID, so separate requests cannot overwrite each other’s iterator. Closing
the client iterator now closes the worker iterator too. Native async
generators preserve ordering for overlapping `next()`, `return()`, and
`throw()` calls.

Generation remains serialized per loaded model instance. Stream creation
no longer reserves its place in the queue (iteration does). This does
not add parallel generation on one model instance or introduce
checkpointing.

The existing chunk-generation code is extracted into helpers for reuse
by follow-up resumability work.

Adds opt-in checkpointing for chat requests so generation can continue
after a page reload or browser crash. The prompt, generated tokens, and
RNG state are saved to OPFS. Recovery loads a KV checkpoint when one is
available. Otherwise, it re-prefills the saved prompt and replays the
saved tokens before generating more. KV checkpoints require a compatible
model library while token replay does not.

Supports streaming, non-streaming, and worker use, with configurable
durability and persistence failure handling. Includes session
inspection, listing and deletion, cross-tab locking, and checkpoint
cleanup. Most orchestration lives in a coordinator outside `engine.ts`.

Ordinary requests keep their existing prefix reuse behavior. Each new
resumable request prefills the full conversation, but this does not add
a persistent prefix cache between requests.

Currently limited to single-choice, text-only chat without constrained
decoding or custom `LogitProcessors`. Recovery requires the same model
artifacts and configuration.

Adds a Qwen3 example for saving a generation session, reloading or
reopening the page, and restoring the saved text before continuing.
Supports dedicated-worker and main-thread execution, KV checkpoint
recovery, token replay, exact and relaxed persistence, and session
listing and deletion. The model WASM URL is configurable. KV recovery
requires a checkpoint-capable model library while token replay works
without one.

Fix placeholders for copyright year and owner in LICENSE file

Add audio input to chat completions for models whose package includes
`mlc-model-manifest.json`. Gemma 4 E2B is the first one
(mlc-ai/mlc-llm#3559). The manifest is read only when the model record
sets `model_manifest`, so every other model loads and runs as before.

1. Manifest: parse and validate `mlc-model-manifest.json` and the
matching `_metadata.artifact` in the compiled library, check that their
`interface_id` and `parameter_schema_id` agree, and resolve the chat
config and `tensor-cache.json` from it. A manifest that is invalid or
does not match fails the load
2. API: accept `input_audio` content parts, either base64 WAV as in the
OpenAI API or mono PCM as a `Float32Array`
3. Audio: decode WAV, including the extensible header, downmix to mono,
low-pass and resample to the rate the manifest asks for
4. Runtime: build the prompt from text and audio segments, call the
functions the library names under its roles, and split audio embeddings
across prefill chunks. A library declares `prefill_tokens` and
`decode_tokens`, which take token IDs, or `prefill_embeds` and
`decode_embeds`, which take embeddings only (mlc-ai/mlc-llm#3564).
Chunks after the first are copied on the GPU because the compiled
kernels need inputs with no byte offset
5. Conversation: `render_empty_system_message` lets a template omit the
system turn when there is no system message
6. Docs for the audio input and its limits

Tested in Chromium 149 on an M2 Ultra with a q4f16_1 Gemma 4 E2B build.
One text and four audio requests give the same token IDs as native
Metal, cold and warm, with no WebGPU errors. A Qwen3 0.6B build without
a manifest loads and generates as before and makes no manifest request.
The same Qwen3 build with a manifest that declares the embedding roles
gives the same token IDs as without one.

Input must be WAV or PCM. URLs, compressed formats, vision and video are
out of scope. There is no prebuilt model record yet.

Follow-up to #793. The audio section repeated details that belong in the
MLC manifest docs.
1. Drop the paragraph on the prefill and decode roles and the record
count for the Gemma 4 weights
2. Shorten the comments on prompt assembly, the audio size check, and
the resampler

`completion_tokens` was the number of decode steps, so it left out the
token sampled after prefill and reported one fewer than the tokens the
client received (127 for a 128-token reply).

1. Count tokens as they are committed to the output: the prefill token,
every decoded token, and a terminating stop token. Resumed generation
still adds its already-emitted tokens on top.
2. `decode_tokens_per_s` and `time_per_output_token_s` keep using the
decode step count, and no longer divide by zero when a reply ends during
prefill
3. The mock pipeline counts the same way, and the existing usage tests
expect one more token per choice

`interruptGenerate()` sets a flag that the non-streaming
`chatCompletion` and `completion` paths check before generating each
choice, but only `_generate()` and the streaming path reset it. An
interrupt raised after one request had finished aborted the next
non-streaming request with an empty reply.

1. Reset the flag when a non-streaming request takes the model lock, so
an interrupt only affects the request that was running when it was
raised
2. Tests for the chat and text completion paths

… reply (#866)

When a reply ends with `abort` or `length`, its last sampled token is in
the visible text but was never fed back into the KV cache. A client that
continues that conversation matches the multiround path, the cache is
reused, and the next reply is conditioned on a prefix one token shorter
than what the client saw.

1. Treat a conversation whose previous reply ended with `abort` or
`length` as new: reset the cache and rebuild it from the caller's
messages, as a fresh request would
2. `resetChat` clears the recorded finish reason, so a reset
conversation never counts as interrupted
3. A test for each finish reason

A continuation after such a reply now costs a full prefill of the
conversation instead of the new turn only.

A manifest can declare an image input (mlc-ai/mlc-llm#3566). This reads
it the same way the audio path does. Models without a manifest keep the
existing image path.

1. Manifest: parse the `image_decode` processor (pixel format, layout,
resize rule, embedding count) and the optional `adapter_dtypes` a
compiled program may carry. The library is rejected when the input has
no adapter or prompt, or when it names an adapter the program does not
export.
2. Runtime: decode the image, scale and crop it to the declared size on
a canvas, hand the compiled adapter an `[1, height, width, 3]` tensor in
the declared dtype, check it returns the declared number of embeddings,
and splice them into the prompt at the placeholder like audio. WebGPU
has no 8 bit storage type, so a WebGPU library takes the pixels as
uint32.
3. Request validation accepts `image_url` parts when the loaded model
declares an image input
4. Hybrid models: a program that declares `create_rnn_state` next to
`create_kv_cache` is a KV cache plus recurrent state model, and the
library's `kv_state_kind` must agree. Prefill and decode receive every
active state in role order, as the legacy path already does.
5. Docs and tests

Verified in Chromium with q4f16_1 builds of LLaVA 1.5 7B
(mlc-ai/mlc-llm#3566), Gemma 3 4B (mlc-ai/mlc-llm#3567) and Qwen3.5 0.8B
(mlc-ai/mlc-llm#3568).

`getArtifactAudioEmbeddings` throws on a wrong-shaped adapter output
before closing the scope it opened. The prefill's cleanup then closes
that scope instead of its own, so the prefill scope stays open for every
later request.

1. Close the adapter scope in a `finally` block
2. Attach the result to the caller's scope before validating it, so a
rejected tensor is freed with that scope
3. One test, for both the audio and image adapters, that a rejected
output is disposed when the caller's scope closes. It replaces the image
test's scope count, which could not see a leaked tensor.

Update WebLLM to 0.2.86 with `@mlc-ai/web-runtime@0.28.0-dev0` and the
rebuilt `v0_2_86/base` model libraries. Add Gemma 4 E2B
instruction-tuned records for `q4f16_1` and `q4f32_1`, including their
package manifests and 192 MiB storage-buffer requirement. The FP16
record requires `shader-f16`; the FP32 record does not.

Manifest loading remains opt-in through `model_manifest`. Existing
records retain the legacy loading path and use the rebuilt libraries.

Use 4K-prefill Phi-3.5 vision libraries so a default square image, which
produces 2,509 image tokens, fits in one prefill. Update its memory
estimates and add Gemma memory estimates. The binary release retains the
previous 2K Phi variants.

Validation: 466 WebLLM tests and 105 runtime tests passed. All 224
libraries (112 each in base and sg32) compiled and loaded for metadata
inspection, with 334 model/library parameter-schema checks passing.
Browser checks cover Gemma text, streaming, audio, and prompts spanning
multiple prefill chunks; legacy Qwen3, Qwen3.5, and Llama generation;
Arctic embeddings; and square-image Phi vision for both quantizations.
These checks passed in both base and subgroup variants. Gemma and Phi
FP32 were tested with `shader-f16` disabled on the device.

Published model packages:
- https://huggingface.co/mlc-ai/gemma-4-E2B-it-q4f16_1-MLC
- https://huggingface.co/mlc-ai/gemma-4-E2B-it-q4f32_1-MLC

The [binary
release](https://github.com/mlc-ai/binary-mlc-llm-libs/tree/main/web-llm-models/v0_2_86)
includes source revisions, configuration snapshots, checksums,
validation results, and a rebuild script. It uses Apache TVM main
`27576ce2d1` with a WebGPU FP32 prefill fix and MLC-LLM main
`8978ea9aec` with compatibility updates for the current TVM APIs. The
exact compiler commits are recorded in `build-manifest.json`.