{{ message }}
Repository navigation
feat: accept inline text input_file content on typed Responses paths - #471
Merged
maralbahari merged 8 commits intoOct 22, 2026
Merged
Conversation
A message input_file is now accepted when file_data carries an inline UTF-8 text file: a base64 data URL with a text media type (text/*, application/json, application/xml, application/yaml), or raw base64 / application/octet-stream with a common text extension in filename. The file is decoded only in the model-facing copy built by normalized_model_input, where it becomes an input_text part in the same position, prefixed with "File: <filename>" when a filename is set. This matches the presentation proposed for vLLM. Stored history and compacted windows keep the client's original input_file, and the text is re-inlined on every continuation, so nothing is persisted that the client did not send. Validation keeps running on new input, rehydrated history and every upstream build, now with a specific 400 reason for file_id, file_url, a missing file_data, non-text or PDF data, non-base64 data URLs, invalid base64 and non-UTF-8 content. No reason echoes the file data or URL. The HTTP handler routes store:false requests that carry an inline text file to the executor, so stateless and stored requests give the model the same input. Other files on store:false keep the byte-transparent raw proxy. Compaction counts an inline file as meaningful context. Also removes the duplicated "File inputs" section in the compaction guide and documents the new behaviour. Closes vllm-project#469 Signed-off-by: MetehanAydemir <mmetehanaydemir@gmail.com>
MetehanAydemir
requested review from
bbrowning,
franciscojavierarceo,
haoshan98,
jiahuei,
leseb,
maralbahari,
noobHappylife,
qandrew and
tjtanaa
as code owners
October 20, 2026 13:29
Closed
5 tasks
maralbahari
reviewed
Oct 21, 2026
maralbahari
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for contributing this fix! I added inline comments
haoshan98
requested changes
Oct 21, 2026
…yFrom Stop routing every request with an inline text file to the typed executor. Raw store:false proxy requests now reach the upstream byte for byte again, so a mixed text and PDF request keeps its upstream status and provider fields such as chat_template_kwargs are no longer dropped. Text files are converted only on paths that already need typed execution. Express the model-facing conversion as TryFrom<&InputFileContent> for InputTextContent with the public InputFileError, and keep validation on InputFileContent without building the filename-prefixed text. normalized_model_input now returns a Result and propagates conversion failures instead of leaving an unsupported file in the upstream request. InputFileContent's serde representation is unchanged. Add regression tests for the raw proxy path and for automatic compaction sending decoded text to the summarizer while storing the original file. Signed-off-by: MetehanAydemir <mmetehanaydemir@gmail.com>
Replace the multi-line format literal and push_str in the InputTextContent conversion with one format string, and write the raw proxy test bodies with explicit escapes. Behaviour is unchanged. Signed-off-by: MetehanAydemir <mmetehanaydemir@gmail.com>
Add record_text_file_input_cassettes.sh, modelled on the image-input recorder, with notes.txt fixtures whose release codeword appears only in the file. Every recording is validated to answer with that codeword before it is promoted. The gateway set was recorded through Agentic API against vLLM serving Qwen/Qwen2.5-3B-Instruct: a single file turn and a previous_response_id continuation, each streaming and non-streaming. The compaction summary and its follow-up on the compacted window were recorded directly against vLLM, because the gateway makes those calls internally. TEXT_FILE_RECORD_SET=openai records the optional OpenAI reference set. text_file_input_test.rs replays every recording through the gateway and checks that the model receives the File: notes.txt input_text with no input_file, that stored history and the compacted window keep the original input_file, that continuation decodes the stored file again, and that the summarizer receives exactly the recorded input. OpenAI recordings are replayed and compared with the gateway set once they exist. Signed-off-by: MetehanAydemir <mmetehanaydemir@gmail.com>
An explicit media type used to win outright, and only text/*, JSON, XML and YAML counted as text, so JavaScript, shell, SQL, TOML, JSON-LD and TypeScript files were rejected. Filenames were matched only by a short extension list, so Dockerfile, Makefile, dotfiles and many source extensions never matched. Move the classification into file_text/kinds.rs with reviewed lists: any +json or +xml type and text application/* types count as text media types; the extension list covers common source and config files; and known extensionless names and dotfiles are matched on the basename. A text filename now decides for raw base64 and for any media type except PDF, image, audio and video, and video/mp2t with a .ts, .mts or .cts name is treated as TypeScript. Strict UTF-8 decoding still rejects binary content. The not-text error now explains that raw base64 or a generic media type needs a filename with a text extension. Signed-off-by: MetehanAydemir <mmetehanaydemir@gmail.com>
Signed-off-by: MetehanAydemir <mmetehanaydemir@gmail.com>
maralbahari
reviewed
Oct 21, 2026
maralbahari
left a comment
Collaborator
There was a problem hiding this comment.
thanks for the comprehensive test support.
Collaborator
There was a problem hiding this comment.
nit: Now that the OpenAI cassettes are committed, this conditional fallback is unnecessary. Could you simplify the loading and remove the outdated “optional until recorded” comments?
Contributor
Author
There was a problem hiding this comment.
Thanks @maralbahari, cleaned it up the outdated "optional" comments are gone. Thank you for giving your time to review this, I'm happy to contribute!
maralbahari
approved these changes
Oct 21, 2026
The OpenAI reference recordings are committed, so load both providers' cassettes unconditionally, drop the Option plumbing and skip note from the replay test, and remove the wording that described the OpenAI set as optional from the test, the cassettes README, and the recorder script. Signed-off-by: MetehanAydemir <mmetehanaydemir@gmail.com>
haoshan98
approved these changes
Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Closes #469.
The typed Responses executor now accepts a user-message
input_filewhen itsfile_datacarries an inline UTF-8 textfile, and sends it to the model as text. Until now every message
input_filegot a 400 (#256/#258). That meant atext-only model behind the gateway could not read a
.txt,.md,.jsonor.yamlattachment a client sent inline.What is accepted:
text/*,application/json,application/xml,application/yaml).Parameters such as
charsetare ignored, and an explicit media type takes precedence over the filename;application/octet-stream/ an empty media type, with afilenamethat has a common text extension.How it works:
types/io/input/validation.rs, newtypes/io/input/file_text.rs). The existing validation runs onnew input, rehydrated
previous_response_id/conversationhistory and every upstream build (including thecompaction summary). It now decodes the file (
base64crate, already a dependency, then strict UTF-8). Each othercase gets a specific 400 reason:
file_id(no Files API),file_url(no fetching), missingfile_data, non-text orPDF data, a non-base64 data URL, invalid base64, non-UTF-8. The
input[i].content[j]:prefix is kept, and no reasonechoes the file data, URL or media type.
normalized_model_input). The file becomes aninput_textpart in the same position,prefixed with
File: <filename>\nwhen a filename is set. This is the same presentation proposed in vLLM #62676, soboth paths would give the model the same prompt.
input_file, and the textis re-inlined on each continuation. This follows the fix: preserve and verify image input through Responses #263 rule: never persist content the client did not send.
store: falserequest with an inline text file goes to the executor instead of the raw proxy, sostateless and stored requests give the model the same input.
store: falserequests withfile_id,file_urlorPDF files are still raw-proxied byte for byte. The predicate (
ResponsesInput::has_inline_text_file) only classifiesthe reference fields and the media type, without decoding. It is checked in the HTTP routing decision next to the
tool-search state, not in
in_process_feature().in_process_feature()also gates split execution inagentic-llm-d, and split execution already builds its upstream body throughto_upstream_request, so it can servethese files.
executor/compaction/context.rs). An inline file counts as meaningful context, so a compaction whoseinput is only a file no longer fails with "requires non-empty input".
docs/guides/responses-compaction.md. The section appeared twice, sothis also removes the duplicate. Added an
[Unreleased]entry.Not in scope:
file_idresolution (#356), fetchingfile_url, PDF/DOCX extraction, Chat Completionsfileparts(that route is a passthrough), and a separate decoded-size limit. Inline files are bounded by the existing
max_request_body_size_bytes, and the body shrinks after decoding. No new dependencies, no I/O. No changes toevents/, the ingestion state machine, the relay orengine.rs.Open questions for reviewers:
store: falsetext-file requests go through the executor, as here, or should the raw proxy staybyte-transparent for them? Dropping the routing change is a two-line revert in the HTTP handler.
File: <filename>header the format you want?ResponsesConfig? Adding a field would break struct literals, so it isleft out here.
length? It is unchanged here, which overestimates.
Test Plan
Unit tests (
agentic-server-core):file_text.rs: accepted media types (includingcharsetparameters and uppercase types), anapplication/octet-streamor empty media type falling back to the filename, raw base64 with.yaml/.YML, emptyand multi-byte UTF-8 files, a PDF rejected by media type (even with a
.txtfilename) and by filename,file_id,file_url, missing data, a non-base64 data URL, invalid base64, non-UTF-8, and routing classification (textfiles are detected without decoding; references, PDFs, images and tool-output files are not).
validation.rs: file errors keep theirinput[i].content[j]position and never echo the URL. Inline text files passin new input and in restored history. The previous
input_filecase in the position test now uses an unknown parttype, and
file_urlis covered by its own test.conversions.rs: the model copy hasinput_textwith the header in the original order. Canonical items keepinput_file. Input with no files staysCow::Borrowed.Integration tests (
crates/agentic-server/tests/input_file_repro.rs, mock upstream and in-memory SQLite). All 30existing tests still pass unchanged. They use a
file_urlPDF, which is still rejected before inference and beforepersistence, and the raw-proxy byte-for-byte controls still hold. 9 new tests:
store: truewith a data URL: 200. The upstream getsinput_text"File: notes.txt\n..."with nofile_data, andthe stored item is the original
input_file;filename: config.yamlis inlined;previous_response_idfollow-up re-inlines the stored file, and storage still holds the single originalinput_file;response.createis accepted and inlined;store: falsewith a text file runs through the executor: inlined, nothing persisted;store: falsewith invalid base64 gets a 400 from the gateway instead of being proxied;/v1/responses/compactwith a message containing only a file succeeds. The summary request has the text, and thereturned window keeps
input_file;file_id,file_url, missing data, PDF data URL, PDF filename, non-base64 data URL, invalidbase64, non-UTF-8) gives a 400 with the specific message: no inference, nothing persisted.
Commands, all run in WSL Ubuntu-22.04 with Rust 1.98.1:
cargo fmt --all --check: passcargo clippy --all-targets -- -D warnings: passcargo test -p agentic-server --test input_file_repro: 39 passed, 0 failedcargo test -p agentic-server: 710 passed, 0 failed (42 test binaries)cargo test -p agentic-server-core: 1776 passed, 1 failed, 17 ignored. The failure istool_search_characterization_test::provider_parity_recorder_generated_matrix_has_one_semantic_flow, which assertsthat checked-in cassettes are not executable. It fails only because the checkout is on a Windows drive mounted in WSL
(
/mnt/c), where every file reports mode 0777. This change touches no cassettes.python scripts/check_rust_file_sizes.py(therust-file-sizeshook): pass, 291 production files checked. The newfile_text.rshas 155 production lines.types/io/input.rshas 485 production lines, plus onemodline.git diff --check: clean