iframe-proxy

juliendenize · 2026-10-06T15:59:14Z

Note

Supersedes #258. Identical changes, minus the version bump to 1.13.0. Keeping the version at 1.12.0 means the branch can be installed directly without colliding with the eventual published 1.13.0 release, while we keep adding to mistral-common:

uv pip install "mistral-common @ git+https://github.com/mistralai/mistral-common@feat/response-format-model-settings-no-bump"

Why

Some models constrain their output to a JSON schema. To support this end-to-end, the encoded ModelSettings (which is what the tokenizer emits to condition the model) needs to carry the JSON schema derived from a request's response_format. Grammar consumers also need purpose-specific schema selection: strict grammar uses the custom schema, while non-strict grammar uses a generic object schema.

Summary

Adds json_schema support to ResponseFormat:
- New ResponseFormats.json_schema enum value.
- New JsonSchema model (name, description, custom_schema aliased to schema, strict) with Draft 7 validation.
- ResponseFormat validation requires a schema for json_schema.
- New SchemaRenderingMode and purpose-aware ResponseFormat.get_schema():
  - model_settings always returns the custom schema.
  - grammar returns the custom schema when strict, or a generic object schema when non-strict.
  - json_object returns a generic object/array anyOf; text returns None.
Adds json_schema to encoded ModelSettings (tokenizer >= v15). Encoding now sorts only the top-level setting names instead of sort_keys=True, so the schema's property order (which conveys generation order) is kept.
Reworks ModelSettingsBuilder / FieldBuilder to a typed InputT -> OutputT convert-then-validate design, allowing response_format to produce the json_schema setting.
Adds JSONSchemaBuilder and Draft 7 validation via mistral_common/utils/json_utils.py. Invalid schemas in model settings raise InvalidRequestException; tool-schema validation reuses the same helper and still raises InvalidToolSchemaException (Invalid tool schema: ...).
ChatCompletionRequest.to_openai serializes by alias so schema is emitted correctly.

get_schema() selects the schema for its consumer; grammar generation itself remains the caller's responsibility.

Reviewer guide

src/mistral_common/protocol/instruct/request.py — JsonSchema, SchemaRenderingMode, purpose-specific schema selection, and OpenAI serialization.
src/mistral_common/tokens/tokenizers/model_settings_builder.py — typed field conversion and schema validation for model settings.
src/mistral_common/utils/json_utils.py — Draft 7 schema validation helper.
Tests cover strict/non-strict grammar schema selection separately from the model-settings schema, plus conversion and tokenizer behavior.

Test plan

pytest tests/ — 1619 passed, 17 skipped.
Ruff check/format and mypy across src/ and tests/ — clean.

joa-stdn · 2026-10-12T08:33:08Z

joa-stdn · 2026-10-12T09:29:20Z

+        r"""Render the response format's schema for model-settings encoding."""
+        return input_value.get_schema()
+
+    def _validate_built_value(self, field_name: str, value: JSONSchemaDict) -> None:


do we need this function even?

yeah to implement parent class

but in the other repo this is not used?

joa-stdn · 2026-10-12T10:02:05Z

-            exclude={"messages", "tools", "truncate_for_context_length", "tool_choice"}, exclude_none=True
+            exclude={"messages", "tools", "truncate_for_context_length", "tool_choice"},
+            exclude_none=True,
+            by_alias=True,


are we confident that there is recursively a single alias? should we test for that?

Yes: JsonSchema.custom_schema → schema is the only alias in src/, and messages/tools are excluded from this dump anyway (serialized separately). test_to_openai_renames_custom_schema_to_schema covers it. I didn't add a guard test: aliases exist to match OpenAI field names, so by_alias=True is the behavior we want for any future alias too.

Committed with AI assistance.

Remove the partially-migrated tests/unit/ tree. The migration to unit/integration structure was started prematurely; keep all tests in the flat tests/ directory until the full migration is planned. Removes: - tests/unit/mistral_common/tokens/tokenizers/test_model_settings_builder.py - tests/unit/mistral_common/tokens/tokenizers/test_tokenizer_v15.py - tests/unit/mistral_common/experimental/app/test_models.py Committed with AI assistance.

The strict field is accepted, stored, and round-tripped through OpenAI conversion, but no code path consults it. get_schema() returns the custom schema unconditionally regardless of strict. Tighten the docstring to make this explicit so readers don't expect strict=False to loosen the constraint. Committed with AI assistance.

… None check - Remove ReasoningEffortType TypeVar; use ReasoningEffort | None directly in ModelSettingsConv Protocol (Joachim's suggestion) - Remove redundant if value is not None in JSONSchemaBuilder.validate_built_value; the check in _validate_built_value handles None (Joachim's suggestion) - Keep ModelSettingsConv Protocol for duck typing; it documents the required attributes for build_settings even if ChatCompletionRequest is the only caller Committed with AI assistance.

Require JSON schemas during ResponseFormat validation, remove duplicate runtime checks, and make chat request typing explicit. Drop the unused pickle-specific enum subclass and default invariant JSON schema builder options at the class definition. Committed with AI assistance.

Committed with AI assistance.

Add purpose-aware schema rendering so grammar consumers can honor the OpenAI strict flag without weakening the schema encoded in model settings. Committed with AI assistance.

Committed with AI assistance.

Raise InvalidRequestException for invalid JSON schemas in model settings and cover it, share Draft 7 validation with the tool validator, drop the redundant validation in build_settings, document top-level-only key sorting, remove tests that only exercised Pydantic, and tidy docstrings and call sites. Committed with AI assistance.

Committed with AI assistance.

Rename the settings-to-request field map after the removed protocol and test that it stays in sync with the builder fields. Committed with AI assistance.

A schema attached to a text or json_object response format was silently ignored; ResponseFormat validation now rejects it. Committed with AI assistance.

This reverts commit 18217fc. OpenAI's spec does not forbid a json_schema on text or json_object response formats and vLLM accepts and ignores it, so rejecting it would only break compatible clients. Committed with AI assistance.

joa-stdn · 2026-10-21T06:55:33Z

+        if self.type == ResponseFormats.json_schema:
+            assert self.json_schema is not None
+            if purpose == SchemaRenderingMode.grammar and not self.json_schema.strict:
+                return {"type": "object"}


hmm just realized that technically here it should be {"anyOf": [{"type": "object"}, {"type": "array"}]} no?

yep good catch i updated it

…rammar fallback (#401)

This was referenced Oct 6, 2026

Add response_format support to model settings #258

Closed

Add response format support to generated chat templates #356

Merged

juliendenize added this pull request to stack #357 October 7, 2026 09:39

juliendenize force-pushed the feat/response-format-model-settings-no-bump branch from 2e98ddb to 50a5f5f Compare October 12, 2026 07:54