studio: serve installed mlx models through the openai-compatible api by mahiatlinux · Pull Request #8768 · unslothai/unsloth · GitHub
Skip to content

studio: serve installed mlx models through the openai-compatible api - #8768

Merged
mahiatlinux merged 38 commits into
unslothai:mainfrom
mahiatlinux:fix/studio-mlx-api-model-discovery
Aug 28, 2026
Merged

mahiatlinux merged 38 commits into
unslothai:mainfrom
mahiatlinux:fix/studio-mlx-api-model-discovery

Conversation

@mahiatlinux

Copy link
Copy Markdown
Contributor

Fixes #8748.

An MLX model downloaded through Unsloth Desktop was invisible to the OpenAI-compatible API until someone loaded it by hand. GET /v1/models did not list it, and a chat-completions request naming its exact id returned 404 model_not_found with "is not downloaded on this server", while installed GGUF models in the same setup were both listed and switchable.

/v1/models and the "Switch model by request" resolver both filtered the local catalog through a GGUF-only check (local_gguf_quants, resolve_local_gguf). The loaded-model branch is backend-agnostic, which is why the same model worked the moment it was loaded in the UI.

What changed

studio/backend/core/inference/local_model_resolver.py

_local_weights_entry classifies a local non-GGUF checkpoint as servable when a root config.json sits beside safetensors weights. Diffusers pipelines, partial downloads and bare LoRA adapters are rejected: the first belong to the image and video backends, the other two hand out an id whose load is guaranteed to fail. _local_servable_entry tries GGUF first and falls back to that, and _build_index indexes both kinds, so resolve_local_gguf now resolves an MLX model with gguf_variant = None.

local_gguf_quants and info_has_local_gguf are replaced by local_servable_model, which returns (is_gguf, quants) from one on-disk scan.

local_target_is_gguf decides which backend serves a switch target by reading the resolved on-disk path rather than re-querying the index, so an index rebuild mid-switch cannot classify an MLX target as GGUF and strand its alias on the wrong backend.

studio/backend/routes/inference.py

_servable_catalog_rows (module scope, always called through asyncio.to_thread) does the catalog pass off the event loop. _openai_catalog_objects lists non-GGUF models with no quant, since their quantization is baked into the weights, and checks residency without the llama-only restriction for those rows.

_maybe_auto_switch_model resolves the serving backend once into target_backend and routes a non-GGUF load to the inference orchestrator, recording the requested repo id as _openai_advertised_id there. _load_model_impl clears that alias on every orchestrator load, mirroring the GGUF branch.

_orchestrator_public_model_id gives GET /v1/models, GET /v1/models/{id}, the chat and audio response bodies and the API monitor a single id for an auto-switched model. The monitor reads it only while active_model_name is set, since an unload leaves the alias behind. _loaded_satisfies, _loaded_identity_satisfies and _resident_id_is_namespaced match on that alias too.

studio/backend/core/inference/mlx_inference.py

load_model clamps max_seq_length to 2048 when it arrives as 0, matching core/inference/inference.py. LoadRequest defaults the field to 0, and an API switch with no saved per-model override sends that value straight through.

Settings copy

The Settings > API description said the toggle loads a downloaded GGUF. It now says model, in all twelve locales. autoDownloadDescription still says GGUF, because automatic download remains GGUF-only.

Verification

End to end on Apple Silicon (M4, DEVICE: MLX) against the issue's own reproduction, driving the real router over HTTP with the real resolver and a real mlx-lm load of mlx-community/Qwen3-0.6B-4bit:

step result
GET /v1/models, model never loaded listed, loaded: false, no quant
POST /v1/chat/completions naming it 200, loaded on demand and generated
response model field mlx-community/Qwen3-0.6B-4bit
alias placement on the orchestrator, llama.cpp backend untouched
/v1/models row, /v1/models/{id}, monitor label the same id on all three
second request served without a reload
after an unload monitor reports nothing

Studio UI checked by hand. 2824 backend tests pass, including test_openai_auto_switch.py, test_openai_catalog.py, test_openai_auto_download.py, test_async_singleton_access.py, test_api_monitor.py and test_refactor_guard.py. New regression tests cover the resolver's rejections, the switch to an unloaded MLX model, residency of a resident non-GGUF row, id consistency across the /v1 surfaces, and the monitor after an unload. ruff check and the i18n parity check are clean.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 84835610d4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread studio/backend/routes/inference.py
Comment thread studio/backend/routes/inference.py
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4bae121fef

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread studio/backend/routes/inference.py
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 87cc929b42

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/core/inference/local_model_resolver.py
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 14, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 67ea601668

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/routes/inference.py Outdated
Comment thread studio/backend/routes/inference.py
Comment thread studio/backend/routes/inference.py
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 14, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 14, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 14, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 14, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ffc2e60fc0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread studio/backend/routes/inference.py Outdated
Comment thread studio/backend/core/inference/local_model_resolver.py
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 14, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 14, 2026
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 14, 2026
@Lyxot

Lyxot commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ffc2e60fc0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/routes/inference.py Outdated
Comment thread studio/backend/routes/inference.py
@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 14, 2026
@mahiatlinux

Copy link
Copy Markdown
Contributor Author

@codex review

@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 14, 2026
@mahiatlinux

Copy link
Copy Markdown
Contributor Author

@codex review

@unslothai unslothai deleted a comment from chatgpt-codex-connector Bot Aug 15, 2026
@mahiatlinux
mahiatlinux force-pushed the fix/studio-mlx-api-model-discovery branch from ffc2e60 to 8407b3d Compare August 15, 2026 02:24
@mahiatlinux

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d4a6c1655a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread studio/backend/routes/inference.py Outdated
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/routes/inference.py
@mahiatlinux

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d2d82219f8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
@mahiatlinux

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9c5b2be285

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/routes/inference.py Outdated
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/routes/inference.py
@mahiatlinux

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 473ca1b57c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/core/inference/local_model_resolver.py
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/routes/inference.py
Comment thread studio/backend/core/inference/local_model_resolver.py Outdated
Comment thread studio/backend/routes/inference.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9e36894888

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread studio/backend/routes/inference.py Outdated
@mahiatlinux

Copy link
Copy Markdown
Contributor Author

Exact head 09a854d4a477ac2f94686f13cf7371b81f131b66 contains the latest review repairs.

Confirmed and fixed:

  • 796c45d92 preflight empty anthropic images: native Anthropic image blocks with data: "" set the vision requirement but were omitted from preflight. A model switch could evict the resident model before translation dropped the empty image. _anthropic_local_image_payloads() now passes every string to the existing decoder, so invalid empty data returns HTTP 400 before any load. The route regression covers native base64, data URLs, and empty native base64 with zero load calls and the resident model unchanged.
  • 26df333f4 restrict conditional audio model discovery: the Transformers audio registry included speech classification, speech seq2seq, and waveform model types that the chat backend cannot serve. Local discovery now admits only the supported conditional audio paths, CSM and Whisper. Focused regressions prove Speech2Text and MusicGen checkpoints stay out of /v1/models and cannot trigger a destructive auto-switch.
  • 09a854d4a deduplicate resident non-gguf catalog rows: a manually loaded LM Studio checkpoint could appear twice as a path-derived basename and a scanned namespaced id, both marked loaded. The catalog now drops the exact resident non-GGUF scan row when the orchestrator entry is already present. Regressions preserve context metadata and prove a distinct unloaded path with the same basename remains listed.

Investigated and rejected:

  • BOM-prefixed config.json discovery was not changed. Every supported Transformers line reads local configs as plain UTF-8, and pinned mlx-lm does the same. Accepting BOM only in discovery would advertise the checkpoint, evict the resident model, then fail the real load.

Verification at this head:

  • 26 focused resolver, catalog, and Anthropic route cases passed.
  • All 20 test_openai_catalog.py cases passed.
  • Ruff passed on all four touched Python files.
  • Python 3.9 parsed all four touched Python files.
  • Fresh independent post-push resolver, routing, and CI reviews are running. Final upstream and staging CI will be tracked after convergence.

@mahiatlinux

Copy link
Copy Markdown
Contributor Author

Why this diff is large:

  • The first eight feature commits already contained 1,796 additions and 149 deletions across 18 files. The model discovery and auto-switch feature crosses local resolution, API routing, catalog output, request preflight, settings, and locale labels.
  • The current diff is 2,745 additions and 209 deletions across 26 files. Repair and convergence added 949 insertions and 60 deletions.
  • Most of that growth is regression coverage. test_openai_auto_switch.py grew by another 613 insertions and 5 deletions, and test_openai_catalog.py grew by 66 insertions. The route implementation grew by 155 insertions and 31 deletions.
  • The remaining added files came from real merge and CI failures: the frontend runtime-key cycle, the cross-browser dismissal probe, source-encoding checks, and the official tool-activity browser smoke.
  • The latest three confirmed fixes added 102 insertions and 13 deletions across four files. Ninety-five of those insertions are focused tests.

There are no model files, generated bundles, new dependencies, or unrelated product features in the diff. The size is mainly the original broad feature plus defensive regression coverage. Extra agent review passes are stopped; only existing convergence threads and CI remain.

@mahiatlinux

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 09a854d4a4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread studio/backend/routes/inference.py Outdated
@mahiatlinux

Copy link
Copy Markdown
Contributor Author

Convergence fix a00f3417b handles the final OpenAI empty-image edge at exact head a00f3417b. An empty data:image/...;base64, part now reaches non-GGUF preflight, returns HTTP 400, makes zero load calls, and preserves the resident model. Five focused image-preflight cases, Ruff, and Python 3.9 parsing passed.

@mahiatlinux

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a00f3417b3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread studio/backend/routes/inference.py
@mahiatlinux

Copy link
Copy Markdown
Contributor Author

Convergence fix bc7f2b1e2 rejects unsupported remote image URLs before a non-GGUF model switch. The regression covers remote and empty data URLs, returns HTTP 400, makes zero load calls, and preserves the resident model. Six focused image-preflight cases, Ruff, and Python 3.9 parsing passed.

@mahiatlinux

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: bc7f2b1e2a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mahiatlinux

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: bc7f2b1e2a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

# Conflicts:
#	.github/workflows/studio-frontend-ci.yml
#	tests/studio/probe_dismiss_guard.py
@mahiatlinux

Copy link
Copy Markdown
Contributor Author

Final repair and convergence summary

Final head: 19d172b5c6acef84cc9ed707b33ac242b8494f6e

Confirmed defects fixed

  • Local MLX checkpoints were absent from /v1/models and could not be selected by an OpenAI-compatible request. The resolver now indexes complete local non-GGUF checkpoints, rejects incomplete, adapter-only, and Diffusers trees, and routes each resolved target to the correct backend.
  • Model identity could drift across catalog rows, request matching, response bodies, the API monitor, unloads, failed loads, and aliases with the same basename. The fixes use exact paths and advertised ids, restore the previous alias after a failed load, publish completed switches before late cancellation, and deduplicate only the exact resident non-GGUF row.
  • Audio discovery admitted checkpoint families that the chat backend cannot serve. Discovery now limits conditional audio models to supported CSM and Whisper paths while keeping unsupported Speech2Text and MusicGen checkpoints out of the switchable catalog.
  • Media validation sometimes happened after a switch had already evicted the resident model. OpenAI, Anthropic, and GGUF paths now reject malformed, empty, mixed, unsupported, or remote image and audio payloads before any model load. Valid prior-turn images remain allowed for audio switches.
  • Local revision scans and source encoding checks had CI compatibility gaps. Local checkpoints now skip inappropriate GGUF revision scans, the memory contract declares its encoding, and CPU replay metadata matches the launcher.
  • Frontend CI exposed a runtime-key import cycle and gaps in the browser smoke wiring. The storage key moved to an acyclic module, tool activity runs in official CI, and the dismissal probe now requires menu closure plus focus restoration before Space can reopen the menu.
  • Latest main conflicted in studio-frontend-ci.yml and probe_dismiss_guard.py. The merge keeps main's current --json Chromium tool-activity invocation and its cleaner focus facts while preserving the stricter pre-Space closure assertion.

Why the diff is large

The original feature already crossed local model resolution, API routing, catalog output, request preflight, MLX loading, settings copy, and twelve locale files. Repair rounds added focused regression coverage for every destructive switch edge case and the CI-only frontend failures. The final GitHub diff is 2,691 additions and 192 deletions across 19 files; most growth is tests. There are no model artifacts, generated bundles, new dependencies, or unrelated product features. The local conflict merge lists 41 files because it incorporates current main; those upstream files disappear from the PR diff against that same base.

Final verification

  • 372 focused backend catalog, routing, auto-switch, and API monitor regressions passed on the final head.
  • 8 focused workflow and Playwright-suite wiring tests passed.
  • Ruff, Python 3.9 syntax parsing, YAML parsing, ancestry checks, and git diff --check passed.
  • Focused fork evidence: https://github.com/mahiatlinux/unsloth/actions/runs/33144588261
  • Review state: approved, mergeable, and 0 unresolved review threads.

@mahiatlinux

Copy link
Copy Markdown
Contributor Author

@mahiatlinux
mahiatlinux merged commit 37d3b6f into unslothai:main Aug 28, 2026
44 of 50 checks passed
@mahiatlinux
mahiatlinux deleted the fix/studio-mlx-api-model-discovery branch August 28, 2026 05:26
lonexreb added a commit to lonexreb/unsloth that referenced this pull request Aug 29, 2026
…ually shipped

Two source pins in test_openai_auto_switch.py landed asserting a shape
their own commit then refactored past, and Backend CI has carried them
red since:

- openai_responses and anthropic_messages hoist the image answer into a
  local (_responses_has_image / _anthropic_has_image) so the image
  preflight can reuse it; the pins still expected the guard to call the
  detector inline. Pin both halves: the derivation and the guard
  consuming it. anthropic_count_tokens still passes the call inline and
  keeps its original pin.
- The non-streaming /v1/responses path delegates its preflight to chat,
  whose switch call leaves gguf_only at its False default rather than
  passing it, so captured["gguf_only"] KeyErrors. Read the kwarg with
  the same False default the callee applies; the streaming arm still
  requires the explicit True.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Installed MLX models are missing from /v1/models and cannot be loaded by API model auto-switch

4 participants