{{ message }}
Repository navigation
coreml : revisit instructions and model generation scripts #2783
Description
The CoreML creation process has been flaky from the very beginning (#566), mainly because the python packages were not compatible with each other. Things likely have changed since the initial support was added and hopefully are more stable now.
- Update CoreML model conversion/creation instructions
- Add CI workflows to exercise CoreML conversion (can utilize ggml-ci for Apple Silicon nodes)
Activity
furqan4545 commentedon Mar 17, 2025
how to integrate .mlmodelc into my swiftui app? I am stuck here? I want to use coreml support. is there any example that I can follow?
danbev commentedon Mar 17, 2025
The swiftui example currently does not have support for coreml, but the objective-c example does, and it contains a link to how to convert models. Perhaps these could be helpful to implement in swift.
furqan4545 commentedon Mar 19, 2025
I am very beginner in obj C... Would it be possible for you guys to make swift ui example? I think a lot of people would appreciate that because as saw in many other repos people were requesting the same thing.
danbev commentedon Mar 19, 2025
Perhaps we can update the existing whisper.swiftui example similar to whisper.objc. I've created #2907 to track this.
ggerganov commentedon Mar 19, 2025
If I am not mistaken, it might be as simple as adding
WHISPER_COREML=1somewhere in the build settings and providing a CoreML model to the project.danbev commentedon Mar 20, 2025
Nice, I'll give this a try. Thanks
danbev commentedon Mar 20, 2025
Hmm, this was not as simple because of the introduction of the xcframework. We are no longer compiling in XCode but instead using the compiled library from the xcframework. And Core ML support requires the above mentioned macro to be set when compiling. My initial throught was that we could provide builds for Core ML for macos and ios and include them in the xcframework, but that does not work as the selection is based on platform and architecture, so trying to create the xcframework results in:
A library with the identifier 'macos-arm64_x86_64' already exists.I'll need to investigate this a little more to understand what our options are.
ggerganov commentedon Mar 20, 2025
I think we can enable both
WHISPER_COREML=ONandWHISPER_COREML_ALLOW_FALLBACK=ONin the CMake builds and provide only this option in the XCFramework. This way, if at runtime it fails to find a CoreML model, it will fallback to not using it.danbev commentedon Mar 20, 2025
That would be much things much easier. Great, I'll try this out.
danbev commentedon Mar 21, 2025
@furqan4545 We've added an example of using Core ML to the whisper.swiftui example now.
11 remaining items
coreml: fix Whisper to CoreML conversion by disabling SDPA [no ci] (#…
examples : clarify Core ML encoder model usage [no ci]
examples : clarify Core ML encoder model usage [no ci] (#2987)
danbev commentedon Apr 2, 2025
I think this could be considered completed by the following commits:
itsthisjustin commentedon Jul 1, 2025
Reviving an old thread here but can't seem to find documentation on this. What will happen when I run on an Intel mac? Will it just fail completely or will it skip the coreML process?
ggerganov commentedon Jul 2, 2025
Hm, I'm actually not sure. I suspect that it will fail, but I no longer have hardware to test this.
itsthisjustin commentedon Jul 2, 2025
Ha yeah same. What I did for now (and i have no idea if it'll work) is I rename the coreml model so the app can't find it when it loads on Intel
Use CoreML model for better performance.
Use CoreML model for better performance.
QVAC-20557 test[notask]: relpos0-internal sub-probes to pin the Mali …
QVAC-20557 fix[api]: zero-pad matmul contraction dim K on Mali (value…