Something that might be principled / worth doing down the line is to Pip-package our LLVM binaries so they can be listed as a build-time dependency of our wheels. We could host them from the build master. Unfortunately, they are far too large to be hosted on PyPI. We might be able to further shrink the size by truncating un-needed files in bin/ (e.g. llvm-objdump), but we cannot simple exclude or delete them as LLVM's CMake package will complain about their absence.

Doing the same with wabt / flatbuffers might be profitable too. These are small enough to upload to PyPI, but I suspect that upstreams wouldn't appreciate us stealing those names. We'd have to coordinate with them or host from a private registry.

Something that took way longer to diagnose than it should have and that I'm saving here for posterity (and perhaps the search results of other unfortunate souls):

cibuildwheel allows you to set environment variables for Windows via the CIBW_ENVIRONMENT_WINDOWS environment variable. I had written in my YAML config:

env:
  CIBW_ENVIRONMENT_WINDOWS: >
    CMAKE_GENERATOR=Ninja
    CMAKE_PREFIX_PATH="${{ github.workspace }}/opt"

Woe on me for using double quotes! The string is parsed as if it were a bash command and ${{ github.workspace }} expands to D:\a\Halide\Halide so these backslashes become escape sequences! When using single quotes, the backslashes are literal.

I needed to SSH into the runner using mxschmitt/action-tmate@v3 to diagnose this.

Supporting 32-bit targets: This is largely untested anyway.

IIRC we explicitly state that Python binding for Halide are only supported for 64-bit targets. IMHO it's not worth supporting 32-bit targets at this point in history.

Summoning the build bots for full review.

Failures are due to linux-worker-4 issues:

• missing Python package dependencies (c.f. Create Python virtual environment as part of build build_bot#129)
• missing native PNG libs
• missing -lz
• disconnections

Landing to confirm Test PyPI upload works. Docs to come.

Fantastic! Thanks for doing this 🎉 I am looking forward to be able to ‘pip install halide’ 😊

Did I understand correctly that the default wheels will support WebAssembly as the target? If so, that’s awesome for integrating Halide into Jupyter notebooks.

Fantastic! Thanks for doing this 🎉 I am looking forward to be able to ‘pip install halide’ 😊

Thank you for the kind words, @dragly! 😊

You can try it right now from the Test PyPI index:

$ pip install halide --pre --extra-index-url https://test.pypi.org/simple

When we release Halide 19.0.0 in a few weeks, then plain old pip install halide will work.

Did I understand correctly that the default wheels will support WebAssembly as the target? If so, that’s awesome for integrating Halide into Jupyter notebooks.

Yes. WebAssembly is now a required backend (along with X86). See #8344

I'm curious to learn more about the Jupyter notebook usage. Do you have an example of using Halide's WebAssembly backend in that context?

When we release Halide 19.0.0 in a few weeks

We probably won't release a Halide 19 until LLVM 19 is final, I don't think there's a date for that yet, but it is in RC so hopefully not too long.

We probably won't release a Halide 19 until LLVM 19 is final, I don't think there's a date for that yet, but it is in RC so hopefully not too long.

19.1.0 final is expected on the 17th. Unless we have major blockers, our release shouldn't take long after that.

https://discourse.llvm.org/t/llvm-19-1-0-rc4-released/81039

You can try it right now from the Test PyPI index

Very nice! Worked right out of the box here.

I'm curious to learn more about the Jupyter notebook usage. Do you have an example of using Halide's WebAssembly backend in that context?

I have been playing around with a wrapper around IPython's display function for Halide objects. The goal is to make an interactive viewer where you can adjust the Param values using HTML sliders or other controls.

You can get pretty far on something like this without WebAssembly as well, but then you need a running Jupyter kernel.

With WebAssembly, you can include interactive viewers in an HTML version of a notebook and it will just keep working with static hosting. I have for instance been using Jupyter Book both for some personal projects and for internal documentation of our Halide code at work. So I am hoping to include some interactive visualizations there too.

The work-in-progress API looks something like this:

factor = hl.Param(hl.UInt(8), "factor", 0)
darken = hl.Func("darken")
darken[x, y, c] = image[x, y, c] / factor
slider = Slider(factor, min_value=1, max_value=10, step=1)
display_halide(darken, controls=[slider])

Which then results in output looking something like this:

In the implementation I am compiling the Halide Func to WebAssembly and then invoking Emscripten to link it and generate the final .js and .wasm files. These are then loaded by a <script> tag that I display together with some JavaScript glue code. And that code is rendered together with some HTML using IPython's display function.

The code is currently a mess. I first wanted to see if making such a viewer was at all doable. However, now that it has become so easy to include Halide in a Python project, I will probably write up a post about it and share it in the near future :)

@dragly -- This is super cool! Hope I get to play with it soon 🙂

@alexreinking I wrote up a post about it here: https://dragly.org/2024/09/08/interactive-halide/

I figured I might as well host it as a package now that Halide is coming to PyPI 🎉 The code is still very hacky, though. You should expect it to fail even with simple examples. But it should be usable with examples like the ones in the post.

And you need to have Emscripten installed and in PATH for it to work. If only there was a PyPI package for Emscripten too 😄