{{ message }}
Repository navigation
Conversation
…code cells See docs/jit-design.md and docs/jit-implementation-plan.md.
See docs/jit-implementation-plan.md and docs/jit-m1.md.
See docs/jit-implementation-plan.md and docs/jit-m2.md.
See docs/jit-implementation-plan.md and docs/jit-m3.md.
See docs/jit-implementation-plan.md and docs/jit-m4.md.
See docs/jit-implementation-plan.md and docs/jit-m5.md.
Tests and a benchmark against Data.Sequence are in lib/unison-util-rope. See the list representation section of docs/jit-progress.md.
See docs/jit-implementation-plan.md and docs/jit-m6.md.
Lists, Text and Bytes on strict finger trees (Unison.Util.Deque, a new strict finger tree with the Deque tests and benchmark run against it, and Unison.Util.Rope on the same tree) with every list, text and bytes primitive native and no fallback to the interpreter; the rest of Text and Bytes (numbers, pack/unpack, indexOf, orderings, the pure foreign functions); all of Int, Nat and Float, pow and the representation casts, bit for bit GHC's results; every array, Ref and Ticket builtin; Universal.murmurHashUntyped. Inline bump allocation with the budget folded into the limit. Private copies of compiled callees and re-entry functions compiled in batches. Native code checks pointer tags before storing into strict fields. The generator is strict throughout: Unison.Util.Text, Deque, strict pairs and fields. A memory leak (a lazy memo update keeping every module's IR alive) and a shutdown race (LLVM's destructors under a compile) fixed. The JIT docs move to docs/jit/, with m7.md the milestone writeup and its closing benchmark tables, and builtins.md the checklist of native builtins.
# Conflicts: # unison-runtime/unison-runtime.cabal
LLVM is no longer linked. jit_llvm.c loads libLLVM (20 or newer) with dlopen when the JIT is turned on, resolving every function with dlsym, so building needs no LLVM and the jit package flag is gone. ucm loads it at startup under --jit on|eager and prints whether the JIT is activated or where it looked; without LLVM everything runs on the interpreter.
…rst one formBatch grows the batch one neighbour at a time. UNISON_JIT_BATCH_RULE picks the rule: breadth (today's walk and gates) or weighted (by the estimated calls across the edge to the batch, gated at a quarter of the threshold). A benchmark where the callee is first reached after its caller is hot runs 3x faster under weighted. Also records the batch's candidates and their counts under UNISON_JIT_LOG, and the tag-store and warm-definitions notes in optimization-ideas.
…uest A request carries its time and the log reports how long it waited: about 40 us with the compile thread idle, but up to 1.2 ms when it shared the interpreter's capability, since it then ran only at that capability's next scheduling point. forkOn puts it elsewhere, and jitRequestGroup yields after queuing so a runnable thread is handed to an idle capability either way.
…tion in the stats
…ES, UNISON_JIT_CODEGEN_LEVEL; what each costs and buys
…_frame), the live slots as its arguments; UNISON_JIT_EXITS=blocks keeps the generated blocks
…record (the settings, the exits and frames tables, the constant pool, the compile thread's queue and the compiler's memos and counters); ucm makes it at startup with newJIT from readConfig, gives it to every runtime it starts (startRuntime, withRuntime and runStandalone take it), and the code cache carries it in a jit field, which is how the interpreter reaches it. Every unsafePerformIO IORef in JIT.hs, Compile.hs, Exits.hs, Frames.hs and Pool.hs is gone, and so is the config CAF; the one process-wide setting left is the replacements switch, which the value decoder and the foreign function table consult without a handle in reach. The code synonyms (MSection and company) moved to Stack.hs so the JIT modules need not import the cache module. The trampoline reads the three tables once per entry rather than per round trip, which the 16-field record row (one round trip per iteration) showed to matter: benchmarks/2026-10-19-jit-handle.md, every row within noise of 2026-10-17.
…debase is a release of unisonweb/testing-codebases (jit-2026-10-19), and scripts/testing-codebase.sh jit prints its directory, downloading and unpacking it under $TMPDIR/unison-testing-codebases/<sha256> only when that directory is missing. The transcripts' headers and development.md point at it; the test matrix script's default is a fresh codebase (its transcript needs only builtins) unless JIT_CODEBASE names one.
…ays MutableArray (PrimState IO) like Stack.hs does; MCode.hs's cell comment gives the C struct view of the 32 bytes and says who reads them; Stack.alloc's comment explains the out-area; Unison.Util.Skews, the unused first attempt at the list structure, is deleted (optimization-ideas.md says where it went).
…e code, startup, the addresses the JIT defines) and the file in those three sections, with the two address tables read by one function; the startup checks' probe calls, which Layout.hs alone uses, are a module of their own, Native/Probes.hs, whose header explains the init/check/test protocol every helper family follows. development.md's test command forks the testing codebase too, since forking a home codebase that awaits migration fails.
…e trampoline, the handler environment, the probes, the exits) and the native builtins are one file per family: jit_builtins_math.c, _lists.c, _ropes.c, _text.c, _bytes.c, _arrays.c, _compare.c and _records.c, with what they share declared in jit_builtins.h. jit_rt.c includes them, so the C runtime stays one translation unit: compiled as separate units the rows whose helpers cross from the ropes into the list internals and from records into the array allocator lost 4 to 7% (benchmarks/2026-10-19-c-split.md); as one unit every row is within noise. The included files are tracked by the package, and a touch of jit_rt.c after editing one makes Cabal recompile it (development.md). The trace flag is jit_trace, since it is shared across the files.
….Internal returns ordinary pairs and Ints (its 72 benchmark rows are within 3% of before, GHC unboxes those results itself), Text.joinChunks is written with Data.Primitive and the one primop wrapper Bytes.casUsed, and only Rope.chunkAt and lookR keep their unboxed pairs, the one place that measured 5 to 8% slower boxed on taking a text apart a character at a time (benchmarks/2026-10-19-unboxing.md). parser-typechecker depends on primitive for it.
…ts.sh and transcripts.sh, both passing).
… exit from a frame of size 0, as in jit-tests.md's unison_720) stored no words into the argument block, so gsMaxExitArgs stayed 0 and the entry block never allocated the block the call names, and LLVM rejected the module (the function was then left interpreted). Every site now claims at least one word. jit-tests.md passes under off, on and eager with identical output, and the module compiles.
…n its Lets (each Let body and each deferred function inlines the whole suffix; eager mode compiles all of them up front) and a preemption counts as a miss at every Let on the stack, so on mode compiles bodies that nothing structural exits at; the fix for each is sketched (chain re-entry functions through the next cell; build the Push frames of a Reenter or GrowStack unwind with a mark that leap does not count).
…owStack exit in a callee unwound every native caller and each caller's Let counted a miss when its frame was popped, so in on mode a long-running function was in time interrupted at every Let it has and got its suffix compiled after each (quadratic in its Lets, for no structural reason). The trampoline now looks the exit up before it builds the frames (Exits.isAdHoc): for an ad hoc exit through a Let whose body has no code the Push frame carries no cell, so the pop interprets without counting, and JIT.noteAdHocExit counts the miss instead, within a per-group budget of UNISON_JIT_REENTRY_ADHOC (3) re-entry functions; the Lets most often on the stack at a tick, where the time goes, are the ones that get in. UNISON_JIT_STATS prints what the ad hoc exits asked for and how many found their group at the cap. The matrix gains the row on, threshold 1, poll=3 (all 19 rows pass), the script exits 0 when every row passes (rmdir of the non-empty temp dir was its exit status), and the suite on the optimized build is within noise (benchmarks/2026-10-20-adhoc-reentry-cap.md). design.md, usage.md, development.md and optimization-ideas.md updated.
… only way left to the quadratic amount of re-entry code in on mode is the deferred copies after slow-path call-outs, which needs the slow path taken often at many sites of one function and is not expected to happen; eager mode pays it by design and structural Let exits are bounded by the estimate. The linear-code idea stays, marked low priority.
…hs header and two jit_rt.c comments, and design.md's worked example is a dump of today's code: the worker shown is the fast entry with the full worker inlined, the write-back blocks are the argument-block stores and a call of the write-back routine with the site's descriptor rather than the wrapper's stores, and the numbers are the dump's. The wrapper listing was still right.
…s/proofs/tests.sh, transcripts.sh --force, formatting.sh, all passing; the first transcripts run failed only on a network error installing base from Share).
…ll-out, and the cloud library's local Remote runtime does that three times per Ref.modify, which is why the broad suite's Remote.Ref row is slower compiled than interpreted (measured 2026-10-20); the 2026-10-16 figure for it is interpreter against interpreter, since the loop ends before the code is installed.
… pairs as C arguments), not the generator's alone; the way to lift it is an argument block on the C stack, as the write-back passes its live slots.
…n combinator is built in place (Codegen.genNameInline: the pool's constant for the combinator copied into one chunk with two fresh segment arrays, as a wide constructor is built), and one of a function value on the stack goes through unison_jit_name with its arguments on the Unison stack, in the scratch slots above the frame's depth, with a base index and a count instead of four pairs as C arguments. The startup probe checks the helper with up to seven arguments. jit-tests.md gains a section with every arity from 2 to 6 and every number of arguments supplied, to the known function, to it as a value and one at a time, on functions that return their arguments as digits so a wrong order shows; the matrix passes (19 rows). The broad suite's Remote.Ref row, which took 30,002 call-outs at three Name sites of the cloud library's local runtime, goes from a third slower than interpreted to two to three times faster, and the suite's three ability-request rows gain 20%, a handler building its next state with a Name per request (benchmarks/2026-10-20-name.md).
…tchClosure switches over every data shape, then loads the packed tag and switches again; the arities fix the shape, so the switch can cover only the shapes the type has and go straight to the arm when a shape has one constructor, loading the packed tag only where two or more share a shape. A note on why the uniform two-array representation with the constructor number in the Haskell constructor isn't the way: the tag load overlaps the field loads, two arrays put a dependent load in front of every field read.
…itches over every data shape and then loads the packed tag for a second switch. The constructor arities fix each constructor's shape, so the switch covers only the shapes the type has a constructor of, a shape with one constructor branches straight to its arm, and only a shape two or more constructors share loads the packed tag and switches among those; anything else exits. A match on a two-constructor type with a three-field node is one compare of the tag bits and one compare of the info pointer where it was four branches and two loads. Binary tree lookups 0.42x, inserts 0.88x, Map.lookup 25 ns and Map.insert 102 ns on the broad suite's rows, the rest of the suite within noise; the matrix passes (19 rows). genRMatch is unchanged since it needs the packed tag for the ability number (benchmarks/2026-10-21-match-dispatch.md).
… (UNISON_JIT=on, threshold 5, in a fresh codebase, since the transcript uses only builtins) and diffs the output, so a change that breaks native code fails the local proof instead of waiting for the CI jit job; a machine where the JIT doesn't activate or compiles nothing gets a warning, not a failure. A --only prefix that selects jit-tests.md runs it too. The embedded IR files join the hashed inputs.
…s/check.sh: 22 and 394 tests passing, every transcript, jit-tests.md with the JIT on as well, formatting; migration cached).
pchiusano
marked this pull request as draft
October 21, 2026 14:46
pchiusano
marked this pull request as ready for review
October 21, 2026 17:49
… the merge queue running every transcript. The generator classifies the new MkCallback instruction (pushes one value, reads two slots) and leaves it to the interpreter; Callback.hs takes the branch's sized alloc with the default stack size; test.yaml keeps the jit job and loses the attested-transcripts skip as trunk did; the proofs files in conflict are dropped for a fresh run. The callbacks transcript matches under the interpreter, eager and on; builtins.md lists the three builtins as call-outs.
…rge of trunk (scripts/check.sh: 22 and 394 tests passing, every transcript with jit-tests.md under the JIT as well, formatting; migration cached).
…med (FFI.newCallback, FFI.functionPtr), followed in builtins.md and development.md; the proofs files in conflict are dropped for a fresh run.
…rge of trunk's builtin rename (scripts/check.sh: 22 and 394 tests passing, every transcript with jit-tests.md under the JIT as well, formatting; migration cached).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

This PR adds an LLVM-based JIT to the Unison runtime, enabled with
ucm --jit on(or the environment variableUNISON_JIT=on), see usage.md for details. Built and tested on macOS (Apple silicon and Intel), with a CI smoke test for Linux (x86-64 and arm64) and Windows.It is up to 200x faster than the interpreter for toy programs (a simple arithmetic loop where all the data fits in registers) and more typical code is often 5-10x faster. More numbers below. Nothing in the language semantics is changed by this PR, and the serialized form of code stays the same.
The big idea behind the design is to allow for partial compilation--a native-compiled function can exit back to the interpreter at any point in its execution. This allows the JIT to be useful immediately rather than requiring it to cover the entirety of the Unison language on day 1. Read more in design.md.
The PR is quite big, however the interesting parts of the code are under 10k lines. Beyond that, there are lots of docs, benchmarks and C implementations of various Unison builtins. See the detailed breakdown below. I used Claude + Fable 5.1 for the implementation, notes on my methodology at the end.
Performance highlights
fib 2038×, float arithmetic 30×.murmurHash25× faster. See builtins.mdAlmost everything is equal or faster according to my benchmarks, with a few exceptions discussed below in the full benchmark table. The places where the interpreter wins are where it "cheats" by using optimized replacements backed by Haskell implementations (for instance, bulk
Mapconstruction of a large map), and I think we can close that gap and make those optimized replacements unnecessary. As is, even though GHC is a little faster for straight-line pattern matching code (like what's used for a bulkMapconstruction), native compiled Unison can still win by avoiding the cost of exiting back to the interpreter.(See Optimization ideas for a somewhat disorganized collection of optimization ideas for later.)
Performance against trunk
The Improvement column means: what to divide the trunk time to get the JIT time for executing the same code. So an improvement of 2× means the JIT took 1/2 the time as the trunk interpreter. Higher improvements are better. Anything less than 1× means the JIT took more time than the trunk interpreter.
The JIT's suite
See
unison-src/transcripts-manual/jit-benchmarks.mdfor source. This was a custom benchmark suite built to exercise different parts of the JIT. Trunk's column is from 2026-10-16, the JIT's from 2026-10-21 (the run).--jit onThe broad suite
See
jit-suite.md, the@pchiusano/misc-benchmarkssuite.--jit onThe rows below 1×:
Set.fromListandMap.fromList(0.28× and 0.39×): on trunk these are HaskellData.Mapcall-outs. Under the JIT theMapfamily runs as compiled Unison code, which wins onlookupandinsert(7.9× and 2.9× above) and on maps of the sizes programs commonly use, but loses on bulk construction and above a few thousand keys.UNISON_REPLACEMENTS=onbrings the Haskell maps back (see theMapcaveat and these measurements).List.rangeper element (0.67×) are not losses; trunk's times reflect accidental laziness in the interpreter. On trunk, the parser andList.rangebuild aData.Sequence, which is lazy in its elements, so a JSON value's children and a range's numbers stay thunks when the result is ignored, as is the case in these benchmarks. The branch's list is strict (native code reads it directly), so it's doing the full work. This was figured out with some more careful benchmarking.Multimap.fromList(0.72×) is another bulkMapoperation with an optimized Haskell replacementWhat changed in the interpreter
The branch doesn't make any major changes to the interpreter, but there were a few things, some for performance, and some to ensure that native code wouldn't ever be given Haskell thunks. Native code now has to pattern match directly on Haskell data types, and any thunks cause it to exit back to the interpreter. Notable differences on the interpreter vs trunk:
GClosurecarry GHC's pointer tags, and texts, lists and bytes are closure constructors of their own rather thanForeignwrappers (Layouts).Deque) andTextandBytesropes over it, inlib/unison-util-rope, with their own tests and benchmarks.GSection,GInstr,Prim1,Prim2,Foreign) are ordered by measured dispatch frequency, so that the hot constructors have pointer tags.BytesandTextuses a copy-on-write slack buffer, making appends faster while still preserving a functional interface.run.compiled) use the JIT (ifUNISON_JIT=on), and the.ucfile carries the constructor arities of the data types the program uses, since the JIT needs this info to compile a match.Intforeign conventions tagged their resultsNat, andUniversal.compareordered every negative float above0.0(bit 64 cleared instead of bit 63). Both are fixed in the interpreter and in native code.An overview of the diff
This is a big PR.
git diff --numstatfrom the merge base withtrunk(cea754e16), 2026-10-18: 114 files,34,655 lines added, 905 removed, 35,560 in all. Lines below are added plus removed.
cbits/jit_rt.c748 (the machinery) and the eightjit_builtins_*.cfiles 3,113 (one per family: lists 979, ropes 838, text 646, arrays 239, bytes 177, records 110, compare 89, math 35) withjit_builtins.h400,jit_helpers.c756,jit_llvm.c562,jit_exit.c83, the other headers 401, the generatedjit_helpers.ll3,939 andjit_exit.ll155 (the IR of the two C files, embedded), and the shell script that generates themRuntime/JIT.hs,Runtime/JIT/*.hsandcbits/gen-helpers.hs, the IR stripperlib/unison-util-rope:Deque, theRoperework, tests and benchmarksMachine.hs+498 -246,Stack.hs+411 -185,MCode.hs+373 -81,Bytes.hs+117 -35,Foreign/Function.hs+106 -65,Interface.hs+94 -25,Util/Text.hs+91 -20 and its tests +121, nine others under 40 lines eachjitjob on five platforms) andscripts/jit-test-matrix.shBy role, counting the new code:
Codegen.hs5,134,Strict.hs,Helpers.hsjit_rt.c3,200 of its sections,jit_helpers.c,jit_internal.h,Native.hsLayout.hs,Native/Probes.hsJIT.hs,Compile.hs,Config.hs,Estimate.hs,LLVM.hs,jit_llvm.cExits.hs,Frames.hs,Pool.hs,jit_rt.h,jit_exit.c,jit_rt.c900, and the modified interpreter moduleslib/unison-util-ropeThe compiler proper (generation, layouts, driver, trampoline) is about 13,200 lines; the native
builtins and the data-structure rework 8,100; tests and docs 9,900.
Notes on development methodology
I wrote almost none of the code in this PR by hand. Here was the process I used:
After it was all working, there was further bunch of back and forth and steering to improve the design and implement further optimizations. My goal was to meet or exceed the interpreter across the board. Initial results were slower than the interpreter on many benchmarks.
I also did a few rounds of code review which led to some more fixes and improvements. I didn't spend much time looking at the code before everything seemed to be working.
Random observations:
TextandBytes), then ported the implementation to C for use by native-compiled Unison code.unsafePerformIOthe declaration of a bunch ofIORefvariables for the JIT state. There was no performance reason for it, it was just strange. It also wrote some bizarre code that was manually using unboxed tuples and ints, when GHCs optimizer did just fine with the code is written in a normal way. It also at one point wrote some newbie-looking code that was like 8 levels of nestedEitherpattern matching instead of using a monad...Overall, I really enjoyed the process and felt like I learned a lot about how to get more mileage out of the tools. The work still felt creative and interesting, even though I wasn't writing every line. Still, like many programmers, I have mixed feelings about it.
What's next?
There are further optimization ideas in
optimization-ideas.md, but I think this branch is in a decent spot for people to be able to try it out. It should be low risk since it's behind a flag and it doesn't require LLVM to be present, it will just fall back to the interpreter in that case.Later, we can look at making the JIT the default.