Matching
Alchemy's C has to compile, with an old
compiler, into exactly the instructions
Camelot shipped. Two ways of writing the same thing
in C can compile to different instructions, so where a line of the source looks
stranger than it needs to be, the reason is usually written down here. The
glossary says what a
matching decompilation is and what
the twelve editions are; measurements on this page
are taken in tbs-en, the English edition, unless they say otherwise.
How a build is judged
make compare-editions builds all twelve editions and checks each ROM's
SHA-1 hash against its line in rom.sha1 (the compare-% rule in the
Makefile). Each edition that matches prints a line ending in : OK, and
twelve of them is a pass. The output is long, so it's easiest to send it to a
log file and count those lines. make compare-tbs checks the six Broken Seal
editions on their own, and make compare-tbs-en just the one.
The reference ROM judges the build and never feeds it (The Lost Age's
scaffolds, below, aside), so the comparison is a hash of the whole image and
is all or nothing. One wrong bit anywhere gives a completely different hash,
which says nothing about where the difference is. To find it, compare the
compiler's output before and after your change. The build keeps each C file's
assembly beside its object (out/
for system.c): copy it from a build that compared OK, rebuild, and diff the
two with the .L label numbers ignored. The optional alchemy-devtools diff
instead compares one function with the original's instructions read from your
own ROM; it is a tool for people and never part of the build.
The Broken Seal's editions need no ROM at all. The Lost Age's need your own
ROMs under roms/, because the parts of The Lost Age that aren't C yet still
come from .incbin "baserom. scaffolds under recon/tla: bytes copied
from the ROM, which don't count as decompiled.
A change made on purpose, such as a mod or a fix for one of
the original's bugs, changes the bytes, so it
fails the comparison. That's expected: the comparison only tells you whether a
tree is still the original game. So that kind of work lives apart from the
matching tree, or behind BUGFIX,
which is off.
What the markers mean
FAKEMATCH
Sometimes the natural way to write a piece of code compiles to slightly different instructions from the original's, and no amount of reshaping finds a plain form that matches. Then the code uses a device: something that changes how the compiler lays the code out without changing what the code does. Every device carries a tag inside the function it steers:
/* FAKEMATCH(kind): the form kept; as <plain form>, N of M instructions
* differ in tbs-en. */The tag names the plain form and says how far it is from matching: N of the
function's M instructions differ, measured over the whole function against the
edition named. A real one, from wait(), the routine
that lets a frame pass:
/* FAKEMATCH(volatile): the size reread through a volatile pointer, a
* load apart from the test's; as *size, 2 of 403 instructions differ in
* tbs-en. */
base -= *(volatile u32 *)size;The kind says which device it is:
wrapperis a call made through a helper, a cast or a library routine in place of the plain call or operator. Calling a declared function through a cast to another function type is awrapper, with the declared call behindUBFIX. Calling code that has no declared function type at all, such as code copied into RAM, is not (see Calls below).volatileis a qualifier that forces a load or store to stay where it's written. One that is part of a macro's own form, such as the fill word ofDma_Fill32, is no device. Nor is one on memory an interrupt handler shares with the code it interrupts, such asgVBlankDoneandgFrameCount(include/): there the C needs it to be correct.system/ frame.h temporaryis a variable holding a value the plain form wouldn't name.deadis code that emits nothing and only changes which registers the compiler picks or the order of the instructions.one-passis a block or loop that runs once (do { } while (0)). GCC marks every C loop with internal loop notes, and those notes end the stretch in which its common subexpression pass (CSE) reuses values it has already computed, and stop its scheduler moving instructions across them.barrieris an emptyasm. It emits nothing, but counts as four bytes when the compiler places literal pools, the blocks of constants it puts among a function's code for the function to load.pinis a variable fixed to a register.signatureis a function declared with a return or parameter type it doesn't have, such as avoidfunction typedint.
A device is the last resort, used only after reshaping the code has failed.
Taking one out again is part of humanizing, the project's second stage after
matching, which renames, types and explains the matched C until it reads
like ordinary C. Some older tags in The Lost Age's code have no kind yet
(/* FAKEMATCH: ... */); they mean the same thing.
The other markers
// DEAD: why marks code the ROM keeps that can't change the result, such as
a test that is always true or an initializer that every pass repeats
(Task_Sort). It's only used where the construct's own compare,
branch, load or loop is there in the ROM, which makes it the source's form
rather than a device.
// BUG: what goes wrong marks a bug of the original game, kept as it is.
Where there's a fix, it sits behind BUGFIX (games/),
which is off. // UB: marks undefined behaviour kept in the form that
matches; the defined form is behind UBFIX, which make modern and BUGFIX
both turn on.
CAMELOT_ marks the few handwritten instructions inside C, each
an asm statement right after it whose instruction no C compiles to:
Dma_Set, Transform_, Iwram_, wait()'s stack
switch, MulQ16, and the swi 2 and swi 3 BIOS calls. The macro expands to
nothing; it is there to be read, and its proof says why the asm is needed.
UNUSED marks a function nothing calls, kept
because its bytes are in the ROM. It's GCC's unused attribute, so a current
GCC doesn't warn about it.
An assembly file in games/ has to show why it is assembly at all: that no
C compiler of the project's set could produce it. Each one opens with that
proof, written as comment lines (@ starts a comment in assembly) that begin
@ Why assembly:, or @ Why <routine> is assembly: where a file holds
several; assembly counts as rebuilt source only with such a proof. In a
handwritten file the proof names the instruction, stack frame or pattern
alchemy-gcc can't emit. The
far-call tables, such as the entry/ and
import/ files of the debug scenes, the menus and the
scenes, are proven once, where their entries are
made: each entry is a veneer, which the
overlay_ macro in gamepak/ writes out whole.
Spellings that match at different sites
The original was compiled from one particular spelling of each line, so where the plain spelling gives different instructions, the matching C keeps the one that gives the original's.
Each idea has one preferred spelling. Where a site matches only with a plain-C
equivalent (an index or a pointer form, a macro or its expansion, a cast), that
site keeps the equivalent, untagged, with a short // note beside it saying
what the plain spelling would change. The patterns below are the ones that
recur, each with one example; the sites themselves carry the rest. They use the
system's words: an Actor is the struct an actor is
read through.
Indexes, pointers and casts
p[n]and*(p + n)can compile differently, and so can&p[n]and&*(p + n).TilesetAnim_writesUpdate (gridState->, where the index form changes 78 of 119 instructions in tbs-en.animations + channelIndex)-> - A member can be reached through a pointer to its own type, or to its own
size with the other signedness (
*(T *)&p->, ormember *&p->member), instead of asp->member. These are valid aliases, kept where the member form would move the access or merge it with a neighbour's: inBattle_each status test written asRollEffectSuccess unit->fieldmerges with its neighbour's into one halfword or word load. - A field can be read or written by its byte offset,
*(T *)((u8 *)p + offsetof(struct S, field)), or a run of fields through a byte cursor indexed withsizeofandoffsetof. These views are valid and keep the original's address bases:Colosso_stores two bytes of the actor system's work that way, because a struct store at a varying address may pass a load the ROM keeps after it. ThroughRestoreWarrior Actor's single/group union, the plain member form usually moves a load. A pointer can likewise be stepped as an integer: a window's tilemap cell iswork->or a byte offset added totilemap + y * 32 + x (u32)work->.tilemap - A function that moves a point's x and z takes a byte address and steps over
y, so it works on any struct whose x, y and z sit together without treating
them as an array (
Vector_).AddPolarOffset - A value can be reached through a view of another type. That is undefined
behaviour, marked
// UB:, with the declared access behindUBFIX:WindowSprite_stores the new tail through aAppendToList struct WindowSprite **view of the list's tail link, which keeps both stores in order. - A table read by columns is a
[][N]array, as alchemy-gcc addresses a struct's later fields from the table's base: an element pair table iss16 [4][2]where the matching code indexes it, withstruct BattleElementStatsbehindUBFIX. - Palette RAM and
VRAMare written through either a volatile pointer or a plain one. Palette RAM is eitherBG_, which is volatile, or a plain pointer to it such asPLTT_ COLOR *(u16 *)&BG_.PLTT_ COLOR(WINDOW_ PALETTE, WINDOW_ BACKGROUND_ COLOR) BattleEffect_'s brightening ramp keeps aRunJudgment volatile u16 *, because with a plain pointer the mask that picks out red is hoisted out of the loop. - A cast to a narrower type can be part of the spelling:
(u16)~(...)makes the halfword mask the ROM holds (MapGrid_).SetShownLayers
Views of one struct
Some memory is read through a struct of its own, a field-for-field copy of
the layout that differs only where its reader does, because that reader's
code matches only through it. An actor's 112 bytes
are struct Actor almost everywhere, but BattleObject_
reads them as BattleActor, with the sprite word untyped; actor_,
ARM code built with GCC 2.95.2, which cannot read Actor's anonymous
unions, keeps ActorArmView; and the operand handlers of SC_SET,
SC_ADD and SC_CMP keep ScriptOperands. The window sprites
(struct WindowSpriteCellView), the selection records
(struct GlyphCursor), the menu cursors (union MenuCursorRef) and each
menu screen's work (ItemMenuWork, StatusScreenWork and the others) follow
the same rule.
Within struct Actor, a member's union is the form most sites match with, and
the few that match only with the plain word reach it through a pointer of the
member's own type (*(s32 *)&actor->, *(s32 *)&actor->):
alchemy-gcc gives an access through a union the union's alias set, which
conflicts with more stores than the plain word's does. Exit_
puts the leader at its cell's middle that way; through x.fixed the store of
the sprite's flags moves. A pointer to an actor kept in memory is likewise a
component of struct Actor itself, so an array of them waits for any store to
an actor: FieldPsynergy_ keeps its two hands as void *.
Macros and their expansions
- A macro and its expansion can compile differently.
Serial_writesInitialize IRQ_LOCKout as its steps, with the IME pointer set inside the save; asIRQ_LOCK, 1 of 123 instructions differs. Other sites keep arithmetic onBLEND_ALPHArather than its|form, its operands written out, or a signed halving as its three steps. - An inline and a direct call of the routine it wraps can compile
differently, because each inline argument is a pseudo of its own and a
constant two calls share is loaded again. An event command therefore keeps
both spellings where its sites need both: events call
Actor_, the inline inFaceDirection event.h, and the sites that match only with the direct call name its routine,Actor_; folding that inline into its routine changes 316 functions in tbs-en. Where an inline and its routine would share a name, the inline takes theArmCallback FieldActor_orFieldEvent_prefix (FieldEvent_), and where folding changes no function the two are one name (ShowMessageThenWait Actor_Stop,Event_Wait). - A statement macro takes the shape all its uses match with. In
BattleEffect_'s second walk,BuildBlitter PUT,COPYandPATCHaredo { } while (0)blocks and the others plain statements or a block. - A register write is either
Io_, which builds the value in a word first and keeps the store where it's written, or plainWrite16(value, ®_ X) REG_(X = value WorldMap_uses both).InitializePerspective - A DMA control word is written with its flags above its count:
(DMA_, orENABLE | DMA_ 32BIT) << 16 | words DMA_.ENABLE << 16 | halfwords - A message that follows a named one is written as
MsgName + n, such asMsgSaveDataCorrupted + 2forMsgSaveRecoveryFailed, where the ROM adds to the register already holding the first.
Statements and variables
- A random value is masked in place, with the mask assigned first
(
speed = 0x3ff; speed &= Random16();). Written asRandom16() & 0x3ff, the mask goes into a scratch register and the result is copied before theand, so 3 instructions differ at each site (LuckyWheels_and several battle effects).Run - Where a statement sits and which variable holds a value decide the
schedule and the registers:
Glyph_writesDrawArm nextRow = row + 1in both arms of a test rather than once ahead of it, andMessage_counts letters in itsDecode s32value rather than iniorj. - A global's member is reached through a local pointer to the global or
through the global's own name, each where it matches: Crossbone Isle's
CrossboneIsle_takesMovePillar &state->withsceneId state = &gMainRoutineWork, because&gMainRoutineWork.folds into one literal.sceneId - A loop can be written with
goto. A C loop's notes let GCC's loop pass (loop.cin its source) move constants and addresses out of the loop, strength-reduce its counters and delete it when it is empty, and they let its jump pass (jump.c) copy its exit test to its top; register allocation also counts the references inside a C loop as more frequent. A loop built from labels andgotogets none of this, so where the ROM shows none of it the loop stays agoto, and the comment beside it gives the measured cost of its loop form. Thegotos without such a comment (the wander retries, the step searches, the scenes' event loops and the debug loops) stay until someone finds their loops. - A function whose result nothing reads keeps its
s32return wherevoidwould change its epilogue, the instructions that end it;UBFIXgives thevoidform. An update routine that the bytes show returning a value does the same, with// UB:above it.
Calls
- One routine can be reached by several routes, each kept where it matches: a
game flag by
GameFlag_Get,GameFlag_SetandGameFlag_or byClear event.h's inlinesEvent_,GetFlag Event_andSetFlag Event_; a unit byClearFlag Unit_Getor, from other banks, the far callUnit_GetFar; a unit copy byIwram_, throughCopyWords _call_via_r3, or throughItem_(tagged). An inline call's argument is a pseudo of its own, so a flag number used twice is loaded twice; called directly, the number is kept in a callee-saved register (Bilibin's chef,CopyWords CottageRoomsC_).TalkToBusyChef - Some calls compile only against an old-style declaration or before any
declaration: with the exact prototype in scope the arguments are set up in
another order. Helpers in Colosso and in the Kolima Forest floodgate's
overlay are called ahead of their definitions, with the correct
declarations in their module headers under
UBFIX, andevent.hgathers the main-image routines the scenes call that way (Import_).GameFlagSet - Code with no function type is called through a cast function pointer. Code
copied to RAM is called through one too, or through
_call_via_r3, the compiler library's call through r3, with the code's address as its fourth argument.System_calls the untyped fixed addressSoftReset Cart_Startthrough a locals32 (*reset)(void). With no declared type to differ from, these casts are nowrapper. - The sound driver, compiled with pret's
old_agbcc, calls its command table through a pointer read from the entry,*(&gMPlayJumpTable[n]); indexing it directly adds an instruction.driver.ckeeps the labelled jumps its matching forms need.
Heap slots
The system keeps each piece of code's work in a block
of the heap whose pointer sits in a numbered
heap slot of gHeapSlots, and many of those
slots also have a name of their own (gFieldWork, gPaletteFadeWork and so
on), so the same block can be reached several ways: HEAP_,
gHeapSlots[n], *(gHeapSlots + n), the slot's own name (gWindowWork[0],
WINDOW_WORK), from a neighbouring slot
(gFieldBlendWork[HEAP_ in
Weather_), through a base pointer (slots[n - base],
EFFECT_SLOT, EFFECT_), or with HeapSlot_, which
returns the existing block. Indexing past a named pointer
((void **)&gBattleCamera) is undefined behaviour, with its defined form
behind UBFIX.
Where data is placed
- A table whose shared extern lacks
constis placed in read-only data with__attribute__((section(".instead (Lunpa'srodata"))) gPushedActor_,DirectionSteps gPushedActor_andFootprintKinds gPushedActor_);FootprintBounds gFont_is placed the same way because aMarkGlyphCodes constextern changes the schedule of its reader. - One table can go by several extern names, given as
__asm__labels, where the ROM keeps a literal for each use (Resource_).LoadTableEntry - A local array of constants compiles to an anonymous read-only constant
copied into the stack frame, and a struct copy of a named table to the same
code; which one a site keeps decides where the data lands
(
ImilFuchinFalls_).SpawnDust
Last updated on
