{{ message }}
Repository navigation
What two runs of the same four filings agree on - #769
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: WaylandYang <wayland0916@gmail.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Two runs of the same four filings, one configuration, so that the remaining open marks in the
extraction section of prior-work are either answered or waiting on
something nameable.
What a run costs when nothing changes
Same four NVDA filings, same 14 classes and 28 properties, same 55 chunks, extracted twice against
the same endpoint with the same contract:
A run is worth about seven per cent of its statements and half a point of misworded. The judge's
own spread is 1.4 points on one graph judged twice, so the two together are the band a claim has to
clear before it means anything. The reasoning cap measured in #759 stays rejected on this reading:
729 statements and 16% of table figures lost are outside both bands, not near them.
This closes the "run-to-run stability number" half of item 9.
The second look at figures is measured, and not built
Item 10 has been waiting on the second-look pass of item 5, for the figures a dense sentence drops.
The four filings are the densest text in the corpora, 25 figures per thousand characters against 7
for the next one, and
figures.mjsreads no model: it counts every figure a passage writes, againstevery statement drawn from that passage, table rows apart from prose.
Over the two runs, prose loses 0 and 1 figure of 35, tables 7 and 18 of 451. A second pass would
chase a dozen table cells at the price of a second call on every chunk, where a chunk already costs
150 seconds. It is not built, and the item now says what would change that: a corpus where prose
figures are lost often enough for the pass to move the number.
Two things the bench sections did not say
figures.mjscounts a figure only when it carries a currency symbol, a percent sign, a unit word ora thousands separator, so a bare digit cannot be matched by accident; and it counts a figure as
reached when it appears in a statement drawn from its own passage, including that statement's
qualifiers and time mentions, which is where a period or an amount often lands.
The judge counts a statement once per piece of evidence, so its denominator is statement-evidence
pairs rather than statements: 845 against 824 on the second run. The same on both sides of any
comparison, but worth writing down before someone reconciles two numbers that were never the same
number.
Also
The extraction page listed alignment producing typed facts and time interpretation as proposed and
not built. Both are built, in 0044 cut 2 and 0045 cuts 1 to 3. Identity profiles and the layer
marker in the interface stay on that list.
🤖 Generated with Claude Code