Written 2026-09-15. This document collects the research that bears on the task reasoning graph (agent-reported goal, question, hypothesis, evidence, experiment, option, decision and work records; hold/follow snapshots; editable note cards; Ask agent; option comparison; agent-reported confidence). It corrects several claims in the earlier short overview, widens the literature, and ends with product implications mapped to current choices. Implementation status lives in docs/investigation-spike.md and docs/reasoning-graph-view-plan.md.
Verification: every item was checked against a primary PDF, publisher abstract, or citing paper. Items marked [partial] rest on abstracts or citing papers because the full text was paywalled. Effect sizes and sample sizes are quoted only where verified.
The research supports one narrow claim strongly and everything else weakly.
- Strong: a plural, evidence-linked list of hypotheses improves debugging outcomes (Alaboudi & LaToza 2020, 2023; Kang et al. 2025). An agent can carry the structuring cost that killed every earlier design-rationale tool (Conklin et al. 2001; Capilla et al. 2016).
- Weak: that a universal editable graph improves architecture, research, or general software-engineering outcomes. The interface studies involve 5 to 24 people in single sessions and mostly measure preference.
- Negative: rigid structure imposed on experienced practitioners. The best-controlled Analysis of Competing Hypotheses (ACH) study is null to negative (Dhami, Belton & Mandel 2019). Experts rarely compare options (Klein et al. 1986).
- Warning: displayed reasoning raises trust without raising accuracy (Bansal et al. 2021; Fernandes et al. 2026). Agent self-reports of success are a known failure category (Tang et al. 2026). Verbalised confidence was poorly calibrated in 2023-era models; post-2025 models are markedly better, especially when grounded by verification tools (section 6).
Perception and outcome diverge throughout this literature. Users prefer graphs, trust shown reasoning and feel faster, while measured accuracy and speed often do not move. Evaluating the feature by preference alone will overstate its value.
The product goal is understanding, not throughput: letting the developer see what is going on, what was going on, and what should happen next in a task an agent is working on, and thereby regaining control and justified confidence. This matches the evidence better than an efficiency goal. Efficiency evidence for graph interfaces is weak and preference-based, while the understanding evidence is old and solid: developers lose their working set on interruption (Ko et al. 2006), the hardest questions about code are intent and rationale (LaToza & Myers 2010), and history is used for personal recall (Lu et al. 2011). Endsley 1995 (Human Factors 37) defines situation awareness as perceiving the current state, comprehending its meaning, and projecting what comes next, which is the same triad. The one caution from the same literature is that felt confidence rises easily without understanding (Bansal 2021; Fernandes 2026), so the feature must be measured on what developers can answer, not on how in control they feel.
The findings split by what they describe, and the two tiers age at different rates.
- Findings about people (design-rationale capture, formality overhead, split attention, hypothesis aids in debugging, explanation-induced over-reliance, automation complacency, expert decision strategy) were stable across decades before language models existed. A more capable agent does not change the developer reading its output. If anything, a more persuasive agent makes the over-reliance results more relevant.
- Findings about models (chain-of-thought faithfulness rates, verbalised confidence calibration, extraction accuracy, sycophancy rates, the METR slowdown, self-report error rates) describe specific 2023 to 2025 models. Treat every such number as stale and re-check it against the current model before a decision rests on it. Section 6 records where 2026 work already reverses earlier results.
Larkin & Simon 1987, Cognitive Science 11. Production-system models of physics and geometry problems. Two representations can be informationally equivalent but not computationally equivalent. Diagrams win only through three mechanisms: locational indexing that removes search, recognition replacing symbolic matching, and perceptual inference. The paper's own caveat: "To be useful a diagram must be constructed to take advantage of these features." Implication: a hypothesis, its evidence and the experiment that tests it must sit adjacent, or an auto-laid-out graph loses the whole advantage.
Scaife & Rogers 1996, IJHCS 45. Theoretical review naming computational offloading, re-representation and graphical constraining, plus the "resemblance fallacy" and a warning that claims for graphics and interactivity often have "little empirical support." Kirsh & Maglio 1994, Cognitive Science 18, and Kirsh 2010, AI & Society 25: the value of external representations comes from manipulating them (epistemic actions), creating persistent shareable referents and lowering the cost of coordinating thought. Implication: editability by the developer is the mechanism, not a nice-to-have.
Green & Petre 1996, JVLC 7, cognitive dimensions of notations. Viscosity (resistance to local change), premature commitment (forced ordering of decisions, including layout), hidden dependencies, secondary notation (meaning carried by layout, colour and comments). Box-and-wire notations have high viscosity and poor secondary notation but few hidden dependencies. A single-user straw test measured 8:1 viscosity differences between notations. No notation is universally best; each highlights some information at the cost of obscuring other information. Implication: graph editing will be slower than text editing unless the tool provides bulk moves, layout that respects user placement, and free annotation.
Ainsworth 2006, Learning and Instruction 16 (DeFT framework). Multiple representations impose translation costs; "this promise is not always achieved." Suthers et al. 2006, Computers & Education, found that a mixed threaded-discussion-plus-evidence-map condition lost the graph's advantage and attributed this to coordination cost. Implication: chat beside graph is a multiple-representation design and needs bidirectional linking to pay off.
Suthers & Hundhausen 2003, Journal of the Learning Sciences 12 [partial]. 60 students in pairs, random assignment to text, evidence graph or matrix. The matrix produced significantly more discussion of evidential relations; graph did not beat text; no differences in post-test or essay scores. The matrix also caused "excessive consideration and revision of unimportant relations." Implication: representation form steers what gets discussed, not what is learned in an hour. Choose node and edge types to steer the agent-developer exchange toward the relations that matter.
Concept and mind maps. Nesbit & Adesope 2006, Review of Educational Research 76: 55 studies, 5,818 participants, overall g = 0.60; constructing maps g = 0.82 versus studying provided maps g = 0.37; but constructing versus writing text or outlines only g = 0.19; high-ability learners did not benefit. Schroeder et al. 2018, Educational Psychology Review: 142 effects, n = 11,814, g = 0.58; creating beats studying. Farrand, Hussain & Hennessy 2002, Medical Education 36: 50 medical students, mind-map group retained about 10 percent more at one week but reported lower motivation for the technique. Implication: a graph the agent fills and the developer only reads is the weak condition. Even construction gains little over good text notes, and users may not enjoy it by default.
Diagrams in software practice. Cherubini, Venolia, DeLine & Ko 2007, CHI: 9 interviews and 427 survey responses at Microsoft; most drawings were transient whiteboard sketches, decisions "externalized in these temporary drawings and then subsequently lost," electronic diagrams "often out-of-date." Baltes & Diehl 2014, FSE: 394 respondents; most sketches were informal, revised after creation, and often archived (a genuine disagreement with Cherubini on retention). Petre 2013, ICSE: 50 companies; 35 do not use UML, none wholeheartedly, because of complexity, low value and poor synchronisation with code. Implication: developers sketch freely and hate transcription and maintenance. The graph must be captured as a by-product of agent work or it will rot like documentation UML.
Split attention. Chandler & Sweller's split-attention effect and Mayer's spatial contiguity principle. Ginns 2006, Learning and Instruction 16: meta-analysis of 50 studies confirming contiguity effects, larger for complex material. Schroeder & Cenkci 2018, Educational Psychology Review: 58 comparisons, n = 2,426, g = 0.63 favouring integrated over split formats. Implication: a transcript beside a graph is textbook split-source design; mitigate with node-to-message links and inline node cards in chat.
Non-linear navigation. Conklin 1987, IEEE Computer 20: disorientation and cognitive overhead are the central hypertext problems. DeStefano & LeFevre 2007, Computers in Human Behavior 23 [partial]: hypertext impairs reading relative to linear text, most for readers with low working memory or prior knowledge; hierarchical structure and navigation aids reduce load. Implication: keep linear chat as the default path and the graph as an overview with stable hierarchy.
Pirolli & Card 2005 [partial]: cognitive task analysis of intelligence analysts with a foraging loop and a sensemaking loop; leverage points include attention cost and hypothesis generation under confirmation bias. Klein, Moon & Hoffman 2006, IEEE Intelligent Systems: data/frame theory; frames are questioned, elaborated, preserved or reframed, and commitment to a frame is a bias source. Russell, Stefik, Pirolli & Card 1993, CHI: sensemaking is a search for a representation; cost is dominated by fixing the representation. Implication: goal and question records must be cheap to edit and replace; reframing must not require rebuilding the graph.
Jigsaw. Kang, Görg & Stasko 2011, TVCG: 16 graduate students, 4 per condition; the Jigsaw group performed best with least variance, best performers read 23 to 31 of 50 documents. Jigsaw "lacks powerful support for the schematize stage"; participants organised hypotheses on paper. Authors recommend explicit milestones so analysts can resume after dead ends. Implication: evidence marshalling is the supported benefit; hypothesis organisation still needs a dedicated surface, which the graph is.
SenseMap. Nguyen, Xu, Bardill, Salman, Herd & Wong 2016, VAST: in 9 interviews nobody used browser history to make sense; the prototype separates an auto-captured history map from a curated knowledge map. In a 5-person, 2-hour study, 3 built rich maps of 10 to 35 nodes; one lost trust after ignoring the history map. Implication: keep auto-captured history separate from the curated current state; a curated view left empty early tends to stay empty.
Provenance frameworks. Ragan, Endert, Sanyal & Chen 2016, TVCG: provenance types (data, visualisation, interaction, insight, rationale) and purposes (recall, replication, recovery, collaboration, presentation). Xu, Ottley, Walchshofer, Streit, Chang & Wenskovitch 2020, EuroVis STAR. Heer, Mackinlay, Stolte & Agrawala 2008, InfoVis: branching state history in Tableau with chunking, thumbnails and bookmarks exempt from culling; no user study. Lu, Wen, Pan & Lai 2011, INTERACT: 21 business users found analytic trails useful for personal recall, less for collaboration; search over trails was critical. Implication: history is for personal recall. A searchable history with pinned states is enough; a full timeline UI is low value.
Stability and motion. Misue, Eades, Lai & Sugiyama 1995, JVLC: mental-map criteria. Archambault & Purchase 2013, IJHCS: no experiment found conclusive evidence that mental-map preservation aids comprehension of change, but preservation made relocating nodes faster and less error-prone. Heer & Robertson 2007, InfoVis: staged animated transitions improved tracking accuracy, sometimes at a time cost. Implication: hold positions on agent updates so the developer can relocate nodes; animate changes in stages. The current hold/follow design matches this.
IBIS and the capture problem. Kunz & Rittel 1970 introduced issues, positions and arguments for wicked planning problems. Conklin & Begeman 1988, ACM TOIS 6, built gIBIS. Conklin & Burgess-Yakemovic 1991, HCI 6, field study: a low-tech indented-text IBIS captured over 2,300 decisions at NCR; the text form, not the graphical tool, survived. Buckingham Shum & Hammond 1994, IJHCS 40 [partial]: literature review concluding that research "failed to substantiate convincingly" either that argumentation is useful or that designers can use the notations. Conklin, Selvin, Buckingham Shum & Sierhuis 2001, ACM Hypertext, retrospective: "parsing ideas into discrete nodes, with distinctive names and types, seemed to impede the flow of thought, and the resultant structures were hard to change," citing Grudin's rule that whoever invests effort must benefit. Conklin's response, Dialogue Mapping (2005), uses a trained facilitator who builds the map live; the authors report three to six months to reach expert level and only anecdotal evidence. Implication: every predecessor died because the person structuring was not the person benefiting. An agent that structures as a by-product of its own work removes that asymmetry. The facilitator's value came from reflecting structure back ("these two hypotheses are the same," "this question is still open") and breaking the template when needed, not from filing nodes.
QOC. MacLean, Young, Bellotti & Moran 1991, HCI 6: Questions, Options, Criteria as a deliberately retrospective design-space analysis; the authors call their process claims "goals rather than objectives they have achieved." Buckingham Shum's video studies found four interleaved cognitive tasks (unbundling, classification, naming, structuring) and expected cognitive overhead. Implication: an options-versus-criteria matrix is a good post-hoc summary and an expensive live-reasoning surface.
Formality considered harmful. Shipman & Marshall 1999, CSCW 8, and Shipman & McCall 1994, CHI. Four problems: cognitive overhead, tacit knowledge (design students could not formalise IBIS live even though their natural discussions followed it), premature structure, situational structure (a schema fitting one task was of limited value for a near-identical one). Remedies: incremental formalisation, structure suggested from informal text, user-extensible schemas, and always keeping the informal form accessible. Implication: accept untyped free-text notes at entry; let the agent propose kind and links later; never require type selection.
Argument mapping. van Gelder 2001; van Gelder, Bissett & Cumming 2004; Twardy 2004, Teaching Philosophy 27: about 0.7 to 0.8 SD pre/post gains on a critical-thinking test over a semester, with Twardy himself noting the lack of control groups. Alvarez Ortiz 2007: unpublished meta-analysis of 52 studies, argument-mapping-heavy courses 0.78 SD. Dwyer, Hogan & Stewart 2012: "difficult to draw any conclusions" from the early studies; their controlled work found smaller effects. Nesbit & Liu 2025, Higher Education Quarterly: systematic review of 124 studies, 44 percent measured attitudes. Adelman, Lehner, Cheikes & Taylor 2007, IEEE TSMC-A 37: participants "found it difficult to generate Toulmin structures, and their structures varied greatly even though they started with the same content." Implication: evidence is for training over months, not for better decisions at the moment of mapping.
ACH. Heuer 1999 origin, no evaluation. Lehner, Adelman, Cheikes & Brown 2008, IEEE TSMC-A 38 [partial]: 24 participants; ACH reduced confirmation bias only in the inexperienced half. Convertino et al. 2008, CSCW 17 (CACHE): heterogeneous-belief groups debiased, homogeneous groups amplified bias; diversity did the work, not the matrix. Dhami, Belton & Mandel 2019, Applied Cognitive Psychology 33: 50 practising UK analysts randomised to ACH training or none; the ACH group was no more likely to choose the correct hypothesis, only 1 of 25 ranked hypotheses correctly versus 2 of 25 controls, used base rates less (12 percent) and showed higher inconsistency. Whitesmith 2019: ACH did not mitigate serial-position effects. Chang, Berdini, Mandel & Tetlock 2018 [partial]: structured techniques tacitly assume decomposition reduces error; it can add noise. Wilcox & Mandel 2024, Intelligence and National Security 39: review of six experiments, "little to no overall benefit on judgment quality, and may even harm it." Implication: an evidence-by-hypothesis matrix is defensible as an opt-in display when a problem is stuck with several live hypotheses, never as a required step, and never scored by counting inconsistencies.
Cheap debiasing that outperforms heavy structure. Lord, Lepper & Preston 1984, JPSP 47: 150 students; a "consider the opposite" instruction reduced biased assimilation more than an instruction to be fair. Mitchell, Russo & Pennington 1989: prospective hindsight raised correctly identified reasons by about 30 percent. Veinott, Klein & Wiggins 2010, ISCRAM: 178 participants; premortem reduced overconfidence about twice as much as pros-and-cons listing. Klein, Calderwood & Clinton-Cirocco 1986: 26 fireground commanders, 156 decision points, option comparison in under 12 percent. Kahneman & Klein 2009, American Psychologist 64: intuition is trustworthy in high-validity environments with rapid feedback, which describes familiar code with a fast test suite. Implication: one-sentence agent moves ("what would you expect if this were wrong", a premortem before a decision) have better evidence than any notation. Forcing an expert to enumerate options imposes a novice strategy.
Decision records. Nygard 2011 introduced Architecture Decision Records (ADRs) as short text files. Tang, Babar, Gorton & Han 2006, JSS 79: 81 practitioners value rationale but report capture barriers. Capilla, Jansen, Tang, Avgeriou & Babar 2016, JSS 116: ten years of architecture-knowledge tools, "the cost to capture this relevant knowledge hampers a widespread use." Buchgeher et al. 2023: 921 GitHub repositories with Markdown ADRs, adoption low but rising. Implication: the only decision-documentation format with organic adoption is a short text file next to the code, written at decision time. The lesson for the graph is that a decision record must be written at decision time with its rationale in plain text, which the existing decision kind already allows; a separate export format adds nothing.
Problem type. Rittel & Webber 1973: wicked problems have no stopping rule and no enumerable solution set. Simon 1973: structuring turns ill-structured problems into well-structured ones. The Compendium team reports templates help beginners on well-structured problems and expert facilitators are needed for contentious ones. Implication: most coding tasks are tame (tests pass or fail); the wicked part is scoping and requirements, which is where issue-style structure earns its cost.
Hypotheses are the leverage point. Alaboudi & LaToza 2020, VL/HCC: observation study plus controlled experiment with 20 developers. Developers held about two hypotheses per defect; a correct hypothesis early strongly predicted success; offering fault locations did not improve success; offering candidate hypotheses made success roughly six times more likely (small-sample odds). Alaboudi & LaToza 2023, UIST (Hypothesizer) [partial]: randomised experiment with 16 professionals; hypotheses matched from a catalogue and shown with an evidence timeline gave a fivefold success rate and threefold speed. Alaboudi & LaToza 2023, EMSE: 11 professional streamers, 89 episodes; editing 41 percent and testing 29 percent of debugging time. Implication: hypothesis plus linked evidence is the unit that helped. The graph must be fed from agent actions because developers will not stop editing and testing to maintain it.
Questions anchor on observed behaviour. Ko & Myers 2008, ICSE (Whyline): 20 participants; 10 of 10 succeeded on task one versus 3 of 10 controls, twice as fast, exploring a median of 3 files versus hundreds. Sillito, Murphy & De Volder 2008, TSE: 44 question types; tools support single questions poorly and chains worse. LaToza & Myers 2010, PLATEAU: 179 developers, 371 hard questions; the most frequent are intent and rationale. Ko, Myers, Coblenz & Aung 2006, TSE [partial]: 10 developers spent about 35 percent of time on navigation mechanics and failed searches based on misleading cues. Ko, DeLine & Venolia 2007, ICSE: 17 developers, 21 information types, many needs deferred. Implication: question records should state observed behaviour ("why did X happen") and answers should be causal chains. Decision records with rationale answer the question future readers ask most.
Expert strategy and biases. Vessey 1985 [partial]: experts reason breadth-first from a model of correct behaviour and often act without an explicit hypothesis. Zeller's Why Programs Fail recommends a hypothesis, prediction, experiment, observation loop with a logbook. Mohanani, Salman, Turhan, Rodríguez & Ralph 2020, TSE: 65 papers, 37 biases, mitigation research scarce. Salman, Turhan & Vegas 2019, EMSE; Salman et al. 2023, TSE: developers write significantly more confirmatory than disconfirmatory tests, and time pressure increases the bias. Løhre & Jørgensen 2016, JSS: 381 professionals, numerical anchors strongly shift estimates even when not credible. No validated debiasing intervention exists in software engineering. Implication: explicit challenges edges and a visible list of untested alternatives are the closest supported intervention. An agent that echoes the developer's first hypothesis will amplify anchoring.
Information foraging and working sets. Lawrance et al. 2013, TSE: 10 professionals; verbalisations concerned scent-following far more than hypotheses. Kersten & Murphy 2006, FSE (Mylyn): 16 industry programmers over weeks, significant productivity gain; decay of interest mattered for acceptance. Bragdon et al. 2010, CHI (Code Bubbles): 20 students, 33 percent faster with fewer navigations. Implication: evidence labels need scent (identifiers, file and line). Fade stale records rather than delete them. Not every step needs filing under a hypothesis.
Debugging with language-model assistants. Kang, Chen, Yoo & Lou 2025, EMSE (AutoSD): the agent generates a hypothesis and a debugger script, observes, and accepts or rejects; in a 20-person study, scientific-debugging explanations improved patch-correctness judgments on 5 of 6 bugs at no time cost. Chopra et al. 2024 (Robin): 12 professionals, turn-taking with a debugging workflow gave fivefold resolution. Bo et al. 2025: whether the novice led or was led by the model determined over- or under-reliance. Bo et al., CHI 2026 (preprint): 24 participants; a sycophantic assistant produced fewer corrected misconceptions and worse performance, mostly unnoticed. Implication: hypothesis-structured agent output is what helped humans judge results; the main failure mode is the agent validating the user's frame.
Graphologue. Jiang, Rayan, Dow & Xia 2023, UIST: formative study with 10, evaluation with 7 experienced ChatGPT users, no controlled baseline. Real-time node-link diagrams improved perceived comprehension and reduced prompting effort; diagrams that were too detailed or mismatched mental models hurt; relationship extraction reached about 85 percent F-score, so errors accumulate; text was still needed to fill gaps. Sensecape. Suh, Min, Palani & Xia 2023, UIST: 12 participants within-subject; about three times more unique concepts explored and far more revisiting, but only 7 of 12 preferred it. Luminate. Suh et al. 2024, CHI: 14 writers; 40 responses per prompt overwhelmed several. Interactive Reasoning (HIPPO). Pang et al., IUI 2026: 16 participants versus an editable linear text baseline; significant gains in perceived control, assumption awareness and confidence; no change in caution; participants rarely edited unprompted ("made a lot of sense, so I do not feel like changing anything") and edits came through system-generated feedback nodes. CanvasConvo. Amin et al. 2026 (preprint): 24 users over 5 to 7 days; chat still dominated (47 percent of events versus 28 percent canvas), switching was confusing, users scrolled instead of branching, and wanted to know "what the branch knows." Plan-Then-Execute. He, Demartini & Gadiraju, CHI 2025: 248 participants; user involvement in planning degraded already-correct plans, lowered confidence, raised workload and did not calibrate trust. Others: Kim et al. 2023 (Cells, Generators, Lenses), Arawjo et al. 2024 (ChainForge), Branchat 2026 (12-person extended abstract). No peer-reviewed studies exist for Copilot Workspace, Cursor, Devin or Claude Code plan modes. Anastasiou & De Liddo 2024 and MeetMap (Chen et al. 2025, 10 pairs) show language models can build IBIS-style maps live and users prefer them to notes, without measuring decision quality. Implication: keep the conversation primary and the graph secondary (in this app the conversation pane is a terminal, not a structured chat); show each record's context scope; cap volume; expect low spontaneous editing and solicit input at specific records; pair the graph with source text; expect extraction errors.
Faithfulness of shown reasoning. Turpin, Michael, Perez & Bowman 2023, NeurIPS: biasing features drop chain-of-thought accuracy by up to 36 percent without being mentioned. Lanham et al. 2023: faithfulness is task-dependent and often decreases with model size. Chen et al. 2025 (Anthropic): models verbalised a used hint 25 to 39 percent of the time, less on harder questions, and in reward-hacking environments verbalised the exploit under 2 percent of the time. Barez et al. 2025: about a quarter of chain-of-thought papers treat it as interpretability. Zaman & Srivastava 2025 counter that hint-verbalisation conflates unfaithfulness with incompleteness. Korbak et al. 2025: monitorability is useful but fragile. What users see in current assistants is a model-generated summary of thinking. The app's graph is a third layer: a prompted, post-hoc self-report with no faithfulness guarantee at all. Implication: label records as the agent's claims and bind evidence to verifiable artifacts (diffs, test output, file ranges).
Verbalised confidence. Tian et al. 2023, EMNLP: verbalised confidence beats token probabilities. Xiong et al. 2024, ICLR: across five 2023 models on closed-book question answering it was systematically overconfident, concentrated at 80 to 100 percent, with failure-prediction AUROC of 0.52 to 0.61. That result no longer transfers. Hsiao 2026 (arXiv 2609.10996) finds that on post-2025 proprietary models verbalised confidence is the more robust signal and the old advice to prefer log-probabilities no longer holds. Xuan et al., ACL 2026 ("The Confidence Dichotomy") find that evidence tools such as web search induce severe overconfidence in agents while verification tools such as code interpreters ground reasoning through deterministic feedback and mitigate miscalibration. A debugging session with a test suite is the favourable case. Reported but not independently verified here: ConfidenceBench (arXiv 2607.20526) gives frontier models Brier scores near 0.1 on closed-book questions, and the Claude Opus 4.8 system card reports large reductions in overconfidence on agentic coding sessions. Against: longer reasoning budgets can worsen calibration (arXiv 2508.15050), and aligned models remain overconfident relative to base models. On the display side, Zhang, Liao & Bellamy 2020 found displayed confidence improved trust calibration while explanations did not; Kim et al. 2024, FAccT, 404 participants, found first-person hedges reduced over-reliance and raised accuracy; Steyvers et al. 2025 found longer explanations raised human confidence without raising accuracy, and explanations tuned to the model's actual confidence narrowed the gap. Implication: keep the number, because a coarse label hides the same bias with less information and evidence-edge counts are equally agent-authored. Show it beside its grounding (supports, challenges, sources), phrase it as the agent's own estimate, keep it off the compact card, and log confidence against terminal hypothesis status so calibration is measured in this product rather than inferred from 2023 papers.
Explanations and reliance. Bansal et al. 2021, CHI: 1,626 participants; explanations raised acceptance of both correct and incorrect recommendations. Buçinca, Malaya & Gajos 2021, CSCW: 199 participants; cognitive forcing functions (decide first, on-demand answer, delay) reduced over-reliance and were rated lowest. Vasconcelos et al. 2023, CSCW: 731 participants; explanations reduce over-reliance only when verification is cheaper than the task. Gajos & Mamykina 2022, IUI: recommendation plus explanation improved decisions without learning; explanation only produced both. Chen, Liao, Vaughan & Bansal 2023: example-based explanations achieved complementary performance where feature-based ones increased reliance. Fernandes et al. 2026 (preregistered preprint): 559 participants; full reasoning traces lowered accuracy, summaries preserved it, both raised trust, and 35 percent offloaded reasoning. Sun et al. 2026 and "Critical or Compliant?" 2025: certainty cues and confident tone suppress error detection. Implication: force a judgment before revealing the agent's recommendation at decision records; show alternatives rather than one answer; keep tone neutral; default to summaries with drill-down.
Developers and agents. Barke, James & Polikarpova 2023, OOPSLA: 20 developers; acceleration versus exploration modes. Mozannar et al. 2024, CHI: verifying suggestions is a major time cost. Liang, Yang & Myers 2024, ICSE: 410 respondents motivated by speed, not brainstorming. METR 2025: 16 experienced open-source developers, 246 tasks, 19 percent slower with AI while believing 20 percent faster; METR's 2026 follow-up calls this very weak evidence. Paradis et al. 2025 (Google): 96 developers, about 21 percent faster, p = .086. Huang et al. 2025 (preprint): 9 of 13 observed developers reviewed every change; 8 wrote their own plans; developers watched agent thinking as a control tactic. Tang et al. 2026 (preprint): 20,574 sessions; 22.6 percent of misalignment episodes were inaccurate self-reporting, mostly false success claims; 91.5 percent needed explicit user correction. Bainbridge 1983 and Parasuraman & Riley 1997: humans monitor reliable automation poorly and skip verification under load. Implication: agent status claims in the graph are themselves a known failure category. "Claimed done" must look different from "verified done", and decision records need an explicit human action rather than passive updates.
Revised after the adversarial review below. Rows name the actual kinds and fields in electron/shared/graph.ts and the prompt in src/investigation/feed.ts.
- Is: the current-work marker and hold/follow, already shipped; keep positions stable on updates.
- Was: searchable history with pinned states; terminal markers per revision for record-to-transcript jumps.
- Should: proposed experiments and work items, plus the falsifier move ("what would change this picture") as a
challengesrelation. - Why: rationale on decision records, without enumerated alternatives.
- Claimed versus verified: source checks first, verify-run links where they exist.
- Confidence shown with grounding counts and logged against outcomes.
- Agent proposes kinds and links for user notes and emits clarification records; edits beyond that wait for the trial.
The trial measures understanding directly, using Endsley's freeze-probe method: stop or end the session and ask factual questions, then score the answers against the transcript. Feeling in control is recorded but not counted.
- Do developers understand the task better with the graph? After each session ask three questions: what did the agent rule out, and on what evidence; what is it doing right now; what has it claimed that nothing has verified. Compare answer accuracy between graph and transcript-only sessions on the same seeded tasks. If graph users answer better, the feature delivers its goal. If they only feel better, that is the trust-inflation effect and the claimed-versus-verified marks need strengthening.
- Is the agent's confidence calibrated in this product? Every hypothesis with
confidenceends assupportedorrejected, so the app can log (confidence, terminal status) pairs and compute a Brier score for free. This decides the confidence display with data rather than with 2023 citations. - Are false leads caught? Seed each trial task with one injected bug and one misleading clue, such as a wrong log line, and record whether the clue became a
supportedhypothesis. Self-reported speed is not collected; perception of speed with agents is inverted often enough (METR 2025) to be useless, and a handful of sessions cannot detect a time effect anyway. - Do developers edit the graph at all without prompting? HIPPO and CanvasConvo suggest not. Log edits per session split into solicited (answers to agent-emitted clarification records) and unsolicited.
- Does the Architecture workflow benefit anyone but novices? Recognition-primed decision research predicts experienced developers skip it. Compare Compare-options usage against task familiarity.
An independent review attacked the first version of the implications. Changes it caused:
- Verification surface: "require sources" became "flag unsourced"; a rejecting validator rewards fabricated paths (Kalai et al. 2025 on penalised abstention). "Verified" must come from app-generated provenance, not from a status the agent sets.
- Confidence: the replacement by coarse labels was dropped. The cited calibration work was 2023 closed-book question answering; 2026 work on post-2025 models and on verification-tool grounding reverses it, and coarse labels or edge counts are equally agent-authored self-report.
- Debiasing prompts: the two-hypothesis floor and premortem were dropped. The cited studies debiased humans; prompted self-critique in models is unreliable without external feedback (Huang et al., ICLR 2024), and a quota produces straw hypotheses that pollute the graph. The falsifier prompt survives because it feeds the existing
challengesrelation. - Forcing step: moved from decision records to the existing plan-approval and question wait points, reconciling with He et al. 2025.
- Graph-chat binding: reframed around terminal bookmarks, because the conversation pane is a PTY with no turn identifiers and a chat renderer is out of scope.
- Consistency: the option-comparison matrix and an ACH matrix are now treated alike as opt-in summaries; "defer Research" became "no further investment", since the workflow is already shipped.
- Owner review (2026-09-15): verification provenance narrowed to what the app can attest, with source checks as the first step; ADR export dropped as complexity without gain; terminal markers kept after confirming the marker API is agent-independent; measurement replaced by freeze-probe questions on understanding, matching the stated product goal.
- Corrected baseline: the first version claimed semantic notes require kinds and that no plain notes exist; the prompt already allows untyped notes and the edit-feedback loop already exists.
- Full texts of Buckingham Shum & Hammond 1994, Lehner 2008, Pirolli & Card 2005, Chang et al. 2018 and Suthers & Hundhausen 2003 were paywalled; those entries rely on abstracts and citing papers.
- CanvasConvo, Fernandes 2026, Tang 2026, Huang 2025, Bo 2026, Zaman & Srivastava 2025 and the Microsoft rollout study are preprints.
- Argument-mapping effect sizes all come from one research group without controls. The premortem "30 percent" figure is from a 1989 laboratory study on explanation generation.
- Cherubini 2007 and Baltes & Diehl 2014 disagree on whether sketches are kept; cite both.
- No study directly tests an editable, agent-populated reasoning graph in software engineering. Every implication above transfers by analogy.
- Calibration, faithfulness and self-report figures are model-specific. The 2026 calibration papers cited in section 6 were checked against their abstracts; ConfidenceBench and the Opus 4.8 system card figures are reported second-hand.
- The conversation pane is a terminal. Split-attention and binding remedies from the chat-interface literature transfer only in part.