[TASK: Biology] Spectral myeloid phenotyping from raw flow-cytometry data by fangimm · Pull Request #2345 · harbor-framework/terminal-bench-science · GitHub
Skip to content

[TASK: Biology] Spectral myeloid phenotyping from raw flow-cytometry data - #2345

Draft
fangimm wants to merge 1 commit into
harbor-framework:mainfrom
fangimm:spectral-myeloid-phenotyping
Draft

[TASK: Biology] Spectral myeloid phenotyping from raw flow-cytometry data#2345
fangimm wants to merge 1 commit into
harbor-framework:mainfrom
fangimm:spectral-myeloid-phenotyping

Conversation

@fangimm

@fangimm fangimm commented Oct 13, 2026 •
edited
Loading

Copy link
Copy Markdown

Draft / work in progress. This PR registers the task scaffold for the scope-updated proposal. The de-identified input fixture, reference solution, verifier and the human-written instruction are not yet included, and placeholders keep the reward at 0. It will be marked ready for review when complete.

Task Proposal

Link to the approved task proposal (GitHub Discussion):

#1992

Scope update posted on the proposal: #1992 (comment). The approved compensation-aware batch-quantification workflow is implemented on a real 11-color Cytek Aurora spectral dataset (mouse lung and colon, two staggered harvest batches) instead of the synthetic two-color fixture, as suggested in review.

Status

Done (privately, not in this PR):

  • raw-only feasibility audit;
  • pre-registered unmixing freeze, with comparison against vendor unmixing;
  • pre-registered independent reference analysis (frozen);
  • extraction of the authors' expert gating, documented adjudication (signed by Fang Xie; co-author confirmation pending) and an adjudicated reference.

Remaining before review:

  • independent second-expert review;
  • reference solution v2 (development / held-out split fixed in advance);
  • verifier endpoints and tolerances, with calibration evidence;
  • de-identified fixture hosted on the benchmark Hugging Face dataset;
  • human-written instruction.md;
  • oracle / nop / harbor check;
  • frontier trials and failure analysis.

Checklist

This task meets the following criteria. If it doesn't match a criterion, I've explained why below.

  • All behavior checked in the test cases is described in the task instruction.
  • All behavior described in the task instruction is checked in the tests.
  • My test cases have informative docstrings that describe which behavior they check.
  • It is hard for the agent to cheat on my task (e.g. by editing data files, looking inside files for strings that represent solutions, training on the test set, etc.).
  • My instruction.md was written by a human.
  • My solution/solve.sh was written by a human (with minimal help from a language model).
  • If the agent produces structured data (e.g. it is tasked with building an API) the exact schema is described in the instruction.md or a separate file.
  • If my task uses external dependencies (e.g. Docker images, pip packages, etc.) their versions are pinned to ensure reproducibility.
  • I ran this task with a strong model (e.g. Claude Opus) using harbor run -p tasks/<task-name> -a <agent> -m <model>.
  • For failing runs (expected for hard tasks), I've added an analysis below to confirm the task itself is valid.
  • I've truthfully disclosed any conflict of interest in task.toml conflicts_of_interest.

Unchecked items are pending; the task is a work in progress.

Agent Run Analysis

Pending. Trials will be run once the task is complete.

Evidence

Oracle Solution

Pending.

Frontier Model Run

Pending.

@fangimm
Add spectral-myeloid-phenotyping task skeleton (work in progress)
f6dc1d1 
Draft scaffold for the scope-updated proposal harbor-framework#1992: raw spectral flow
cytometry (Cytek Aurora, mouse lung and colon), reference-control
unmixing, acquisition QC and hierarchical immunophenotyping.
Fixture, reference solution, verifier and instruction are not yet
included; placeholders keep the reward at 0.
@github-actions

github-actions Bot commented Oct 13, 2026 •
edited
Loading

Copy link
Copy Markdown

@github-actions

github-actions Bot commented Oct 13, 2026 •
edited
Loading

Copy link
Copy Markdown

📁 Task Overview

Task instruction

Work in progress: placeholder. The task instruction will be written by the task authors before this PR is marked ready for review.

You have 18000 seconds to complete this task. Do not cheat by using online solutions or hints specific to this task.

Task metadata

Domain: life-sciences · Field: biology · Subfield: flow-cytometry · Tags: spectral-flow-cytometry spectral-unmixing immunophenotyping gating quality-control myeloid-cells mouse-tissue · Expert time: 8 hours · Agent timeout: 5 hours · CPUs: 2 · Memory: 8 GB

Author information

AuthorFang Xie
Yifan Jin
Emailfangxf@umich.edu
yifanjin@umich.edu
OrganizationUniversity of Michigan
University of Michigan
ProfileFang Xie: https://orcid.org/0009-0002-5800-2132
Yifan Jin: https://github.com/Oshawott324
Referred byYifan Jin
Relevant
experience		Fang Xie designed and analyzed the spectral flow-cytometry experiment used in this task (Cytek Aurora, 11-color mouse lung and colon immune panel) and performs multiparameter immunophenotyping in pharmaceutical-sciences research on immunotherapy and allergy.
Conflict of
interest		Fang Xie and Yifan Jin are co-authors of this task. Yifan Jin is a co-founder of Complexity AI, which develops Datalox software and instrument integrations for scientific agents. This task is independently verifiable and requires no Datalox product or MCP. The dataset was generated in the authors' own laboratory; the authors' expert gating is one input to an adjudicated reference that is subject to independent expert review.

Task details

Difficulty
explanation		Solving this requires the full analytical chain of a spectral-cytometry analyst working from raw detector data. Fluorophore and autofluorescence signatures must be derived from a mixed set of bead and cell reference controls, with a single unstained reference, and judged for whether they adequately describe tissue cells. The analyst must reason about how unmixing residuals and spreading error interact with downstream gates, recognize acquisition-quality problems as distinct from low yield, and build a myeloid/lymphoid hierarchy in two tissues where key populations are not separable by any single marker. Errors early in the chain propagate into plausible-looking but wrong population frequencies.

The data are real, not synthetic: an unpublished Cytek Aurora acquisition of mouse lung and colon in two staggered harvest batches, de-identified for release. This is routine but expert work for immunologists and core-facility analysts who run high-parameter spectral panels on tissue.
Solution
explanation		Work in progress. The reference analysis was first written and frozen without access to the authors' gating. It is being revised against a documented, adjudicated expert reference, with development and held-out evaluation splits fixed in advance. Its approach will be summarized here when `solution/` is complete.
Verification
explanation		Work in progress. Planned outcome-based grading:

- structural and schema checks on all outputs;
- property-based unmixing checks recomputed from the supplied reference controls;
- event-level agreement on high-confidence cores of an adjudicated expert reference, with boundary bands excluded;
- population-frequency tolerances calibrated against independent correct analyses;
- hierarchy and cross-output consistency;
- objectively defined acquisition-QC labels.

No exact gate geometry or proprietary vendor output is graded. Tolerances and their calibration evidence will be documented here.

Task files

tasks/life-sciences/biology/spectral-myeloid-phenotyping/
├── README.md
├── instruction.md
├── task.toml
├── authoring/
│   ├── evidence/
│   │   └── README.md
│   └── provenance/
│       └── README.md
├── environment/
│   └── Dockerfile
├── solution/
│   └── solve.sh
└── tests/
    ├── Dockerfile
    ├── test.sh
    └── test_outputs.py

Ran on f6dc1d1. Runs on each push and on /overview.

@github-actions

github-actions Bot commented Oct 13, 2026 •
edited
Loading

Copy link
Copy Markdown

🔍 Task Validation Results

Task Similarity Docker Oracle Nop
spectral-myeloid-phenotyping ✅ ✅ ❌
47s		 ⏭️

⏱️ Average oracle runtime: 47s (over 1 task(s) · solve 0s · verify 33s)

📋 View run summary for detailed output

Legend
  • Similarity: Task is not too similar to existing tasks
  • Docker: Environment builds successfully (local prebuild on the GH runner)
  • Oracle: Solution (solve.sh) passes all tests; the time under it is the
    whole oracle trial — environment build, solve, verify
  • Nop: Doing nothing fails tests
  • ⏭️ = Skipped (prerequisite failed)
  • ⚠️ = No result produced (run never reached the task; not a verdict)
  • ➖ = Not run (validate_env is not docker; harbor builds remotely)

Ran on f6dc1d1. Ran automatically on push (docker backend, no credentials). A maintainer's /validate re-runs it or covers the tasks listed as manual.

@github-actions github-actions Bot added the new task New task proposal to TB-Science label Oct 13, 2026
@github-actions

github-actions Bot commented Oct 13, 2026 •
edited
Loading

Copy link
Copy Markdown

Static Checks ✅

26 passed ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Check
Canary strings
Dockerfile refs
Dockerfile sanity
Dockerfile platform
Absolute paths
Test refs
test.sh sanity
Task fields
Task path
Task structure
Task timeout cap
Instruction suffix
GPU types
Network policy
Task slug length
Separate verifier
Verifier tooling baked
Trial network fetch
Pip pinning
Pytest version pin
Nproc usage
Compose host binds
Binary reward
Task README
Rubric consistency
Task proposal / issue link

Ran on f6dc1d1. Runs on each push and on /check. See ci_checks.

@fangimm fangimm changed the title [WIP] Add spectral-myeloid-phenotyping (scope update of proposal #1992) [TASK: Biology] Spectral myeloid phenotyping from raw flow-cytometry data Oct 13, 2026
@StevenDillmann

StevenDillmann commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

new task New task proposal to TB-Science

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@fangimm @StevenDillmann