[Task Proposal] Multi-Pursuit PIT Maneuver and Vehicle-State Reconstruction from a Police Chase Compilation · Issue #98 · PhiloLabs/agentic-vbench · GitHub
Skip to content

[Task Proposal] Multi-Pursuit PIT Maneuver and Vehicle-State Reconstruction from a Police Chase Compilation #98

Open

Description

@DDMeer

[Task Proposal] Multi-Pursuit PIT Maneuver and Vehicle-State Reconstruction from a Police Chase Compilation

Task summary

The agent is given a long-form compilation containing multiple police
vehicle pursuits involving Trooper Byrd.

For each distinct pursuit segment, the agent must reconstruct the
chronological sequence of pursuit interventions and vehicle-state
transitions.

The task focuses on identifying:

  1. the temporal boundaries of each pursuit;
  2. each visible PIT maneuver attempt;
  3. whether each PIT attempt succeeds or fails;
  4. the suspect vehicle's immediate state after each intervention;
  5. the event that finally causes the suspect vehicle to stop;

Input media

  • Video ID: doESuMQWVNo
  • Source URL: https://www.youtube.com/watch?v=doESuMQWVNo
  • Source type: edited police-pursuit compilation
  • Final local filename: police_chase_doESuMQWVNo_source.mp4
  • Duration: 00:38:32.59
  • Resolution: 1920 × 1080
  • Frame rate: 60 fps
  • Video codec: H.264
  • Audio codec: AAC
  • File size: 787,623,373 bytes (approximately 751 MB)
  • SHA256: 58ea866b821bb87e5c1413186a66802d7cd0479d7e913b24aeb2eeddf0614c55

Proposed task prompt

Watch the complete police-pursuit compilation.

First, divide the video into distinct pursuit segments. For every pursuit,
reconstruct all visible PIT maneuver attempts and the resulting vehicle
state transitions.

For each pursuit, report:

  • the pursuit start and end timestamps;
  • a neutral identifier for the suspect vehicle;
  • each PIT attempt timestamp;
  • whether the PIT attempt made contact;
  • whether it caused a meaningful loss of control;
  • the vehicle state immediately after the attempt;
  • whether another PIT attempt occurred later;
  • the final stopping event;
  • the timestamp at which the vehicle came to a final stop.

Use neutral segment identifiers such as pursuit_01, pursuit_02, and
pursuit_03.

Write the final answer to:

/workspace/output/solution.json


Proposed output format

{
  "pursuits": [
    {
      "pursuit_id": "pursuit_01",
      "start_time": "00:00:18",
      "end_time": "00:06:42",
      "suspect_vehicle": {
        "type": "sedan",
        "color": "white"
      },
      "pit_attempts": [
        {
          "attempt_id": 1,
          "timestamp": "00:04:12",
          "contact": true,
          "outcome": "temporary_yaw",
          "vehicle_continued": true
        },
        {
          "attempt_id": 2,
          "timestamp": "00:05:37",
          "contact": true,
          "outcome": "full_spin",
          "vehicle_continued": false
        }
      ],
      "final_stop": {
        "timestamp": "00:05:44",
        "stop_type": "full_spin",
        "location_type": "roadway"
      },
      "seconds_from_first_pit_to_stop": 92
    }
  ]
}

Proposed PIT outcome labels:

no_contact
contact_without_visible_effect
temporary_yaw
partial_spin
full_spin
road_departure
barrier_collision
rollover
unknown

Proposed final-stop labels:

full_spin
road_departure
barrier_collision
vehicle_collision
rollover
mechanical_stop
voluntary_stop
other

The exact schema and label set may be refined after a complete manual
review of the video.


Annotation rules

Pursuit boundaries

A new pursuit begins when the compilation transitions to a different
suspect vehicle or clearly different incident.

PIT attempt

An event counts as a PIT attempt only when the police vehicle visibly
moves into contact, or clearly attempts to move into contact, with the
rear-side area of the suspect vehicle in a manner intended to rotate it.

Successful PIT

A PIT attempt will be considered successful only if it visibly causes a
meaningful loss of directional control, such as:

  • substantial yaw;
  • partial or complete spin;
  • departure from the roadway;
  • collision caused by the induced rotation;
  • final immobilization.

A contact that produces no clear change in vehicle motion will be labeled
contact_without_visible_effect.

Final stop

The final stop is the first point after which the suspect vehicle does not
resume meaningful flight within the same pursuit.

Temporary slowing or stopping followed by renewed flight will not count
as the final stop.


Ground truth

The ground truth will be manually annotated from the finalized video
file.

For each pursuit, I plan to record:

  • pursuit boundaries;
  • suspect-vehicle visual description;
  • every visible PIT attempt;
  • contact and outcome labels;
  • immediate post-contact vehicle state;
  • final stopping event;
  • timestamps for each event;
  • whether footage is continuous, replayed, or edited.

Oracle submission

The oracle submission will be a valid solution.json generated from the
verified annotations.

It should receive a verifier score of 1.0.

The oracle run will verify that:

  • all pursuits and scored events are representable;
  • timestamps and labels are valid;
  • event matching works correctly;
  • ambiguous or excluded events are handled consistently;
  • derived elapsed-time fields are calculated correctly.

Deterministic verifier

A deterministic Python verifier will compare the submitted JSON with the
hidden ground truth.

Tentative scoring:

  • pursuit segmentation and ordering: 15%
  • PIT-attempt detection and timestamps: 30%
  • contact and outcome classification: 25%
  • final-stop event and timestamp: 20%
  • first-PIT-to-final-stop duration: 10%

Why this requires long-horizon video understanding

The input is a compilation containing multiple visually similar pursuit
incidents.

The agent must:

  • separate distinct incidents from replays and editorial transitions;
  • track the same suspect vehicle across changing camera views;
  • distinguish separate PIT attempts within one pursuit;
  • connect an earlier intervention to a later vehicle outcome;
  • determine whether a temporary loss of control becomes a final stop;
  • retain incident state across several minutes;
  • aggregate all incidents into one structured ledger.

A single crash frame may show a final outcome, but it does not reveal:

  • whether an earlier PIT attempt occurred;
  • whether that attempt failed;
  • how many attempts preceded the stop;
  • whether the shown footage is a replay;
  • how long the vehicle continued after first contact.


Planned ablations and calibration

Target behavior:

  • oracle score: 1.0;
  • empty and random baselines: near 0;
  • no-media baseline: near 0;
  • limited-frame and transcript-only baselines should perform materially
    below full-media evaluation;
  • strong-agent score should ideally remain below the benchmark target
    threshold.

Activity

  1. techgenmini commented on Sep 1, 2026

    @techgenmini
    Collaborator

    Hi @DDMeer — I reviewed this as an early design checkpoint. This is not a duplicate: replay deduplication and linking multiple interventions to subsequent vehicle state could distinguish it from the merged
    event-chain tasks. However, I would resolve three bounded design questions before beginning full implementation.

    1. Please freeze the audio contract first. The source narration/captions directly announce several incident transitions, replay switches, and final-stop outcomes. With the tentative 15% segmentation and 20%
      final-stop components, an audio/transcript-only answer may exceed the 0.15 degraded-input gate without reconstructing the PIT chain. My preference for this visual task is a silent derived artifact. If
      audio is retained, please first run one forced-answer audio/transcript-only pilot using a provisional key.

    2. Before full annotation, provide a provisional census of pursuit segments and distinct physical PIT attempts after replay deduplication, together with row-level observability status. Please define segment
      boundaries, PIT versus ordinary collision/ramming, no-contact attempts, replay handling, timestamp anchors and tolerances, the immediate-state window, and unknown/exclusion behavior for cuts or
      occlusion. The current outcome labels overlap, so they should be replaced with orthogonal observable fields or given an explicit precedence rule. Please also plan for two independent annotations plus
      adjudication.

    3. Simplify the schema and reward. another_pit_later and seconds_from_first_pit_to_stop are deterministic derivatives and should be computed verifier-side rather than scored separately. Use one-to-one event
      matching, keep omitted events in the recall denominator, penalize duplicates/extras, and validate malformed entries locally rather than zeroing the entire submission.

    Once the media, annotation contract, and exact event count are frozen, run one current Codex pilot. There is no need to begin the full three-harness campaign or final artifact packaging yet.

    This is early design feedback, not implementation or merge approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions