Summary

The query-engine StatsAggregator computes per-interval statistics with a full-length boolean mask per interval. Its cost is therefore O(intervals × samples) per channel and container. This was fine for events with a handful of instances per container, but TimeWindowEvent routinely produces thousands to hundreds of thousands of windows per container, so a scoped StatsAggregator can make the solve extremely slow or time out.

Found during code review of feature/time_window_event (TimeWindowEvent).

Where

src/impulse_query_engine/analyze/query/aggregations/stats_aggregator.py

• build() calls _calculate_aggregations(series, t_start, t_end) once per non-degenerate interval and per input series.

• _calculate_aggregations starts with

  mask = (sample_series.tends > t_start) & (sample_series.tstarts < t_end)

  which scans the whole series for every interval, and then indexes sample_series.values[mask] again for several statistics (start, end, min, max).

• _calculate_cross_channel_statistics clips every input series per interval via series.where(Intervals([t_start], [t_end])), which is again a full pass per interval and series.

Impact (example)

A 24 h container with 10 ms samples (8.6e6 rows) and 1 s windows (86,400 windows) means about 7e11 comparisons per channel inside one pandas UDF call. The default max_windows_per_container (1,000,000) allows even more.

Proposed approach

The samples of a SampleSeries are sorted and non-overlapping (tstart_i < tend_i <= tstart_{i+1}). So each interval's samples form one contiguous range, found with two binary searches:

lo = np.searchsorted(series.tends, t_start, side="right")   # first sample with tend > t_start
hi = np.searchsorted(series.tstarts, t_end, side="left")    # first sample with tstart >= t_end

Then:

• Compute all ranges at once (vectorized searchsorted over the arrays of interval starts and ends), and use slices (views) instead of masks. That makes it O(samples + intervals · log samples).
• Index values / tstarts / tends once per interval and reuse the slice for every statistic.
• Apply the same range lookup to the cross-channel path instead of where(Intervals([...])) per interval.
• Optionally vectorize the common statistics (min / max / duration-weighted mean) over all windows with np.minimum.reduceat / np.add.reduceat on the range boundaries.

Keep the existing semantics:

• the clipping by SampleSeries.where(intervals) upstream in build() (durations are already clipped to the intervals);
• NaN results for empty / all-NaN intervals;
• custom per-channel statistics still receive the interval's SampleSeries (possibly empty).

Check SampleSeries.where for the same per-interval pattern while at it.

Acceptance criteria

• Results are identical to the current implementation. Add a property-style unit test that compares old (mask) and new (range) results on random sorted series and random interval sets, including empty intervals, all-NaN intervals, intervals touching sample boundaries, and overlapping intervals.
• All existing StatsAggregator / TimeWindowEvent tests pass unchanged.
• A benchmark (e.g. under tests/.../benchmark, or a documented script) shows the improvement for a large window count, such as 8.6e6 samples × 86,400 windows.