[Bug] Long chats stall before inference in CPU conversation-archive embeddings; Stop does not cancel archiving

Summary

On Unsloth Studio Desktop v0.1.909-beta (Windows, Python 3.13.15, AMD Ryzen AI MAX+ PRO 395 / Radeon 8060S), a long tool-heavy conversation can spend many minutes in CPU-based conversation-archive embedding before llama.cpp receives any inference request. Clicking Stop in the chat does not interrupt the ongoing embedding: CPU consumption continues at approximately 17–18 logical cores after cancellation.

This is distinct from the string-matching bottleneck fixed by #14098. The user tested with a local application of that PR, but the hang remained. The issue is reproducibly avoided with RAG_CONVERSATION_ARCHIVE=0.

Direct evidence: py-spy + Windows process CPU counters

Read-only py-spy 0.4.2 dump attached to the running Studio backend (PID selected from port 8888), with samples at baseline, early request, peak CPU, and after pressing Stop. No local variables or user conversation contents were captured.

All 3 peak-CPU samples and all 3 post-Stop samples showed the same active worker thread (the leaf inside BERT/PyTorch differs as computation proceeds):

gguf_generate_with_tools
 -> generate_chat_completion_with_tools
  -> _archive_and_recall (core/inference/llama_cpp.py:2287)
   -> archive_turns (core/rag/conversation_archive.py:460)
    -> embeddings.encode_with_identity (core/rag/embeddings.py:1452)
     -> embeddings.encode
      -> _st_encode (core/rag/embeddings.py:674)
       -> SentenceTransformer.encode (sentence_transformers/SentenceTransformer.py:1094)
        -> BERT forward (transformers/models/bert/modeling_bert.py)
         -> PyTorch CPU operators

Measured cumulative backend-process CPU time:

Between peak samples the process used ~17.2 logical cores; after cancellation ~17.2–17.8 logical cores, with an unchanged archive_turns -> SentenceTransformer.encode stack. The process was otherwise relatively idle at baseline. The application remained responsive to other HTTP endpoints.

The relevant llama.cpp log in a previous run showed an idle model slot throughout the pre-inference delay; no prefill was started before the user aborted that run.

Reproduction

Start v0.1.909-beta on Windows with a GGUF model and a very long existing agent chat, after multiple compactions and containing substantial tool-call history.
Set RAG_CONVERSATION_ARCHIVE=1, restart the backend, and send a short follow-up prompt.
Observe that llama.cpp remains idle while Studio begins heavy CPU activity. In the measured case, ~17–18 logical CPUs are consumed by SentenceTransformer.encode invoked from conversation archiving.
Press Stop in the chat; the UI stops the request, but the backend continues the CPU-intensive archiving computation.
Repeat with RAG_CONVERSATION_ARCHIVE=0: the minutes-long CPU pre-inference phase disappears and inference begins normally.

The reporter previously observed approximately 16 minutes of pre-inference wait with archiving enabled, and quick entry into prefill when disabled. A targeted local application of #14098 did not remove the stall.

Code areas

studio/backend/core/rag/conversation_archive.py::archive_turns: after accumulating all pending chunks, calls embeddings.encode_with_identity([chunk.text for entry in pending for chunk in entry[3]], ...) synchronously at line ~460 in v0.1.909.
studio/backend/core/rag/embeddings.py::_st_encode: calls SentenceTransformer.encode(texts) while the generation worker waits.
The archive embedding call accepts no generation cancellation signal, and the observed CPU computation persists after Chat Stop.

Expected

A short follow-up prompt on an already-open long chat should not block the start of inference for many minutes due to CPU archiving.
Clicking Stop should promptly halt or at least promptly stop scheduling additional archive embedding work.
Conversation archives and recall should remain correct, without losing already committed entries.

Actual

High sustained CPU load in SentenceTransformer.encode prior to model inference.
The same encode is still running after Stop; CPU load does not promptly decrease.
Disabling conversation archiving avoids the delay, but sacrifices archival recall.

Questions / possible improvements

Is there an unexpectedly large number of new chunks being embedded, a CPU thread/batch oversubscription issue, or unnecessary re-embedding? Please add timing and counts (pending turns/chunks/token totals/batch size) around archive_turns and the embedder.
Could embedding be batched with bounded work and cooperative cancellation checks between batches?
Could first-time archival work be deferred or scheduled without holding the pre-inference generation path, while preserving consistency/recall guarantees?

No original conversation, database, screenshots containing private information, or raw profiler dumps are being published. Sanitized stack trace and CPU counters above are sufficient to locate the hot path. Further sanitized diagnostics can be supplied if needed.

Related: #14098 (archive tool-call string matcher, merged; does not resolve this measured SentenceTransformer.encode bottleneck).

qa: benchmark synthetic long-chat archive CPU embedding hotpath (unsl…

@danielhanchen

Confirmed on current main: archive_turns still does one synchronous embeddings.encode_with_identity([...all pending chunks...]) on the generation path with no cancel hook, which matches the reporter’s py-spy stack (_archive_and_recall → archive_turns → SentenceTransformer.encode). That is separate from the #14098 string-matcher fix, and RAG_CONVERSATION_ARCHIVE=0 avoiding the stall is expected.

Opened a fix PR: #14546

Batch archive embeds and poll Chat Stop via cancel_check between batches
Cap chunks per call (RAG_ARCHIVE_EMBED_MAX_CHUNKS, default 256); leftover groups archive later (idempotent by digest)
Keep already-written groups; do not write incomplete ones

@danielhanchen — please take a look when you have a moment; happy to tune batch/cap defaults.

Update: Windows CI reproduction tests and findings

To investigate the reported issue further, we conducted several reproduction tests using GitHub Actions on Windows, focusing on conversation archiving, CPU embedding performance, and repeated compaction across agent prompts.

Here is what we found so far.

Production observation

Long tool-heavy agent conversations repeatedly stall for approximately 9–16 minutes before inference starts.
CPU profiling identifies the blocking path: archive_turns → encode_with_identity → SentenceTransformer.encode → CPU/PyTorch.
The process uses approximately 17–18 logical CPU cores during the stall. Pressing Stop does not immediately interrupt the embedding operation.
Setting RAG_CONVERSATION_ARCHIVE=0 eliminates the observed pre-inference stall, but disables conversation archival recall.

Performance measurements

Using a synthetic CPU-BERT benchmark with 40 agent turns (320 embedding chunks):

Implementation	Full archive processing
Mainline without fix	~2.96 s
PR #14546	~3.32–3.90 s
Experimental early-chunking optimization	~3.71 s

These are measurements from different CI runners, not a controlled proof of performance differences. A separate same-runner comparison also found no consistent mainline speedup.

PR #14546 improves cancellation between batches and limits work per request, but does not demonstrate a reduction in total processing time or eliminate synchronous pre-inference embedding.

Repeated-compaction investigation

We added tests covering Prompt 1 → agent tool activity → context growth → another compaction → Prompt 2.

On the exact v0.1.909-beta release:

Successful archival correctly deduplicates previously stored turns: repeating an identical compaction produces zero new embeddings.
If embedding fails before committing, the next request recomputes all previously uncommitted chunks.
If the embedding identity changes, previously stored turns are fully re-embedded, even though their document count remains unchanged.
The original code performs the large embedding pass before writing the resulting archive documents.

We reproduced these behaviors in Windows CI: exact-release reproduction.

We also implemented opt-in, privacy-safe tracing of identity_mismatch, cache hits, new chunks, embedding duration, database writes, and failures. All seven trace QA tests passed: diagnostic CI.

Current hypothesis and next steps

The recurring production stalls may result from repeatedly uncommitted embeddings, an embedding-backend/identity mismatch, or substantial genuinely new agent content. We have not yet identified which mechanism is responsible in the affected installation.

Next, we will inspect the local rag.db and embedding configuration read-only, then capture decision traces across consecutive real agent prompts to distinguish these possibilities.

We are holding off on further optimization or production changes until the repeated workload is conclusively explained.

Update: Reproduced a long archive stall in portable Windows QA

We copied the affected chat's studio.db and rag.db into an isolated portable Windows installation for reproduction.

After installing the embedding model, we observed:

09:23: Prompt started; UI remained on "Working".
09:41–09:43: Noticeable CPU activity.
09:42:29: conversation_archive.ingest_failed after approximately 19.5 minutes.

The traceback identifies:

archive_turns → encode_with_identity → llama-server → POST /v1/embeddings → httpx.ReadTimeout

The request failed after a retry. Since the archive embeds all pending chunks before committing them, the unsuccessful operation can cause the same workload to be repeated on a subsequent compaction.

This reproduces a long pre-inference archive stall, although the portable installation uses llama-server, whereas the original CPU profile showed SentenceTransformers. We therefore cannot yet conclude that both stalls have the same root cause.

Next: We have installed opt-in timing diagnostics in the isolated portable QA environment for each embedding batch, HTTP timeout/retry, server restart, and archive commit decision. We will reproduce the compaction again and analyze the timing traces to explain the elapsed time and identify which work is repeated before proposing a fix.

Update: Python profiling identifies a second bottleneck in conversation archiving (Windows portable QA, v0.1.909-beta)

I reproduced the long pre-inference "Working" stall again in the isolated Windows portable QA installation using the affected long agent conversation.

Observed timeline (October 22, 2026; local CEST):

11:31 — Request started; the UI remained on "Working".
11:58 — The backend was still running. A py-spy dump captured an active+gil thread in the archive occurrence matcher.
11:59:13 — POST /api/inference/cancel returned HTTP 200.
12:01:29 — Archive ingestion reported an httpx.ReadTimeout on the llama-server embedding request. The POST /v1/chat/completions request completed after 1,795,723 ms (~29 min 56 s).

Active Python stack at the time of the stall:

_same                         conversation_archive.py:1382
<genexpr>                     conversation_archive.py:1399
_occurrences                  conversation_archive.py:1398
archive_turns                 conversation_archive.py:435
_archive_and_recall           llama_cpp.py:2224
generate_chat_completion_with_tools

The installed QA copy is instrumented with our opt-in diagnostic patch; the matcher itself still uses the release's full transcript-position scan. In _same(), tool-call comparisons can repeatedly split/join long strings and search both halves for each candidate position. This is a plausible CPU-bound scaling bottleneck before embedding starts. The snapshot proves this code path was active, but does not yet establish how much of the total stall duration it consumed. CPU deltas suggested roughly one occupied core in this phase, not the earlier 17–18-core SentenceTransformer spike.

A separate embedding failure was also observed later in the same request:

httpx.ReadTimeout: timed out
RuntimeError: llama-server embedder POST /v1/embeddings failed after retry

The cancellation endpoint returned 200, but the completion did not finish until ~136 seconds later; this is consistent with cancellation not interrupting the in-progress archive operation promptly.

Current interpretation: At least two distinct expensive phases may block the generation path: (1) transcript occurrence matching, and (2) embedding requests/retries. This does not establish that both share a single root cause, nor that the matcher alone explains the entire 30-minute delay. The earlier production CPU-SentenceTransformer trace remains a separate observation.

I am reviewing the actual QA source and an experimental indexed-matcher branch, with regression/parity tests planned for repeated turns, tool-call-only groups, partial turns, branch matching, and cancellation. I can provide the stack dump and full server log if helpful.

Follow-up: indexed archive matcher and embedding timeout reproduced (Windows Portable QA, October 22, 2026)

Using an isolated QA copy of Studio v0.1.909-beta and the affected long tool-heavy conversation, we ran three prompts with an experimental indexed occurrence matcher:

Run	Chat request duration	Archive result
Indexed matcher v3, first	283.7 s	HTTP embedding timeout
Indexed matcher v3, follow-up	187.2 s	HTTP embedding timeout
Indexed matcher v3, llama-server embedding batch=8	294.8 s	HTTP embedding timeout; zero writes

The third run had explicit archive tracing enabled:

matched 2549 / 2549 groups in 2463.4 ms
prepared 2549 pending groups, 5160 chunks, 0 cache hits, in 36867.4 ms
first planned batch: 8 chunks (645 batches planned overall)
HTTP attempt 1: ReadTimeout at 60006.6 ms, server_exited=false
HTTP attempt 2: ReadTimeout at 60024.0 ms, server_exited=false
archive_final: 157708.8 ms, written=0

The job failed on the first 8-chunk embedding batch; 645 batches were planned, not performed. A separate one-chunk request returned in 6.6 ms after failure, but did not complete the archive.

Matching is now about 2.5 seconds for this real workload; the synchronous embedding stage remains a blocker. Chat completion can return HTTP 200 even after archive ingestion fails, so response success must not be interpreted as archival success. The llama-server model log also reports 78,821 prompt tokens evaluated in ~97 seconds, separate from archive overhead.

This is a sequential QA observation, not a controlled A/B benchmark, and does not yet establish why the 8-chunk embedding request times out. We are investigating embedding server logs and a QA-only bounded, resumable, incremental-commit archive strategy. Production remains unchanged. The issue remains unresolved.

@danielhanchen

@danielhanchen quick scope check on #14546 vs the later QA updates:

#14546 is intentionally narrow: cooperative cancel between archive embed batches + a per-call chunk cap on the generation path (the original CPU SentenceTransformer.encode / Stop-does-not-cancel stack).

Griggelor’s follow-ups look like additional issues on top of that:

transcript occurrence-matcher cost (_occurrences / _same) before embed
llama-server /v1/embeddings HTTP timeouts with large pending sets (and written=0 when the whole pass fails before commit)

Happy to either:

keep Studio: cancel + bound conversation-archive embeddings before inference (#14519) #14546 as-is and land the cancel/cap fix, or
extend it (or open a follow-up) for matcher indexing / embed timeouts / incremental commit — whichever you prefer.

No further change from me until you say which way you want to go.

Major progress on #14519: 5,117-chunk archive backfill completed, 100% reuse after restart; remaining ~35s pre-archive latency to profile

Following our earlier reproductions in this issue, we now have a successful full-backlog archive run and a subsequent persistence/reuse check after a Windows Portable QA restart.

Environment: Unsloth Studio v0.1.909-beta, isolated Windows Portable QA instance, original affected long tool-heavy conversation. These are experimental QA-only patches, not an upstream-ready change or modifications to the production installation. This is also separate from the narrower cancellation/per-call-cap work discussed in #14546.

Original bottlenecks

The affected compaction produced ~2,549 archivable conversation groups and initially ~5,160 embedding chunks. Prior runs spent many minutes in synchronous archiving or aborted at the embedding stage with no committed archive data. We observed:

Excessive CPU time in repeated transcript occurrence matching.
Large monolithic embedding requests timing out.
An additional embedding-server HTTP 503: The KV cache is full with other conversations; try again shortly. Short retries alone were not reliably sufficient.

QA changes — implementation outline (v3 → v8)

The main archival changes are in studio/backend/core/rag/conversation_archive.py; the separate embedding-server handling is in embed_llama_server.py.

Indexed matching (v3): Construct occurrence/position indexes once for the persisted transcript and live window, then use indexed lookups to calculate group positions and the live-aware write budget. In a full-workload trace, matching all 2,549 groups took ~2.46s.
Bounded incremental ingestion (v5): Prepare and embed complete groups, not one giant embedding request. Commit each group's document and its vectors together; do not mark a document completed before all its chunks are written. Preserve committed groups on subsequent failures; on the next prompt, the existing content-hash/embedding-identity checks skip them. Default QA foreground limits: four new groups / twelve chunks per request, with a whole-group boundary.
Embedding resource control (v6/v7): Retry only the identified KV-cache-busy 503 error with bounded backoff, and optionally launch the separate embedding llama-server with --parallel 1; keep RAG_EMBED_BATCH=1. Three successive bounded real-world QA runs committed 12 additional groups without the earlier KV503. This is not a controlled proof that --parallel 1 alone fixed the issue.
Explicit full-drain benchmark (v8): RAG_ARCHIVE_QA_FULL_DRAIN=1 makes archive_turns() scan the entire current archivable backlog instead of stopping at the v5 foreground group/chunk caps. It deliberately remains synchronous/blocking for measurement; it is not a background worker. For each pending group, call encode_with_identity(...), check the returned identity, then enter the write transaction (BEGIN IMMEDIATE), recheck whether that digest is already sufficiently archived, and commit the document + chunk vectors atomically. Only the exact known KV503 condition gets extra bounded group-level cooldown/retries. Trace separate prepare/embed/commit durations and prepared_groups, processed_groups, written, hits, and completion state.
Completion semantics: complete=true requires no top-level error, all eligible groups scanned, and all pending groups processed. It does not mean every processed group requires a new document row: the under-write-lock deduplication check can skip a now-already-satisfied group. The subsequent hit-only runs are the stronger verification that no archive backlog remains.

Relevant QA configuration:

set "RAG_ARCHIVE_INCREMENTAL_QA=1"
set "RAG_ARCHIVE_QA_MAX_GROUPS=4"
set "RAG_ARCHIVE_QA_MAX_CHUNKS=12"
set "RAG_EMBED_BATCH=1"
set "RAG_EMBED_KV503_RETRY_QA=1"
set "RAG_EMBED_SLOT_POLICY_QA=single"
set "RAG_ARCHIVE_TRACE=1"
set "RAG_ARCHIVE_QA_FULL_DRAIN=1"
set "RAG_ARCHIVE_QA_FULL_GROUP_RETRIES=2"

The full-drain flag explicitly overrides the two per-prompt group/chunk limits only during this QA benchmark. The base v7 behavior remains bounded when the flag is disabled.

Actual v8 full-drain result

The measured embedding-phase throughput was ~24.5 chunks/s (including per-group request overhead). Unlike the earlier monolithic approach, the whole backlog was processed without a top-level abort. Preparation, embedding, and commit figures are measured sub-times, not a complete exclusive breakdown of wall time.

Critical post-backfill verification

A follow-up in the same chat showed hits=2549, pending=0, written=0, chunks=0 in 3.24s.

We then restarted the isolated QA backend and reloaded the main model. Two more requests recognized all 2,549 groups with zero re-embeddings and completed archive checks in approximately 3.04s and 2.42s. Conversation recall also retrieved four earlier passages, and normal model responses completed.

This verifies that the full-drain result was persisted and recognized after restart, not merely cached in memory.

Remaining latency is separate from successful archiving

Two post-restart requests still exhibited an approximately 34–35s gap before archive_begin, despite their ~2–3s archive checks.

The first also had a normal cold-context cost: llama-server evaluated ~90,056 prompt tokens in ~111.6s after the model reload. The second reused ~90,078 cached tokens and evaluated only ~1,377 new tokens in ~2.64s, so its model-processing time was ~3.60s. This distinguishes the reproducible pre-archive delay from both archival embedding and LLM prefill. The current logs do not identify its cause.

Next steps

Profile the remaining ~34–35 seconds with per-stage monotonic timing around chat-run ingress, conversation loading, transcript reconstruction, compaction/window projection, archive invocation, recall, and dispatch to llama-server; investigate any other unexpectedly long waits without assuming they are caused by RAG.
Preserve v8 as a known-good QA benchmark baseline; validate stability, error recovery, and the impact of repeated use.
Design a cancellable/resumable background or idle archival path so future large backlogs do not block time-to-first-token. Keep existing atomic and identity/dedup guarantees.
Prepare a minimal reviewable upstream diff/test case after the remaining latency has been isolated.

Assessment: This is a substantial breakthrough for the core archive failure, but not yet a complete fix for chat latency or cancellation. All results above are from an isolated QA environment. We can supply a focused patch/diff and test details for upstream review.

Fixed! Before a reply could start, Studio re-indexed every old message of a long chat in one go on the CPU, and pressing Stop did not interrupt it. Fixed in #14546: each reply now waits for at most a small bounded slice of that work, Stop ends it right away, and the rest is indexed in the background while the model is idle (between turns and during tool calls), so later replies usually wait for nothing.
To update: Unsloth Desktop will have an update banner. For Studio, use unsloth studio update. For Docker, use docker exec <container> unsloth-studio-update. Reopen if it still happens.

Studio: cancel + bound conversation-archive embeddings before inferen…

Follow-up: Long pre-inference stall persists in v0.1.910-beta — request to reopen #14519

Thank you for merging #14546 and for implementing bounded foreground embeddings, cancellation and idle-gap pre-embedding.

I have now retested the original long, tool-heavy conversation with its original persisted database, using the official Unsloth Desktop v0.1.910-beta on Windows. No custom QA patches were applied.

Unfortunately, the original multi-minute pre-inference stall is still reproducible.

First prompt:

Model ready at approximately 20:05:33 local time.
Chat completion started at approximately 20:05:36.
Sustained high CPU load for several minutes.
llama.cpp received the generation request at approximately 20:16:11 — around 10 minutes 35 seconds after completion start.
Prefill: 80,174 tokens in 101.19 seconds.
Total backend completion duration: 737.06 seconds (12 minutes 17 seconds).
Response completed at approximately 20:17:53.

Second prompt in the same conversation:

Total backend completion duration: 210.64 seconds (3 minutes 31 seconds).
Approximately 105 seconds before llama.cpp received the request.
Conversation recall returned four passages.
Significantly faster than the first request, suggesting that some preparation or archival work may have been completed or reused. However, the exact archive persistence state has not yet been verified.

Separate llama.cpp prompt-cache observation:

The second prompt unexpectedly required another full prefill:

Previous prompt: 80,199 tokens
New prompt:      80,418 tokens
Longest common prefix: 864 tokens
Cached tokens:   0

llama.cpp explicitly reported:

forcing full prompt re-processing due to lack of cache data

This resulted in another approximately 101-second prefill. This may be a separate prompt-construction or KV-cache reuse issue rather than part of #14546.

We will collect another py-spy profile on v0.1.910-beta to identify the remaining CPU hotspot and determine whether it is still in archive_turns / SentenceTransformer.encode or elsewhere in the pre-inference path.

The new changes appear to improve subsequent requests, but the main user-visible failure — several minutes of high CPU load before the first inference — remains.

Can you reopen this issue? I don't have permission to reopen it myself.

Thank you again for your work on this.

Update: We traced the remaining pre-inference delay using py-spy. During the delay, the backend repeatedly spends CPU time in conversation_archive._occurrences() / _same().

However, we then discovered a release mismatch: Desktop reports v0.1.910-beta, but its Python backend is unsloth==2026.10.8 and the installed conversation_archive.py does not contain the changes from merged PR #14546 (no prewarm worker, no 256-chunk foreground cap). The local file exactly matches the installed package's SHA-256 RECORD entry.

Therefore our previous timing measurements are not a valid retest of PR #14546. We'll test the latest main backend in a separate portable QA environment and report back.

Sorry for confusion!

Follow-up: Mainline archiving test results

Tested the October 24, 2026 mainline snapshot (76e94d5650e4aa9c5ffd8c81bde32c5a07f2587b) in an isolated Windows Portable QA installation.

Testing an existing large conversation (~80K tokens), we observed:

1. New archive ingestion could not be verified successfully.

Archive ingestion repeatedly fails with conversation_archive.ingest_failed, caused by the embedding server returning HTTP 503 (The KV cache is full with other conversations).

The issue persists even with RAG_EMBED_BATCH=1.

Conversation recall still retrieves 4 earlier passages, but these may originate from previously archived content. We have not observed a successful new archive ingestion cycle.

2. The previously reported ~100-second pre-inference delay remains unchanged.

The delay is reproducible on consecutive prompts. On the second prompt, inference reused ~80K cached tokens and processed only 1,377 new tokens in approximately 1.5 seconds, while the preceding wait remained around 100 seconds.

The archive ingestion failure occurs approximately 97 seconds after the request, followed almost immediately by conversation recall.

Current conclusion: The updated archiving workflow has not yet completed successfully in our tests, and the original latency issue remains unresolved.

As discussed in our previous analysis, we still suspect the bottleneck lies in the pre-inference processing path, particularly context preparation/token counting, rolling context fitting, and synchronous archive ingestion/recall. The repeated delays and embedding failures further support investigating these previously identified functions.

Phase	Local time (2026-10-21)	Cumulative CPU seconds
Baseline	15:47:46	56.4
Early request	15:47:54	58.2
Peak CPU 1	15:50:13	511.9
Peak CPU 2	15:50:19	601.4
Peak CPU 3	15:50:24	691.8
Post Stop 1	15:50:49	1142.5
Post Stop 2	15:50:55	1236.1
Post Stop 3	15:51:00	1327.1

Metric	Recorded result
Total eligible archive groups	2,549
Already archived at start	17
Pending groups	2,532
Pending chunks	5,117
Preparation	37.426s
Embedding processing	208.564s
Commit processing	14.126s
Total synchronous archive duration	264.581s (4m 24.6s)
New groups committed	2,524
Pending groups consumed via late deduplication	8
`processed_groups`	2,532 / 2,532
`complete` / `error`	true / null

Sunbelt Computer Software

PL/B Language Development and Support

Navigation Menu

Repository navigation

[Bug] Long chats stall before inference in CPU conversation-archive embeddings; Stop does not cancel archiving #14519

Summary

Direct evidence: py-spy + Windows process CPU counters

Reproduction

Code areas

Expected

Actual

Questions / possible improvements

Update: Windows CI reproduction tests and findings

Update: Reproduced a long archive stall in portable Windows QA

Update: Python profiling identifies a second bottleneck in conversation archiving (Windows portable QA, v0.1.909-beta)

Follow-up: indexed archive matcher and embedding timeout reproduced (Windows Portable QA, October 22, 2026)

Major progress on #14519: 5,117-chunk archive backfill completed, 100% reuse after restart; remaining ~35s pre-archive latency to profile

Original bottlenecks

QA changes — implementation outline (v3 → v8)

Actual v8 full-drain result

Critical post-backfill verification

Remaining latency is separate from successful archiving

Next steps

Follow-up: Long pre-inference stall persists in v0.1.910-beta — request to reopen #14519

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Sunbelt Computer Software

PL/B Language Development and Support

[Bug] Long chats stall before inference in CPU conversation-archive embeddings; Stop does not cancel archiving #14519

Description

Summary

Direct evidence: py-spy + Windows process CPU counters

Reproduction

Code areas

Expected

Actual

Questions / possible improvements

Activity

indrajeetapache commented on Oct 21, 2026

Griggelor commented on Oct 21, 2026

Update: Windows CI reproduction tests and findings

Griggelor commented on Oct 22, 2026

Update: Reproduced a long archive stall in portable Windows QA

Griggelor commented on Oct 22, 2026

Update: Python profiling identifies a second bottleneck in conversation archiving (Windows portable QA, v0.1.909-beta)

Griggelor commented on Oct 22, 2026

Follow-up: indexed archive matcher and embedding timeout reproduced (Windows Portable QA, October 22, 2026)

indrajeetapache commented on Oct 22, 2026

Griggelor commented on Oct 23, 2026

Major progress on #14519: 5,117-chunk archive backfill completed, 100% reuse after restart; remaining ~35s pre-archive latency to profile

Original bottlenecks

QA changes — implementation outline (v3 → v8)

Actual v8 full-drain result

Critical post-backfill verification

Remaining latency is separate from successful archiving

Next steps

danielhanchen commented on Oct 23, 2026

Griggelor commented on Oct 23, 2026

Follow-up: Long pre-inference stall persists in v0.1.910-beta — request to reopen #14519

Griggelor commented on Oct 24, 2026

Griggelor commented on Oct 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions