You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug]: RealtimeAgent split-transcript merge replaces the previous user turn's content, dropping attached DataBlocks #3237
I have searched the existing issues and discussions, and this is not a duplicate.
This is a bug, not a usage question. (For questions, please use Discussions instead.)
Background / Description
RealtimeAgent._merge_user (src/agentscope/agent/_realtime/_agent.py:835) runs when a settled transcript continues the previous user turn — e.g. endpointing split one sentence in two, and the second half arrives inside the TurnAggregator merge window (800 ms by default). Its docstring says it appends text to the previous user turn, but the implementation replaces the whole content of that turn with a single TextBlock:
Every non-text block of that turn is destroyed. User messages are documented and validated to carry DataBlocks (_assert_user_content_blocks, src/agentscope/message/_base.py:32-38, and the UserMsg docstring states that only TextBlock and DataBlock are allowed), and RealtimeAgent.send appends any Msg handed to it to state.context (src/agentscope/agent/_realtime/_agent.py:528). So a photo a user attached between the two halves of a split utterance silently disappears from the conversation context once the second half settles.
Expected: merging the second half should append its text to the previous user turn while leaving the attached blocks in place, the way Msg.append_event handles TOOL_RESULT_TEXT_DELTA (src/agentscope/message/_base.py:432-435).
Error Messages
No exception — the loss is silent. agent.state.context[-1].content changes from [TextBlock, DataBlock] to [TextBlock].
Steps to Reproduce
Code (repro.py; it drives the internal transcription handler directly so the split is deterministic without audio hardware):
importasyncioimportbase64fromagentscope.agentimportRealtimeAgentfromagentscope.messageimportBase64Source, DataBlock, TextBlock, UserMsgfromagentscope.realtimeimport_eventsasmeasyncdefmain() ->None:
agent=RealtimeAgent("friday", "Be brief.", None)
# Half 1 of a split turn: endpointing settles early.agent._on_transcription(
me.InputTranscriptionEvent(item_id="u1", text="帮我订张机票"),
)
# Between the halves the user attaches a photo (RealtimeAgent.send# appends any Msg to state.context).agent.state.context.append(
UserMsg(
name="user",
content=[
TextBlock(text="这是行程参考图"),
DataBlock(
source=Base64Source(
data=base64.b64encode(b"png-bytes").decode(),
media_type="image/png",
),
name="trip.png",
),
],
),
)
print("context before merge:")
formsginagent.state.context:
print(" ", [type(b).__name__forbinmsg.content])
# Half 2 arrives inside the 800 ms merge window.agent._on_transcription(
me.InputTranscriptionEvent(item_id="u2", text="去上海的"),
)
print("context after merge:")
formsginagent.state.context:
print(" ", [type(b).__name__forbinmsg.content])
if__name__=="__main__":
asyncio.run(main())
Run: python repro.py
Output on current main:
context before merge:
['TextBlock']
['TextBlock', 'DataBlock']
context after merge:
['TextBlock']
['TextBlock']
The photo's DataBlock is gone from the context after the second half settles; expected the merged turn to keep [TextBlock, DataBlock] plus the appended text.
It's yours, @ffffff55. If there is no pull request and no word from you by 2026-10-25 the claim is released so nobody is blocked — a comment here renews it.
Activity
ffffff55 commentedon Oct 11, 2026
I would like to work on this — I will follow up with a fix and a regression test shortly.
github-actions commentedon Oct 11, 2026
It's yours, @ffffff55. If there is no pull request and no word from you by 2026-10-25 the claim is released so nobody is blocked — a comment here renews it.