{{ message }}
Repository navigation
feat(simulate): store and show sub-actors and their turns - #3539
Draft
KarthikAvinashFI wants to merge 22 commits into
Draft
KarthikAvinashFI wants to merge 22 commits into
KarthikAvinashFI wants to merge 22 commits into
Conversation
…actors and sub-actor transcript turns
…or turns and injected tool faults
…rio's sub-actors and count the cast correctly
…sub-actor setting
…' into feat/TH-8651-sub-actors
KarthikAvinashFI
changed the base branch from
dev
to
fix/TH-8411-builder-checks-build
October 16, 2026 20:08
…' into feat/TH-8651-sub-actors
…' into feat/TH-8651-sub-actors
… one sub-actor label
…etting before building
…ctors panel and read faults from the trace
…ored as assistant-forwarded-call
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

1. Why
The harness can now give an environment a cast of sub-actors: people on the caller's side who are neither the agent nor the person it serves, such as a voice near the person, someone who takes the phone, the caller's own phone menu or a second chat participant (companion PR below). The platform has to switch that on per deployment, keep what the harness produces, and show it: which sub-actors a world has, which scenarios bind them, and who said what in a call.
2. What changed
futureagi/tfc/settings/settings.pyALK_SUB_ACTORS(default off)futureagi/simulate/services/hosted_harness_gateway.pyHARNESS_SUB_ACTORS=1to the build, run and builder chat sandboxes when the setting is on; readscast.jsonfrom the live sandbox, the authoring archive and the bundle, and returns it as acaststage output ({"primary": {...}, "sub_actors": [...]}) with a summary that counts sub-actors correctlycastoutput when the environment has nonefutureagi/simulate/models/hosted_harness.py, migration0098_sub_actorsHostedHarnessScenario.sub_actors(JSON, nullable): the scenario's bindings as[{"id": ..., "trigger": {...}}]futureagi/simulate/services/harness_scenarios.pysub_actors; rows return it; asub_actorsfilter (categoryconditions) by id ornone(the person alone), offered only when some scenario binds a sub-actorfutureagi/simulate/models/test_execution.py, migration0098_sub_actorsCallTranscript.SpeakerRole.SUB_ACTORandCallTranscript.speaker_name(blank by default)futureagi/simulate/models/chat_message.py,pydantic_schemas/chat.py, migration0098_sub_actorssub_actorchat rolefutureagi/simulate/services/hosted_harness_ingestion.py,alk_simulate_ingestion.pysub_actortranscript turn is stored with its role and name, and a chat sub-actor message is its own row; both writers keep a speaker name only onsub_actorturnsuserandassistant, so sub-actor turns stay out of themfutureagi/simulate/serializers/alk_simulate_ingestion.py,serializers/test_execution.pyspeaker_nameaccepted on ingestion and returned on transcriptsfutureagi/simulate/tasks/alk_sim.py,utils/chat_simulation.pyfutureagi/simulate/services/harness_environment.py,run_results_v3.pyactor: the sub-actor the sub-goal is about, orprimaryfor the person served; the results cache key is bumped so cached columns pick it upfutureagi/simulate/views/harness_environment.py,serializers/harness_environment.pysub_actors: whether the setting is on, so the frontend gates template casts on the same switchfrontend/src/api/simulate-environments/environment.js,scenarios.js,runDetail.jssubActors; a turn'sspeakerNamefrontend/src/components/VoiceDetailDrawerV2/TranscriptView.jsx,transcriptUtils.jsfrontend/src/sections/simulate/environments/workspace/...frontend/src/api/simulate-environments/runCalls.js,workspace/runs/detail/trace/RunTraceTable.jsxfrontend/src/api/simulate-environments/_fixtures/actors.js,_fixtures/prebuiltEnvironments.js,environments.js,cards/TemplateBuildPanel.jsxapi_contracts/openapi/swagger.json,frontend/src/api/contracts/*.generated.js,frontend/src/generated/api-contracts/*sub_actors; the MCP tool manifest (futureagi/mcp_server/catalog/tools.generated.json) regenerated to match3. Behavior callouts
ALK_SUB_ACTORSunset the sandboxes never receiveHARNESS_SUB_ACTORS, so builds and runs are unchanged, and adopted templates bring no cast.0098_sub_actors(scenario bindings, transcript role and speaker name, chat role). It is additive; no data is rewritten.held=Truewith a "Not applicable" reason, so it counts as held in pass rates. There is no separate not-applicable verdict.prebuilt.jsandenvironments.js, so the per-vertical casts live there and follow the list response'ssub_actorsswitch. When the templates endpoint lands it should return the samecastshape.speaker_role = "sub_actor"with aspeaker_name; rows written before this PR havespeaker_name = "". Readers that switch onuser/assistantsee the new value only for calls that bound a sub-actor.4. Tests
test_an_environment_with_a_cast_shows_it_and_one_without_shows_nonesimulate/tests/test_hosted_harness_gateway.pytest_a_suite_indexes_each_scenarios_sub_actors_and_filters_by_themsimulate/tests/test_harness_sub_actors.pynone, negated; field countstest_a_suite_with_no_sub_actors_offers_no_sub_actor_filtertest_a_sub_actor_turn_is_stored_with_its_nametest_a_second_chat_participant_is_its_own_message_with_its_nametest_the_sandboxes_are_told_about_sub_actors_only_when_the_setting_is_ontest_the_environments_list_says_whether_sub_actors_are_ontest_a_sub_goal_column_names_the_actor_it_is_aboutsimulate/tests/test_analytics_functional.pyactorstageOutputsToWorld,scenarioFromApi,mapCallDetail,functionCallTranscriptRows,subGoalColumnsFor, template castsfrontend/src/api/simulate-environments/__tests__/subActors.test.js.../scenarios/__tests__/ScenarioTable.test.jsx.../cards/__tests__/TemplateBuildPanel.test.jsxThe touched backend suites (
test_harness_sub_actors.py,test_hosted_harness_gateway.py,test_hosted_harness_channels.py,test_alk_simulate_ingestion.py,test_harness_scenario_amend.py,test_harness_scenario_labels.py,test_analytics_functional.py,test_run_results_v3_facets_catalog_parity.py,test_harness_provider.py,test_ended_reason.py): 722 passed; the one failure,test_voice_simulator_uses_single_dataset_language, fails the same way ondev. The frontendsimulate-environmentsAPI andsimulate/environmentssection suites: 1571 passed, 2 skipped.makemigrations --checkreports no changes, andyarn contracts:checkand the MCP manifest check pass. CI does not run on this stacked PR until it targetsdev.Tested end to end
On a local platform with this branch,
ALK_SUB_ACTORS=trueand the companion ALK branch: one voice agent (ride booking, real calls) and one chat agent (order status), each built fresh.castoutput lists each sub-actor with its persona and origin, for the built cast and for one added later in the builder chatsub_actorson the two bound scenarios, null on the other eightsub_actors=<id>returned the one scenario binding it;sub_actors=nonethe eight withoutspeaker_role = sub_actorwithspeaker_name("Alex", "Taylor"), in time order between the caller's and the agent's turnsactor: friend_in_thread; the restprimarysub_actors: trueFound and fixed while testing: a sub-actor added from the builder chat never reached the environment, because checkpoint promotion only copied the contract, scenarios, sub-goals and coverage outputs. It now copies the cast as well (
test_a_cast_added_in_the_builder_chat_reaches_the_environment, which fails without the fix).5. Commands to run the tests
Terminal output
<image: backend and frontend test output>
6. Steps to test on local
ALK_SUB_ACTORS=truefor the backend and run it with the companion ALK branch.Sub-actors; confirmnoneshows only the scenarios without one.ALK_SUB_ACTORS, rebuild, adopt the template again, and confirm none of the above appears.<video: actors panel, scenario filter, and a call with a sub-actor turn>
7. Scope in / scope out
ui_event,page_content,repo_change,peer_agent): they need harness adapters for those agent kinds firstE2E: exempt (harness-gap sub-actors are off by default and only appear in hosted harness runs)
8. Design choices
user, so attribution survives to evals and metrics stay limited to the person and the agent.Companion PR: future-agi/agent-learning-kit#177
Linear
Closes TH-8651