Activity

1. BlueX888 commented on Oct 11, 2026

   

   BlueX888

   on Oct 11, 2026

   Author

   More actions

   Reproduced this. Root cause: src/agents/run_internal/tool_actions.py:608 — the truthiness check if shell_output_payload: treats the valid empty payload computed at tool_actions.py:578 as absent, so control falls into the elif output_text: branch (tool_actions.py:610-618), which wraps render_shell_outputs()'s "(no output)" placeholder into the structured channel as if it were executor stdout and persists it at tool_actions.py:632.

   Minimal reproduction:

   python3 - <<'EOF'
   import asyncio
   import json
   
   from openai.types.responses import ResponseFunctionShellToolCall, ResponseOutputMessage, ResponseOutputText
   
   from agents import Agent, Runner
   from agents.testing import ScriptedModel
   from agents.tool import ShellResult, ShellTool
   
   
   def empty_output_executor(request):
       # Executor for a command that succeeded and printed nothing.
       return ShellResult(output=[])
   
   
   def shell_call(call_id="call_shell"):
       return ResponseFunctionShellToolCall(
           id="sc1",
           call_id=call_id,
           type="shell_call",
           action={"type": "shell_command", "commands": ["true"]},
           status="completed",
       )
   
   
   async def main():
       model = ScriptedModel(
           [
               [shell_call()],
               [
                   ResponseOutputMessage(
                       id="m",
                       type="message",
                       role="assistant",
                       content=[
                           ResponseOutputText(text="done", type="output_text", annotations=[], logprobs=[])
                       ],
                       status="completed",
                   )
               ],
           ]
       )
       agent = Agent(name="t", model=model, tools=[ShellTool(executor=empty_output_executor)])
       result = await Runner.run(agent, input="run true", max_turns=5)
   
       for item in result.new_items:
           if type(item).__name__ == "ToolCallOutputItem":
               print("MODEL-VISIBLE structured output:", json.dumps(item.raw_item["output"]))
               print("PERSISTED shell_output:", json.dumps(item.raw_item.get("shell_output")))
       print("final_output:", repr(result.final_output))
   
   
   asyncio.run(main())
   EOF
   

   Observed output:

   MODEL-VISIBLE structured output: [{"stdout": "(no output)", "stderr": "", "outcome": {"type": "exit", "exit_code": 0}}]
   PERSISTED shell_output: [{"stdout": "(no output)", "stderr": "", "status": "completed", "outcome": "success"}]
   final_output: 'done'
   

   Happy to open a PR with this fix, or to be assigned if you prefer to take it.
2. jbeckwith-oai commented on Oct 13, 2026

   

   jbeckwith-oai

   on Oct 13, 2026

   Collaborator

   More actions

   Thanks for the detailed report and for checking the neighboring cases! The code path does promote the display placeholder into structured stdout when ShellResult.output is empty.

   For a command that succeeds without printing anything, you can currently return a ShellCommandOutput with empty stdout/stderr and ShellCallOutcome(type="exit", exit_code=0) inside the result list. That retains the command's outcome without adding placeholder text to its stdout.

   Could you share a little more about the workflow where your executor returns no entries, and whether the placeholder has affected a downstream consumer or a model response? A small, sanitized example would help us understand the practical impact and prioritize a focused fix. Thanks again for taking the time to investigate this.
3. BlueX888 commented on Oct 14, 2026

   

   BlueX888

   on Oct 14, 2026

   Author

   More actions

   Thanks for taking a look, and for confirming the workaround. To answer both questions honestly:

   Where an executor returns no entries. Our evidence is code reading plus the scripted repro in the report — we don't have a production incident to cite, so please treat this as a design sharp edge rather than a live failure we hit. The scenario we're concerned about is user-supplied executors for local environments: ShellExecutor returns str | ShellResult, and nothing requires ShellResult.output to be non-empty, so two natural executor shapes land on output=[] for a successful quiet command (mkdir, cp, git commit -q, …):

   • chunk-per-entry executors that emit one ShellCommandOutput per captured output segment — a command that printed nothing produces zero segments;
   • result-filtering executors that only record commands which printed something or failed — a fully quiet batch returns an empty list.

   Both read as a faithful "the command ran and printed nothing", and the type system gives no hint that this is unsupported.

   Practical impact (sanitized). The fabricated entry is presented as real stdout in two places, and nothing marks the string as a placeholder in either:

   • the model-visible shell_call_output for the next turn ({"stdout": "(no output)", ...}), so the model can reasonably quote it back as the command's output;
   • the persisted raw_item["shell_output"], so a trace or log consumer that parses these entries records stdout the command never produced.

   The plain-text channel carries the same placeholder (ToolCallOutputItem.output), which is fine for human display but is exactly what leaks into the structured channel here.

   The explicit-ShellCommandOutput workaround works for us, and we're happy to leave the fix itself to you. If a focused change is on the table, the seam we'd point at: when the executor returned a structured ShellResult, an empty output list means "structured result: nothing to report", so skipping the elif output_text: fallback in that case (keeping it for plain str returns) would stop the fabrication without changing what render_shell_outputs() shows humans. Happy to send that as a PR if you'd like — otherwise we'll defer to whatever you prefer.
4. zhgao commented on Oct 17, 2026

   

   zhgao

   on Oct 17, 2026

   More actions

   Hi, I've implemented a fix for this issue and it's ready on my fork: zhgao/openai-agents-python branch fix/issue-5348.

   Compare view: main...zhgao:fix/issue-5348

   The branch has two commits ("Fix ShellTool fabricating stdout for empty structured results") which:

   • In ShellAction.execute (src/agents/run_internal/tool_actions.py), change elif output_text: to elif output_text and shell_output_payload is None: so the "(no output)" placeholder path from render_shell_outputs() only applies when there is genuinely no structured result (plain string or exception). A structured ShellResult(output=[]) now keeps output == [] with no fabricated stdout and no persisted shell_output entry.
   • Add test_shell_tool_empty_structured_result_does_not_fabricate_stdout in tests/test_shell_tool.py, asserting output == [] and no shell_output key for ShellResult(output=[]).

   However, this repository restricts pull request creation to collaborators, so I'm unable to open the PR myself. Could a maintainer please open a PR from the fix/issue-5348 branch on my fork (zhgao/openai-agents-python), or grant me collaborator access so I can submit it directly? I'm happy to address any review feedback. Thank you!
5. EngineeringWithJay commented on Oct 17, 2026

   

   EngineeringWithJay

   on Oct 17, 2026

   More actions

   Complementary offline coverage for #5348: on 9abc9182d06dbd28330af888306031269d57a175 (openai-agents 0.24.0, Python 3.12.13, openai 3.34.0), Runner + ScriptedModel reproduces (no output) as stdout in both the actual second model call and persisted raw_item["shell_output"] when the executor returns ShellResult(output=[]).

   Two fixed-behavior assertions fail; eight checks pass, consistently across two runs. Controls cover empty/nonempty strings, quiet structured entries with unspecified or successful exit outcomes, real stdout/stderr with newlines and ordering, and literal executor-provided (no output) that must not be stripped. The human-readable display placeholder is allowed to remain. A temporary one-line fallback guard made all ten checks pass twice; it was then reverted. No source fix is included, and this is synthetic evidence, not a production incident or live-provider test.

   I see a fix branch has already been offered. If complementary coverage would help, I can share these cases for a collaborator-led test-only PR, or open a test-only PR if permitted under the collaborator-only contribution policy. Otherwise, no duplicate implementation or PR is intended.

   AI disclosure: this repro, test coverage, and summary were prepared with Astra Coder (OpenAI GPT-6 Astra, via OpenCode). The reported checks were executed locally.
6. AbarnaaSree commented on Oct 18, 2026

   

   AbarnaaSree

   on Oct 18, 2026

   More actions

   Hi, I’d like to contribute to the validation of this fix. Since an implementation and complementary test coverage have already been shared, would it be okay if I review the proposed changes and check for any remaining edge cases around empty ShellResult outputs, plain-string returns, and persisted shell_output? I’ll coordinate with the existing contributors to avoid duplicating work and can help strengthen the regression tests if needed.
7. MuneebMalik244535 commented on Oct 18, 2026

   

   MuneebMalik244535

   on Oct 18, 2026

   More actions

   I have investigated this issue and confirmed the root cause in src/agents/run_internal/tool_actions.py.

   Root Cause

   In ShellAction.execute:

   raw_entries: list[dict[str, Any]] | None = None
   if shell_output_payload:
       raw_entries = shell_output_payload
   elif output_text:
       raw_entries = [{"stdout": output_text, ...}]
8. 0-Actual commented on Oct 20, 2026

   

   0-Actual

   on Oct 20, 2026 – with ChatGPT Codex Connector

   More actions

   Executable regression coverage for the empty-structured-result fix

   Building on BlueX888's report, jbeckwith-oai's response, and zhgao's existing
   proposal, here is a tested one-line correction plus a self-contained regression
   suite. The additional contribution is coverage of both runner modes, preservation
   of metadata and display behavior, and the actual next model input. This is
   corroboration and regression coverage, not a claim of original discovery.

   On October 20 I checked src/agents/run_internal/tool_actions.py at main
   64cd5dfc5a1579870dcbed0ea42b27aa84033fee: it is byte-identical to the
   openai-agents==0.24.0 file tested below. The presence/truthiness distinction is
   still relevant. Synthetic tests on Windows, Python 3.12.14:

   • Baseline: 32 tests, 28 pass and four fail. The failing cases are empty structured
     results, with/without metadata, in ordinary and streamed runs.
   • With the single predicate correction: all 32 pass, no errors or skips.
   • The eight approval/state controls already pass on the baseline; no approval
     bypass is claimed. The other 24 cases test shell-result behavior.

   The correction keeps the human-readable display placeholder while preserving
   output=[] in the retained item, its input conversion, and the next scripted
   model request. It does not invent a successful exit code for an absent entry.
   Explicit quiet success, unknown exit code, failure, timeout, multiple entries,
   plain strings (including the literal placeholder), rejection, output limits,
   executor exceptions, and provider metadata are checked.

   No shell command, live model, credential, billing experiment, or production incident
   is involved. These are bounded synthetic checks, not full SDK or security validation.
   The installed SDK was not modified during evaluation.

   Patch

   --- a/src/agents/run_internal/tool_actions.py
   +++ b/src/agents/run_internal/tool_actions.py
   @@ -624,7 +624,7 @@
                    log_tool_action_error("Shell executor failed", exc)
    
                raw_entries: list[dict[str, Any]] | None = None
   -            if shell_output_payload:
   +            if shell_output_payload is not None:
                    raw_entries = shell_output_payload
                elif output_text:
                    raw_entries = [

   Regression file

   Save the following as test_integrity.py. With openai-agents==0.24.0 installed,
   run python -m unittest -v test_integrity. The unpatched package should fail the
   four cases above. Apply the one-line correction in a separate SDK development
   checkout to test the corrected path; do not patch an installed production package.
   Our evaluation loaded that corrected class only inside the test process.

   Complete runnable regression suite

   # SPDX-License-Identifier: MIT
   # Copyright (c) 2026 Sherman G. Laing
   """Synthetic runner regressions. No shell commands or model API requests are made."""
   from __future__ import annotations
   
   import copy
   import json
   import unittest
   
   from agents import Agent, RunConfig, Runner, RunState, function_tool
   from agents.exceptions import ModelBehaviorError
   from agents.items import ToolCallOutputItem
   from agents.testing import ModelStep, ScriptedModel, assistant_message, function_call
   from agents.tool import ShellCallOutcome, ShellCommandOutput, ShellResult, ShellTool
   from openai.types.responses import (
       Response, ResponseCompletedEvent, ResponseFunctionShellToolCall,
       ResponseOutputItemDoneEvent,
   )
   
   
   async def run_agent(agent, data, mode):
       config = RunConfig(tracing_disabled=True)
       if mode == "stream":
           result = Runner.run_streamed(agent, data, max_turns=6, run_config=config)
           async for _ in result.stream_events():
               pass
           return result
       return await Runner.run(agent, data, max_turns=6, run_config=config)
   
   
   class ApprovalControls(unittest.IsolatedAsyncioTestCase):
       async def scenario(self, mode, case):
           executed = []
   
           @function_tool(needs_approval=True)
           async def record(value: int) -> str:
               """Record a synthetic value in memory."""
               executed.append(value)
               return "recorded"
   
           @function_tool
           async def permitted_sibling(value: int) -> str:
               """Record a permitted synthetic sibling action in memory."""
               executed.append(value)
               return "recorded"
   
           steps = [[function_call("record", {"value": 1}, call_id="approved-id")]]
           if case == "changed_id_with_sibling":
               steps.append([
                   function_call("record", {"value": 2}, call_id="approved-id"),
                   function_call("permitted_sibling", {"value": 3}, call_id="sibling-id"),
               ])
           steps.append([assistant_message("finished")])
           model = ScriptedModel(steps)
           agent = Agent(name="approval-controls", model=model, tools=[record, permitted_sibling])
           paused = await run_agent(agent, "synthetic input", mode)
           self.assertEqual(executed, [])
           self.assertEqual(len(paused.interruptions), 1)
           self.assertEqual(len(model.calls), 1)
           if case == "pause_without_permission":
               return
           state = paused.to_state()
           if case == "reject":
               state.reject(paused.interruptions[0])
           else:
               state.approve(paused.interruptions[0])
           # Real JSON round-trip at the supported trusted-storage boundary.
           # This does not claim that from_json authenticates untrusted state.
           state = await RunState.from_json(agent, json.loads(json.dumps(state.to_json())))
           if case == "changed_id_with_sibling":
               with self.assertRaises(ModelBehaviorError):
                   await run_agent(agent, state, mode)
               self.assertEqual(executed, [1])
           else:
               completed = await run_agent(agent, state, mode)
               self.assertEqual(completed.final_output, "finished")
               self.assertEqual(executed, [] if case == "reject" else [1])
               self.assertEqual(completed.interruptions, [])
               model.assert_complete()
   
   
   def shell_call(*, cap=None):
       action = {"type": "shell_command", "commands": ["synthetic-no-execution"]}
       if cap is not None:
           action["max_output_length"] = cap
       return ResponseFunctionShellToolCall(
           id="shell-id", call_id="shell-call", type="shell_call", action=action,
           status="completed",
       )
   
   
   class ShellIntegrity(unittest.IsolatedAsyncioTestCase):
       async def scenario(self, mode, case):
           executed = []
           cap = None
           if case == "empty":
               value = ShellResult(output=[])
               expected = []
           elif case == "empty_with_metadata":
               cap = 9
               value = ShellResult(output=[], max_output_length=4, provider_data={"label": "synthetic"})
               expected = []
           elif case == "quiet_success":
               value = ShellResult(output=[ShellCommandOutput(outcome=ShellCallOutcome(type="exit", exit_code=0))])
               expected = [{"stdout": "", "stderr": "", "outcome": {"type": "exit", "exit_code": 0}}]
           elif case == "unknown_exit":
               value = ShellResult(output=[ShellCommandOutput()])
               expected = [{"stdout": "", "stderr": "", "outcome": {"type": "exit", "exit_code": None}}]
           elif case == "failure":
               value = ShellResult(output=[ShellCommandOutput(stdout="partial", stderr="failed", outcome=ShellCallOutcome(type="exit", exit_code=7))])
               expected = [{"stdout": "partial", "stderr": "failed", "outcome": {"type": "exit", "exit_code": 7}}]
           elif case == "timeout":
               value = ShellResult(output=[ShellCommandOutput(stderr="timeout", outcome=ShellCallOutcome(type="timeout"))])
               expected = [{"stdout": "", "stderr": "timeout", "outcome": {"type": "timeout"}}]
           elif case == "multiple":
               value = ShellResult(output=[ShellCommandOutput(stdout="one"), ShellCommandOutput(stdout="two")])
               expected = [{"stdout": x, "stderr": "", "outcome": {"type": "exit", "exit_code": None}} for x in ("one", "two")]
           elif case == "plain_empty":
               value = ""
               expected = []
           elif case == "zero_limit":
               cap = 0
               value = ShellResult(output=[])
               expected = []
           elif case == "executor_exception":
               value = RuntimeError("synthetic executor failure")
               expected = [{"stdout": "synthetic executor failure", "stderr": "", "outcome": {"type": "exit", "exit_code": 1}}]
           elif case == "literal_placeholder":
               value = "(no output)"
               expected = [{"stdout": "(no output)", "stderr": "", "outcome": {"type": "exit", "exit_code": 0}}]
           elif case == "rejected":
               value = ShellResult(output=[])
               expected = None
           else:
               raise AssertionError(case)
   
           original = copy.deepcopy(value)
   
           async def synthetic_executor(request):
               executed.append(request.data.call_id)
               if isinstance(value, Exception):
                   raise value
               return value
   
           call = shell_call(cap=cap)
           first_step = [call]
           if mode == "stream":
               # Shell calls require explicit normalized events in ScriptedModel.
               response = Response(
                   id="synthetic-response", created_at=0, model="synthetic",
                   object="response", output=[call], tool_choice="auto", tools=[],
                   parallel_tool_calls=True,
               )
               first_step = ModelStep.stream([
                   ResponseOutputItemDoneEvent(type="response.output_item.done", sequence_number=0, output_index=0, item=call),
                   ResponseCompletedEvent(type="response.completed", sequence_number=1, response=response),
               ], output=[call])
           model = ScriptedModel([first_step, [assistant_message("finished")]])
           agent = Agent(name="shell-integrity", model=model, tools=[
               ShellTool(executor=synthetic_executor, needs_approval=(case == "rejected"))
           ])
           result = await run_agent(agent, "synthetic input", mode)
           if case == "rejected":
               self.assertEqual(executed, [])
               self.assertEqual(len(result.interruptions), 1)
               state = result.to_state()
               state.reject(result.interruptions[0])
               result = await run_agent(agent, state, mode)
               self.assertEqual(executed, [])
               self.assertEqual(result.final_output, "finished")
               return
   
           self.assertEqual(executed, ["shell-call"])
           if not isinstance(value, Exception):
               self.assertEqual(value, original)
           outputs = [x for x in result.new_items if isinstance(x, ToolCallOutputItem)]
           self.assertEqual(len(outputs), 1)
           item = outputs[0]
           self.assertEqual(item.raw_item["output"], expected)
           self.assertEqual(item.to_input_item()["output"], expected)
           replayed = [x for x in model.last_call.input if x.get("type") == "shell_call_output"]
           self.assertEqual(len(replayed), 1)
           self.assertEqual(replayed[0]["output"], expected)
           if expected == []:
               self.assertNotIn("shell_output", item.raw_item)
           if case == "empty_with_metadata":
               self.assertEqual(item.raw_item["provider_data"], {"label": "synthetic"})
               self.assertEqual(item.raw_item["max_output_length"], 4)
           if case == "zero_limit":
               self.assertEqual(item.raw_item["max_output_length"], 0)
           if case == "executor_exception":
               self.assertEqual(item.raw_item["status"], "failed")
           if case in ("empty", "empty_with_metadata"):
               self.assertEqual(item.output, "(no output)" if case == "empty" else "(no ")
           self.assertEqual(result.final_output, "finished")
           model.assert_complete()
   
   
   def install_cases(cls, cases):
       for mode in ("run", "stream"):
           for case in cases:
               async def test(self, selected_mode=mode, selected_case=case):
                   await self.scenario(selected_mode, selected_case)
               setattr(cls, f"test_{mode}_{case}", test)
   
   
   install_cases(ApprovalControls, ("pause_without_permission", "reject", "approved_json_resume", "changed_id_with_sibling"))
   install_cases(ShellIntegrity, ("empty", "empty_with_metadata", "quiet_success", "unknown_exit", "failure", "timeout", "multiple", "plain_empty", "literal_placeholder", "rejected", "zero_limit", "executor_exception"))
   
   if __name__ == "__main__":
       unittest.main(verbosity=2)

   Contribution license and attribution

   Prepared with Codex assistance at Sherman G. Laing's direction. The code supplied
   in this comment is offered under the repository's MIT terms, including permission
   for commercial use, modification, and redistribution. This grant covers these
   contribution copies; it does not relicense unrelated work or change attribution
   for the original issue and earlier proposals.

   MIT license for the contributed code

   MIT License
   
   Copyright (c) 2026 Sherman G. Laing
   Copyright (c) 2025 OpenAI (upstream-derived patch)
   
   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal
   in the Software without restriction, including without limitation the rights
   to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
   copies of the Software, and to permit persons to whom the Software is
   furnished to do so, subject to the following conditions:
   
   The above copyright notice and this permission notice shall be included in all
   copies or substantial portions of the Software.
   
   THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
   IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
   FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
   AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
   LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
   OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
   SOFTWARE.
   
9. EngineeringWithJay commented on Oct 20, 2026

   

   EngineeringWithJay

   on Oct 20, 2026

   More actions

   Thanks for the detailed correction and regression coverage. My independent offline packet observed the same empty ShellResult(output=[]) case: (no output) was included in both the next model input and persisted shell_output. Your is not None check targets that distinction. I haven't independently run your patch, so I won't claim it passes from my side; I won't duplicate the suite. Appreciate you taking it forward. As noted, my reproduction was AI-assisted and verified offline.
10. 0-Actual commented on Oct 21, 2026

    

    0-Actual

    on Oct 21, 2026 – with ChatGPT Codex Connector

    More actions

    Thanks, @EngineeringWithJay, for the independent observation and for being precise about what you verified. Preserving the explicit empty list in the retained item and next model input, while keeping the display placeholder, is exactly the distinction covered by the correction above. I appreciate your corroboration of the original behavior; the patch results remain the bounded synthetic results reported in my submission.