Summary

A generation span that finishes with usage=None -- which is how the agents SDK ends a span whose model call raised (bad API key, 429, connection error, timeout) -- crashed update_span_properties_from_generation_span_data. The exception escapes DeepEvalTracingProcessor.on_span_end, which the SDK swallows (agents/tracing/provider.py:170), so the observer is never exited: the LlmSpan stays in current_span_context and trace_manager.active_spans, its Observer stays in self.span_observers, and every later span in the process parents onto that stale, never-finished span.

Why

deepeval/openai_agents/extractors.py:155-158 (pre-fix numbering) binds input_tokens / output_tokens only inside the if usage: guard, but extractors.py:168-169 reads them unconditionally:

    usage = generation_span_data.usage
    if usage:
        output_tokens = usage.get("output_tokens")
        input_tokens = usage.get("input_tokens")
    ...
    span.input_token_count = input_tokens      # line 168
    span.output_token_count = output_tokens    # line 169

usage is None is a contractually valid input -- GenerationSpanData.usage is typed dict[str, Any] | None = None, and it is assigned only after a successful response -- so this raises UnboundLocalError. The sibling update_span_properties_from_response_span_data already initialises its optional locals to None before the same kind of guard (extractors.py:101-102); this function was missing that.

A second defect on the same code path makes the first one fatal rather than merely wrong. extractors.py:175 indexes span.metadata["invocation_params"], but an LlmSpan created by the tracing processor carries no metadata (BaseSpan.metadata: Optional[Dict[str, Any]] = None, deepeval/tracing/types.py:98). With the unbound-local crash fixed and nothing else changed, the update raises TypeError: 'NoneType' object does not support item assignment instead -- so the span still never closes. Both fixes are required together; I verified each in isolation (see Testing).

Changes

deepeval/openai_agents/extractors.py

• Initialise input_tokens / output_tokens to None before the if usage: guard, mirroring update_span_properties_from_response_span_data. A usage-less span now records input_token_count=None / output_token_count=None, matching the optional field types on LlmSpan.
• Default span.metadata to {} before assigning span.metadata["invocation_params"].

tests/test_integrations/test_openai_agents/test_generation_span_without_usage.py (new)

• Two regression tests that open a generation_span(...) and leave span_data.usage at its default None. The first asserts the LlmSpan is finalized (end_time set, model recorded, token counts None, invocation_params.model_config preserved); the second asserts current_span_context and trace_manager.active_spans are empty afterwards. They use the existing session-scoped DeepEvalTracingProcessor fixture from the directory's conftest.py, need no API key, and make no network call.

Testing

Red -- the new test against the unpatched tree (origin/main):

$ pytest tests/test_integrations/test_openai_agents/test_generation_span_without_usage.py -q
FF                                                                       [100%]
...
>       assert llm_span.end_time is not None
E       AssertionError: assert None is not None

>       assert current_span_context.get() is None
E       AssertionError: assert LlmSpan(...) is None
...
ERROR    openai.agents:provider.py:170 Error in trace processor during on_span_end
FAILED tests/test_integrations/test_openai_agents/test_generation_span_without_usage.py::test_generation_span_without_usage_is_finalized
FAILED tests/test_integrations/test_openai_agents/test_generation_span_without_usage.py::test_generation_span_without_usage_leaves_nothing_open
2 failed in 0.15s

The processor exception the SDK swallows is:

  File ".../deepeval/openai_agents/callback_handler.py", line 133, in on_span_end
    update_span_properties(current_span, span.span_data)
  File ".../deepeval/openai_agents/extractors.py", line 168, in update_span_properties_from_generation_span_data
    span.input_token_count = input_tokens
UnboundLocalError: cannot access local variable 'input_tokens' where it is not associated with a value

With only the token-local hunk applied and the metadata hunk omitted, the same span still fails to close, now with:

  File ".../deepeval/openai_agents/extractors.py", line 177, in update_span_properties_from_generation_span_data
    span.metadata["invocation_params"] = {
TypeError: 'NoneType' object does not support item assignment

Green -- both tests on this branch:

$ pytest tests/test_integrations/test_openai_agents/test_generation_span_without_usage.py -q
2 passed in 0.06s

Existing tests in the module:

$ pytest tests/test_integrations/test_openai_agents/ -q
15 failed, 2 passed, 1 skipped in 0.95s

The same directory on origin/main with the new test file absent reports 15 failed, 1 skipped -- the identical 15 test ids, each the openai SDK's own OpenAIError from building a client with api_key=None (openai/_client.py:1027). Those tests call the live OpenAI API and are gated upstream in .github/workflows/test_integrations.yml on if: ${{ env.OPENAI_API_KEY != '' }}. No new failures.

Lint/format (the repo's configured tools, pyproject.toml):

$ ruff check deepeval/openai_agents/extractors.py tests/test_integrations/test_openai_agents/test_generation_span_without_usage.py
All checks passed!

$ black --check --line-length 80 deepeval/openai_agents/extractors.py tests/test_integrations/test_openai_agents/test_generation_span_without_usage.py
All done! 2 files would be left unchanged.

Notes

I could not run the recorded-schema integration tests (test_sync.py, test_async.py, test_scenerios/) locally because they require an OPENAI_API_KEY; the change only affects the GenerationSpanData path, and the response-span extractor is untouched.

Fixes #3314