fix(eval): handle None score in LocalEvalSampler to prevent TypeError crash by nileshpatil6 · Pull Request #5420 · google/adk-python · GitHub
Skip to content

fix(eval): handle None score in LocalEvalSampler to prevent TypeError crash#5420

Open
nileshpatil6 wants to merge 2 commits intogoogle:mainfrom
nileshpatil6:fix/eval-sampler-none-score
Open

fix(eval): handle None score in LocalEvalSampler to prevent TypeError crash#5420
nileshpatil6 wants to merge 2 commits intogoogle:mainfrom
nileshpatil6:fix/eval-sampler-none-score

Conversation

@nileshpatil6
Copy link
Copy Markdown

Problem

When a metric evaluation fails (transient API error, rate limiting, JSONDecodeError), local_eval_service.py catches the exception and returns an EvaluationResult with score=None. However, local_eval_sampler.py line 292 then calls round(eval_metric_result.score, 2) unconditionally, crashing with:

TypeError: type NoneType doesn't define __round__ method

Fixes #5403

Fix

Guard the round() call with a None check:

"score": round(eval_metric_result.score, 2) if eval_metric_result.score is not None else None,

@nileshpatil6
Copy link
Copy Markdown
Author

@adk-bot
Copy link
Copy Markdown
Collaborator

adk-bot commented Apr 20, 2026

Response from ADK Triaging Agent

Hello @nileshpatil6, thank you for submitting this pull request!

To help us review it more efficiently, could you please add a testing plan section to your PR description explaining how you tested these changes?

Additionally, since this is a bug fix, could you please provide logs or a screenshot demonstrating that the TypeError is resolved after your changes?

You can find more details in our contribution guidelines. Thanks!

@adk-bot adk-bot added the eval [Component] This issue is related to evaluation label Apr 20, 2026
@nileshpatil6
Copy link
Copy Markdown
Author

Testing Plan

Manual verification:

from unittest.mock import MagicMock
from google.adk.optimization.local_eval_sampler import LocalEvalSampler

# Simulate a metric result with None score (happens when evaluation fails)
mock_metric_result = MagicMock()
mock_metric_result.score = None
mock_metric_result.metric_name = 'test_metric'
mock_metric_result.eval_status.name = 'NOT_EVALUATED'

mock_per_invocation = MagicMock()
mock_per_invocation.eval_metric_results = [mock_metric_result]

mock_eval_result = MagicMock()
mock_eval_result.eval_metric_result_per_invocation = [mock_per_invocation]

Before fix:

TypeError: type NoneType doesn't define __round__ method

at round(eval_metric_result.score, 2) when score is None.

After fix: The None score is passed through as-is (None) instead of crashing, consistent with how the upstream local_eval_service.py already uses None to represent a failed evaluation.

Reproducing the original crash (as described in issue #5403): trigger a transient API error or JSONDecodeError during metric evaluation via adk optimize — the sampler now handles it gracefully instead of crashing.

@rohityan rohityan self-assigned this Apr 20, 2026
@rohityan rohityan added the request clarification [Status] The maintainer need clarification or more information from the author label Apr 20, 2026
@rohityan
Copy link
Copy Markdown
Collaborator

Hi @nileshpatil6 , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Please fix formatting errors by running autoformat.sh

@nileshpatil6
Copy link
Copy Markdown
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval [Component] This issue is related to evaluation request clarification [Status] The maintainer need clarification or more information from the author

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TypeError in LocalEvalSampler when metric evaluation fails

3 participants