{{ message }}
fix: Bedrock reranker credential chain fallback and eval script compatibility with LTM backend split - #1585
Open
ssvaddiparthy wants to merge 1 commit into
Open
Conversation
ssvaddiparthy
force-pushed
the
fix/aws-eval-reranker
branch
3 times, most recently
from
September 6, 2026 06:30
66178bb to
09bbdf7
Compare
…pdate eval scripts for LongTermMemory backend split - Make AmazonBedrockRerankerConf inherit AWSCredentialsMixin so AWS credentials are optional with env-var fallback, enabling SSO/profile/login-based auth - Update RerankerManager to only pass explicit credentials to boto3 when set, falling back to the default credential chain otherwise - Fix agent_utils.py to use DeclarativeBackendParams after MemMachine#1395 changed LongTermMemoryParams to a backend-discriminated union - Add tests for no-creds and env-creds paths, remove obsolete required-field test
ssvaddiparthy
force-pushed
the
fix/aws-eval-reranker
branch
from
September 6, 2026 06:32
09bbdf7 to
1d5aab4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Purpose of the change
Fix two issues that prevent MemMachine evaluation benchmarks from running out of the box:
LongTermMemoryParamsclass which was replaced by a backend-discriminated union in Wire EventMemory as backend for LongTermMemory #1395.Description
Bedrock credential chain fix (
reranker_conf.py+reranker_manager.py):AmazonBedrockRerankerConfpreviously defined its ownaws_access_key_idandaws_secret_access_keyfields as required (Field(...)), meaning any config using the Bedrock reranker had to include explicit credentials — SSO, AWS CLI login, and profile-based auth couldn't work. This PR replaces the hand-rolled fields withAWSCredentialsMixin(which defaults toNonewith env-var fallback) and updatesRerankerManager._build_amazon_bedrock_rerankerto only pass credentials toboto3.client()when they're actually set. When omitted, boto3 falls through to its default credential chain (~/.aws/credentials,~/.aws/config, SSO, instance metadata).Eval script fix (
agent_utils.py):PR #1395 ("Wire EventMemory as backend for LongTermMemory") changed
LongTermMemoryParamsfrom a single concrete class to aDiscriminator-based union ofDeclarativeBackendParams | EventBackendParams. The evaluation helperinit_memmachine_paramsstill used the oldLongTermMemoryParams, which no longer accepts the fields directly. This PR updates it to useDeclarativeBackendParams(backend="declarative", ...).Tests (
test_reranker_conf.py):test_valid_amazon_bedrock_reranker_conf_without_explicit_credentials— verifies the config accepts no AWS credentials when env vars are unsettest_valid_amazon_bedrock_reranker_conf_uses_env_credentials— verifies env-var resolution populates credentials correctlytest_missing_required_field_in_bedrock_reranker— this tested the old required-field behavior which no longer appliesFixes/Closes
None
Type of change
How Has This Been Tested?
Lint (all changed files):
Unit tests (config + manager):
End-to-end eval (LoCoMo benchmark, retrieval_agent mode):
Tested the "no keys in config" path using a personal AWS account via boto3's default credential chain. The Bedrock reranker (
amazon.rerank-v1:0,us-west-2) resolved credentials automatically and completed successfully — noAccessDeniedException, noValidationError. All 10 conversation groups processed with the retrieval agent routing queries throughMemMachineAgent,SplitQueryAgent, andChainOfQueryAgentas expected.