Describe the bug

Garnet accepts and successfully checkpoints a non-inline key, but cannot recover that checkpoint. Restart fails with a NullReferenceException from LogRecord.get_KeyBytes() in RecoverFromPage, and the server exits without listening.

This reproduces with ordinary SET and HSET, with or without --storage-tier. It does not require cluster mode, custom modules, concurrent requests, or injected corruption. All writes and both SAVE commands return success before restart.

The failed server also exits with code 0 after logging Unable to initialize server due to exception, which may hide the startup failure from a service manager.

Steps to reproduce the bug

The script uses fresh containers, preserves the server's writable layer across docker restart, and cleans up everything it creates.

#!/usr/bin/env bash
set -euo pipefail

suffix="$$"
network="garnet-overflow-key-${suffix}"
server="garnet-overflow-key-server-${suffix}"
client="garnet-overflow-key-client-${suffix}"

cleanup() {
  docker rm -f "$client" "$server" >/dev/null 2>&1 || true
  docker network rm "$network" >/dev/null 2>&1 || true
}
trap cleanup EXIT

docker network create "$network" >/dev/null
docker run -d --name "$client" --network "$network" redis:7-alpine sleep 300 >/dev/null
docker run -d --name "$server" --network "$network" --network-alias garnet \
  ghcr.io/microsoft/garnet:2.1.8 \
  --aof --recover --fail-on-recovery-error true \
  --index 16m --memory 64m --page 1m --segment 16m \
  --checkpointdir /data >/dev/null

for attempt in $(seq 1 50); do
  if [ "$(docker exec "$client" redis-cli -h garnet PING 2>/dev/null)" = PONG ]; then
    break
  fi
  sleep 0.1
done

docker exec "$client" redis-cli -h garnet SET seed value
docker exec "$client" redis-cli -h garnet SAVE
key="$(python3 -c 'print("k" * 1024)')"
docker exec "$client" redis-cli -h garnet HSET "$key" field value
docker exec "$client" redis-cli -h garnet SAVE

docker restart --time 2 "$server"
sleep 2
docker inspect --format 'status={{.State.Status}} exitCode={{.State.ExitCode}}' "$server"
docker logs "$server" 2>&1 | grep -E 'Error during recovery|get_KeyBytes|RecoverFromPage|Unable to initialize'

Observed result:

OK
OK
1
OK
status=exited exitCode=0

Error during recovery of store; storeVersion = 0;
System.NullReferenceException: Object reference not set to an instance of an object.
   at Tsavorite.core.LogRecord.get_KeyBytes() in .../Allocator/LogRecord.cs:line 1033
   at Tsavorite.core.TsavoriteKV`2.RecoverFromPage(...) in .../Recovery/Recovery.cs:line 1310
   at Tsavorite.core.TsavoriteKV`2.ProcessReadSnapshotPage(...) in .../Recovery/Recovery.cs:line 1221
   at Tsavorite.core.TsavoriteKV`2.RecoverSnapshotPages(...) in .../Recovery/Recovery.cs:line 1142
   at Tsavorite.core.TsavoriteKV`2.RecoverHybridLogFromSnapshotFileAsync(...) in .../Recovery/Recovery.cs:line 983
Unable to initialize server due to exception: Object reference not set to an instance of an object.

Additional isolated cells:

The exact key-length boundary was not measured.

Expected behavior

The key and value survive checkpoint recovery and are readable after restart. If recovery prevents startup, the server process exits with a nonzero status.

Release version

2.1.8 (ghcr.io/microsoft/garnet@sha256:20d9f41b02bcdf8aae1ad734fb7e31ef98b06efffdfa487eeea49687bdd3ff55); also reproduced on 2.1.7.

OS version

Docker Desktop 29.7.2, Linux arm64 containers.

Additional context

The failure appears to come from using an address-only LogRecord while rebuilding the hash index. RecoverFromPage constructs new LogRecord(pageStartPhysicalAddress + recordOffset) and then calls storeFunctions.GetKeyHashCode64(logRecord). The address-only constructor explicitly says it must not be used when an overflow allocator is needed. It leaves objectIdMap unset, while the non-inline key getter dereferences objectIdMap.GetOverflowByteArray(...).

This explains the stack and the short-key/long-key contrast. Recovery needs the page's object map when hashing a non-inline key; skipping the record would lose data. The same address-only construction is present on main at c323bbf7aa2783c1e9981edd387437d2618ff750 (source inspection only; that revision was not executed).