{{ message }}
Repository navigation
[Bug]: Critical Data Loss in Knowledge Graph Layer - add_edge() and link_docs() erase existing entity ntype and props #271
Description
Description
A critical data-loss bug exists in the DynamoDB Knowledge Graph / GraphRAG layer (src/dynavec/graph.py).
When relationship edges or document links are added to existing graph nodes via GraphStore.add_edge() / Dynavec.graph_add_edge() or GraphStore.link_docs() / Dynavec.graph_link(), all previously stored node attributes (ntype and props) are permanently wiped and overwritten with None and {} in DynamoDB.
Root Cause Analysis
In src/dynavec/graph.py (lines 188–206), add_node defines an unconditional UpdateExpression:
@retry()
def add_node(
self, ns: str, entity_id: str, ntype: str | None = None, props: Props | None = None
) -> None:
self._table.update_item(
Key={"pk": self._node_pk(ns, entity_id)},
UpdateExpression=(
"SET kind = :k, ns = :ns, entity_id = :eid, ntype = :t, props = :p, "
"edges = if_not_exists(edges, :empty), docs = if_not_exists(docs, :empty)"
),
ExpressionAttributeValues={
":k": "node",
":ns": ns,
":eid": entity_id,
":t": ntype,
":p": props or {},
":empty": [],
},
)- While
edgesanddocsare protected withif_not_exists(...),ntypeandpropsare unconditionally updated to:tand:p. - When
add_edge()orlink_docs()callsself.add_node(ns, src)to ensure endpoint existence,:tis passed asNoneand:pas{}. - DynamoDB replaces the node's existing
ntypewithNULLandpropswith an empty Map{}, causing irreversible entity metadata corruption and data loss.
Steps to reproduce
- Create a graph node with entity type
ntype="Person"and metadata propertiesprops={"role": "Admin", "dept": "Engineering"}usingstore.add_node(). - Create an edge from this node to another entity using
store.add_edge("default", "alice", "manages", "bob"). - Fetch the node via
store.get_node("default", "alice")and inspectntypeandprops.
Expected behavior
Existing entity metadata (ntype and props) must remain intact when relationships are added or documents are attached.
Actual behavior
ntype becomes None and props is wiped to {}:
BEFORE add_edge:
ntype: Person
props: {'role': 'Admin', 'dept': 'Engineering'}
AFTER add_edge:
ntype: None
props: {}
BEFORE link_docs:
ntype: Author
props: {'h_index': Decimal('12')}
AFTER link_docs:
ntype: None
props: {}
Environment
- Python version: 3.12+
- dynavec version: 0.1.0 (development branch)
- OS: Windows / Linux / macOS
- Installation method (
pip/uv/ other): uv / pip
Relevant code or configuration
import boto3
from moto import mock_aws
from dynavec.config import DynavecConfig
from dynavec.graph import GraphStore
with mock_aws():
session = boto3.Session(
aws_access_key_id="testing",
aws_secret_access_key="testing",
region_name="us-east-1",
)
session.client("dynamodb").create_table(
TableName="test-table",
KeySchema=[{"AttributeName": "pk", "KeyType": "HASH"}],
AttributeDefinitions=[{"AttributeName": "pk", "AttributeType": "S"}],
BillingMode="PAY_PER_REQUEST",
)
config = DynavecConfig(
vector_bucket="test-bucket", index="test-index", table="test-table", dimension=4
)
store = GraphStore(config, boto_session=session)
# 1. Create an entity node with ntype and rich metadata properties
store.add_node("default", "alice", ntype="Person", props={"role": "Admin", "dept": "Engineering"})
print("BEFORE add_edge:")
print(" ntype:", store.get_node("default", "alice").get("ntype"))
print(" props:", store.get_node("default", "alice").get("props"))
# 2. Add an outbound relationship edge
store.add_edge("default", "alice", "manages", "bob")
print("\nAFTER add_edge:")
print(" ntype:", store.get_node("default", "alice").get("ntype"))
print(" props:", store.get_node("default", "alice").get("props"))
# 3. Create another node and link documents
store.add_node("default", "charlie", ntype="Author", props={"h_index": 12})
print("\nBEFORE link_docs:")
print(" ntype:", store.get_node("default", "charlie").get("ntype"))
print(" props:", store.get_node("default", "charlie").get("props"))
store.link_docs("default", "charlie", ["doc_123"])
print("\nAFTER link_docs:")
print(" ntype:", store.get_node("default", "charlie").get("ntype"))
print(" props:", store.get_node("default", "charlie").get("props"))Additional context
Proposed Fix:
Update GraphStore.add_node() in src/dynavec/graph.py to use if_not_exists clauses for ntype and props:
@retry()
def add_node(
self, ns: str, entity_id: str, ntype: str | None = None, props: Props | None = None
) -> None:
set_clauses = [
"kind = if_not_exists(kind, :k)",
"ns = if_not_exists(ns, :ns)",
"entity_id = if_not_exists(entity_id, :eid)",
"edges = if_not_exists(edges, :empty_list)",
"docs = if_not_exists(docs, :empty_list)",
]
values: dict[str, Any] = {
":k": "node",
":ns": ns,
":eid": entity_id,
":empty_list": [],
}
if ntype is not None:
set_clauses.append("ntype = :t")
values[":t"] = ntype
else:
set_clauses.append("ntype = if_not_exists(ntype, :null)")
values[":null"] = None
if props is not None:
set_clauses.append("props = :p")
values[":p"] = props
else:
set_clauses.append("props = if_not_exists(props, :empty_map)")
values[":empty_map"] = {}
self._table.update_item(
Key={"pk": self._node_pk(ns, entity_id)},
UpdateExpression="SET " + ", ".join(set_clauses),
ExpressionAttributeValues=values,
)
Activity
fix(graph): preserve existing node ntype and props in add_node (codef…