Store copies of values in the memory storage client · Issue #2313 · apify/crawlee-python · GitHub
Skip to content

Store copies of values in the memory storage client #2313

Open

Description

@vdusek

MemoryDatasetClient._push_item() appends the caller's dict, and MemoryKeyValueStoreClient.set_value() stores the value object itself. Mutating the object after pushing it changes the stored data, and pushing the same object twice stores two references to one object. The file system and SQL clients store independent snapshots, and they reject non-JSON values that the memory client accepts.

Store a JSON snapshot of each dataset item, and a copy of each KVS value (or serialized bytes, once #2297 lands), so the memory client behaves like the other backends.

This will be a breaking change.

JS implementation: apify/crawlee#3812, apify/crawlee#3831

✍️ Drafted by Claude Code

Activity

  1. added this to the 2.0 milestone on Oct 13, 2026
  2. added
    t-toolingIssues with this label are in the ownership of the tooling team.
    debtCode quality improvement or decrease of technical debt.
    on Oct 13, 2026
  3. janbuchar commented on Oct 15, 2026

    @janbuchar
    Collaborator

    Frankly this only breaks code that relied on unspecified behavior, so breaking change is a big stretch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    debtCode quality improvement or decrease of technical debt.t-toolingIssues with this label are in the ownership of the tooling team.

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions