Qdrant offers two ways to separate tenants, and they have opposite cost profiles. This issue tracks committing to the scalable one and finishing it, rather than leaving both half-supported.
Measured against Qdrant 1.19.0, one hundred tenants each way:
|
shard key per collection |
payload partitioning |
| admitting a tenant |
450 ms, explicit API call |
no call at all |
| 100 tenants |
45.0 s |
0.3 s including 10,000 points |
| segments |
505 |
5, and still 5 after delete and reuse |
| deleting one tenant |
O(1) drop |
9 ms filter-delete for 100 points |
| fencing of stale writes |
yes, Shard key "t1" not found |
none |
| cluster mode |
required, plus a bootstrap --uri |
not required |
Payload partitioning is the only one that scales, because a tenant is a value rather than a physical structure. Everything below follows from choosing it.
1. Stop creating a shard key per logical collection
QdrantConf.is_distributed switches to custom sharding and gives every logical collection its own shard key:
sharding_method=models.ShardingMethod.CUSTOM if self._is_distributed else None
...
if self._is_distributed:
await self._ensure_shard_key(entry.native_collection_name, entry.partition_key)
The cost is per logical collection. A deployment serving thousands of tenants — a tenant being a user, or a group of users — has at least that many logical collections and in practice more, since a tenant's memory is typically partitioned further. At a thousand logical collections that is ~7.5 minutes of shard-key creation and ~5,000 segments; at ten thousand, ~75 minutes and ~50,000 segments, before any data is written.
Qdrant's own guidance recommends against per-tenant physical structures: "Creating a separate collection for each tenant is rarely the most efficient approach. Each collection carries its own resource overhead."
The flag's docstring sells the O(1) delete, which is genuine, but it is paid for per tenant. Note the two mechanisms are layered correctly — the payload partition key is written and filtered in both modes, with the shard key passed additionally — so this is a policy problem, not a correctness one.
If a container ever needs splitting, the right lever is shard_number > 1 with automatic sharding: a provisioning knob, no custom sharding, no per-tenant cost.
2. Retire the per-process lock table
_name_locks serialises lifecycle operations within one process, which was the only arbitration available before a cross-process registry existed. It also leaks:
# Keyed by client so locks are garbage-collected when the client is.
_name_locks: ClassVar[
WeakKeyDictionary[AsyncQdrantClient, defaultdict[tuple[str, str], asyncio.Lock]]
] = WeakKeyDictionary()
The outer mapping is weak, but the inner defaultdict gains a lock per (namespace, name) the process ever touches and never drops one, for the client's lifetime. Small individually, unbounded in the number of logical collections a process serves, and easy to miss because the comment reads as though the lifetime question is settled.
Once the registry arbitrates lifecycle across processes (#1524, #1525, delivered by #1526 and #1527), these locks are vestigial and should go rather than be bounded.
3. Recover what payload partitioning gives up
Choosing it means giving up the two things the shard-key path provided, both of which have to be built back:
Tiered multitenancy stays available
Promoting a single oversized tenant to its own shard key, while the long tail stays payload-partitioned, is Qdrant's third documented approach and remains open. sharding_method is immutable per collection — verified, creating a shard key on an auto-sharded collection fails with "Shard Key cannot be created with Auto sharding method" — but since registry entries pin their native collection, promotion later means provisioning a new custom-sharded collection and directing new collections to it, leaving existing ones untouched. Nothing here forecloses it.
Qdrant offers two ways to separate tenants, and they have opposite cost profiles. This issue tracks committing to the scalable one and finishing it, rather than leaving both half-supported.
Measured against Qdrant 1.19.0, one hundred tenants each way:
Shard key "t1" not found--uriPayload partitioning is the only one that scales, because a tenant is a value rather than a physical structure. Everything below follows from choosing it.
1. Stop creating a shard key per logical collection
QdrantConf.is_distributedswitches to custom sharding and gives every logical collection its own shard key:The cost is per logical collection. A deployment serving thousands of tenants — a tenant being a user, or a group of users — has at least that many logical collections and in practice more, since a tenant's memory is typically partitioned further. At a thousand logical collections that is ~7.5 minutes of shard-key creation and ~5,000 segments; at ten thousand, ~75 minutes and ~50,000 segments, before any data is written.
Qdrant's own guidance recommends against per-tenant physical structures: "Creating a separate collection for each tenant is rarely the most efficient approach. Each collection carries its own resource overhead."
The flag's docstring sells the O(1) delete, which is genuine, but it is paid for per tenant. Note the two mechanisms are layered correctly — the payload partition key is written and filtered in both modes, with the shard key passed additionally — so this is a policy problem, not a correctness one.
If a container ever needs splitting, the right lever is
shard_number > 1with automatic sharding: a provisioning knob, no custom sharding, no per-tenant cost.2. Retire the per-process lock table
_name_locksserialises lifecycle operations within one process, which was the only arbitration available before a cross-process registry existed. It also leaks:The outer mapping is weak, but the inner
defaultdictgains a lock per(namespace, name)the process ever touches and never drops one, for the client's lifetime. Small individually, unbounded in the number of logical collections a process serves, and easy to miss because the comment reads as though the lifetime question is settled.Once the registry arbitrates lifecycle across processes (#1524, #1525, delivered by #1526 and #1527), these locks are vestigial and should go rather than be bounded.
3. Recover what payload partitioning gives up
Choosing it means giving up the two things the shard-key path provided, both of which have to be built back:
facetreturns top-N approximate counts, not a listing — so they must be recorded at deletion time and purged. Tracked in Qdrant integration never reclaims native collections or records under dead partition keys #1565.Tiered multitenancy stays available
Promoting a single oversized tenant to its own shard key, while the long tail stays payload-partitioned, is Qdrant's third documented approach and remains open.
sharding_methodis immutable per collection — verified, creating a shard key on an auto-sharded collection fails with "Shard Key cannot be created with Auto sharding method" — but since registry entries pin their native collection, promotion later means provisioning a new custom-sharded collection and directing new collections to it, leaving existing ones untouched. Nothing here forecloses it.