KAFKA-20292: Avoid copying share group assignable partitions in GroupSpecBuilder by squah-confluent · Pull Request #23896 · apache/kafka · GitHub
Skip to content

KAFKA-20292: Avoid copying share group assignable partitions in GroupSpecBuilder - #23896

Open
squah-confluent wants to merge 2 commits into
apache:trunkfrom
confluentinc:squah-skip-groupspec-assignable-partitions-copy
Open

squah-confluent wants to merge 2 commits into
apache:trunkfrom
confluentinc:squah-skip-groupspec-assignable-partitions-copy

Conversation

@squah-confluent

@squah-confluent squah-confluent commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Pass the share group state partition metadata to ShareGroupSpecBuilder
and look up assignable partitions in a ShareGroupSpecImpl subclass,
instead of copying the initialized partitions into GroupSpecImpl on
every share group assignment, and again for assignor offload. The
metadata is unmodifiable, so no copy is needed.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

@squah-confluent

squah-confluent commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor Author

@airlock-confluentinc
airlock-confluentinc Bot force-pushed the squah-skip-groupspec-assignable-partitions-copy branch from 93f9b64 to b7d27bc Compare October 21, 2026 12:03
@squah-confluent squah-confluent changed the title MINOR: Avoid copying share group assignable partitions in GroupSpecBuilder KAFKA-20292: Avoid copying share group assignable partitions in GroupSpecBuilder Oct 21, 2026
squah-confluent and others added 2 commits October 21, 2026 18:22
… lazily (apache#23883)

Replace GroupSpecImpl's inverted target assignment with a BitSet of the
assigned partitions of each topic, computed from the members' current
assignments on the first call to isPartitionAssigned. Only the uniform
homogeneous assignor calls isPartitionAssigned, so other assignors no
longer compute it.

This makes build in TargetAssignmentBuilderBenchmark ~40% faster for
homogeneous groups, because isPartitionAssigned no longer looks up
partitions in the inverted target assignment's TimelineHashMaps, which
are slower than regular maps.

It also fixes a bug where the assignor saw the partitions of members
removed in the same request as still assigned. The bug is not reachable
today, since we do not compute a new target assignment when a member
leaves, only on the next heartbeat, and static member replacement keeps
the replaced member's partitions assigned.

Remove ModernGroup's inverted target assignment, since it is no longer
read. For share groups, it was never correct, because a partition can be
assigned to several members of a share group, while the inverted target
assignment holds a single member per partition.

Rename buildAssignment in TargetAssignmentBuilderBenchmark to
buildGroupSpecAndAssignment, which builds a new group spec on every
invocation, so that the cost of computing the assigned partitions is
included, and replace buildGroupSpecWithoutOffload and
buildGroupSpecWithOffload with an assignorOffload parameter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Reviewers: David Jacot <dajac@apache.org>
…SpecBuilder

Pass the share group state partition metadata to ShareGroupSpecBuilder
and look up assignable partitions in a ShareGroupSpecImpl subclass,
instead of copying the initialized partitions into GroupSpecImpl on every
share group assignment, and again for assignor offload. The metadata is
unmodifiable, so no copy is needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@airlock-confluentinc
airlock-confluentinc Bot force-pushed the squah-skip-groupspec-assignable-partitions-copy branch from b7d27bc to 4983f8b Compare October 21, 2026 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant