test(flagd): withhold @reinitialization, and say what was measured · open-feature/python-sdk-contrib@7bba47e · GitHub
Skip to content

Commit 7bba47e

Browse files
committed
test(flagd): withhold @reinitialization, and say what was measured
spec@fc99d5ac gates "A provider that was shut down can be initialized again" behind a new @reinitialization tag. Neither resolver declares it, and the reasoning is recorded next to each declaration because the arithmetic that suggested declaring it was wrong in an instructive way. The premise was that the scenario had been untagged at the previous pin, so it ran, and since it was not among the two failures it must have passed. It did not run. lifecycle.feature carries @lifecycle at the feature level, so every scenario in it inherits that tag, and the gate skips a scenario when any capability gating it is undeclared. Neither resolver declares LIFECYCLE, so the scenario is one of the six @lifecycle skips each resolver already reported -- absence from the failure list was absence from the run. Measured rather than reasoned about, by declaring LIFECYCLE and REINITIALIZATION together locally and letting the scenario run: in-process passes -- reuse genuinely works rpc fails -- boolean-flag resolves to the code default because grpc.py:420 raises "Cannot invoke RPC on closed channel!"; shutdown() closes the channel and the second initialize() does not rebuild it So the two resolvers genuinely differ, and RPC exhibits precisely the shape the specification's note on this capability describes: the provider evaluates against a closed connection rather than failing outright. Withheld for both regardless. For RPC because reuse does not work; for in-process because declaring REINITIALIZATION without LIFECYCLE leaves the scenario skipped on @lifecycle and the claim unexamined, which is the vacuous declaration this suite keeps the reserved tags out for. Whether to declare LIFECYCLE is a separate question and is not settled here. No KnownDeviation entry either way: Requirement 2.5.2 says a provider SHOULD revert to its uninitialized state and that "some providers MAY allow reinitialization", so declining reuse is a choice the specification offers rather than a requirement to deviate from. Tallies are unchanged from the previous pin, which is the check that no scenario silently changed status: rpc 29 passed / 2 failed / 9 skipped, in-process 28 / 3 / 9, nine skips per resolver being six @lifecycle and three @numeric-coercion. The failures remain large-integer-flag and huge-integer-flag, absent from flagd-testbed, plus in-process's boolean-flag-as-Float, which is #417. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
1 parent f0307d3 commit 7bba47e

3 files changed

Lines changed: 59 additions & 1 deletion

File tree

‎providers/openfeature-provider-flagd/tests/tck/conftest.py‎

Lines changed: 1 addition & 1 deletion

‎providers/openfeature-provider-flagd/tests/tck/test_in_process_conformance.py‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,33 @@
7373
# implementation to the lossless rule; when openfeature-flagd-core follows
7474
# it, this is declared again.
7575
#
76+
# REINITIALIZATION
77+
# New at spec@fc99d5ac, which gated the scenario "A provider that was shut
78+
# down can be initialized again" that had been untagged before it. Withheld
79+
# here even though this resolver does support reuse, which is the
80+
# interesting half of the story and was measured rather than assumed:
81+
# declaring LIFECYCLE and REINITIALIZATION together locally makes the
82+
# scenario run, and in-process passes it, while RPC fails it against a
83+
# closed channel. The two resolvers genuinely differ.
84+
#
85+
# It stays withheld because declaring it would be vacuous. The scenario
86+
# lives in lifecycle.feature, which carries @lifecycle at the feature level,
87+
# so it inherits that tag and carries both; the gate skips a scenario when
88+
# any capability gating it is undeclared, and this suite does not declare
89+
# LIFECYCLE. Declaring REINITIALIZATION alone would leave the scenario
90+
# skipped on @lifecycle and the claim unexamined -- the same
91+
# declare-what-nothing-exercises error the reserved tags below are kept out
92+
# for. Declaring LIFECYCLE is a separate question from this one and is not
93+
# settled here.
94+
#
95+
# Requirement 2.5.2 says a provider SHOULD revert to its uninitialized
96+
# state and that "some providers MAY allow reinitialization", so reuse is
97+
# permitted rather than required and withholding needs no KnownDeviation.
98+
#
99+
# Worth recording that this scenario never ran here, at this pin or the one
100+
# before it: it is one of the six @lifecycle skips each resolver reports,
101+
# not a scenario that used to pass.
102+
#
76103
# TARGETING, CACHING
77104
# Reserved in the Capability enum; no scenario carries either tag. Declaring
78105
# a capability nothing exercises would be a claim with no evidence behind it,

‎providers/openfeature-provider-flagd/tests/tck/test_rpc_conformance.py‎

Lines changed: 31 additions & 0 deletions

0 commit comments

Comments
 (0)