feat(provider-tck): add the Python conformance suite for OpenFeature … · open-feature/python-sdk-contrib@619c20b · GitHub
Skip to content

Commit 619c20b

Browse files
aepfliaepfli
committed
feat(provider-tck): add the Python conformance suite for OpenFeature providers
A conformance suite any Python provider can adopt to verify it implements the provider contract of the specification, and the Python implementation of the cross-language suite defined in Appendix F. It runs the same Gherkin, the same canonical flag set and the same control API as the Go and Java implementations. It uses pytest-bdd, the runner the flagd provider and the flagd testkit already use, so an adopting package gains no new test framework. Adoption is one fixture and one call. The step definitions ship as a pytest plugin registered through a pytest11 entry point, so there is no conftest.py to write and nothing to import for the vocabulary - pytest-bdd resolves steps through the fixture system, and fixtures from an installed plugin are visible everywhere. The feature files and flag set are packaged with the distribution, so adopting needs no git submodule. Capability gating uses pytest.skip from an autouse fixture, so a scenario whose capability was not declared is reported as skipped with the reason attached rather than silently passing. The gate keys off the node's markers rather than its requested fixtures: pytest-bdd resolves a step's fixtures lazily, so tck_config is not in request.fixturenames at setup time, and guarding on that silently disabled the gate. Two self-test suites, plus unit tests for what the Gherkin cannot assert about itself: the SDK's InMemoryProvider, and the TCK's own updatable one. The second exists because the first cannot exercise the configuration-change path at all. Findings, both confirmed by running the suite: * A boolean satisfies an Integer request. The client type-checks with isinstance(value, int) and bool subclasses int in Python, so boolean-flag requested as an Integer returns True with reason STATIC and no error code. This is Python-specific - the identical scenario passes in every other language - which is a fair argument for having more than one implementation. Tracked as open-feature/python-sdk#619, and marked xfail(strict=True) so it stays visible and un-hides itself once fixed. * InMemoryProvider cannot update its flag set, which Appendix A requires of an SDK in-memory provider. Only half the machinery is missing, since AbstractProvider already supplies emit_provider_configuration_changed, so ControllableInMemoryProvider is a small subclass rather than a reimplementation and should port back as a method. Tracked as open-feature/python-sdk#620. Verified locally: 56 passed, 7 skipped, 2 xfailed; ruff and mypy --strict clean. Part of open-feature/spec#417 Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
1 parent 8416d9d commit 619c20b

29 files changed

Lines changed: 3086 additions & 1 deletion

‎.release-please-manifest.json‎

Lines changed: 2 additions & 1 deletion

‎pyproject.toml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ dependencies = [
1616
"openfeature-flagd-api",
1717
"openfeature-flagd-core",
1818
"openfeature-flagd-api-testkit",
19+
"openfeature-provider-tck",
1920
]
2021

2122
[dependency-groups]
@@ -43,6 +44,7 @@ openfeature-provider-unleash = { workspace = true }
4344
openfeature-flagd-api = { workspace = true }
4445
openfeature-flagd-core = { workspace = true }
4546
openfeature-flagd-api-testkit = { workspace = true }
47+
openfeature-provider-tck = { workspace = true }
4648

4749
[tool.uv.workspace]
4850
members = [

‎release-please-config.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,15 @@
9999
"extra-files": [
100100
"README.md"
101101
]
102+
},
103+
"tools/openfeature-provider-tck": {
104+
"package-name": "openfeature-provider-tck",
105+
"bump-minor-pre-major": true,
106+
"bump-patch-for-minor-pre-major": true,
107+
"versioning": "default",
108+
"extra-files": [
109+
"README.md"
110+
]
102111
}
103112
},
104113
"changelog-sections": [

‎tools/openfeature-provider-tck/LICENSE‎

Lines changed: 201 additions & 0 deletions

0 commit comments

Comments
 (0)