Repository navigation
Extra Property: store constraints as their DSL, dropping PHP serialization - #42726
Conversation
…alization The registry `constraints` column held a PHP-serialized object graph, read back with `unserialize()`. This replaces that format with the constraint DSL the back-office "Validation" field already shows, so reading a row means parsing a grammar whose class names come from a core-owned allowlist, never from the stored value. No `unserialize()` remains on this path. Grammar (both implementations, PHP and the TypeScript builder): - a composite may now carry its own options ahead of its children — `Collection(allowExtraFields: true)[ name: NotBlank ]`. This fixes a live bug: a definition holding a Collection could not be re-saved from the back office, because toNames() emitted the `Required` wrappers that fromNames() then refused; - `Required`/`Optional` become internal aliases, parseable only as Collection fields and never exposed in the builder catalog; - rendering resolves an alias through a reverse map instead of getShortName(), so a class outside the grammar is structurally unrenderable; - `callback`, `normalizer` and every `*PropertyPath` option are refused wherever a constraint is built — the round-trip alone does not catch them, they render and re-parse perfectly; - parsing is bounded in depth, length and token count, since the stored text is reachable through the database. ExtraPropertyConstraintCodec owns the storage contract: - encode() validates the object graph handed over by module code, renders it, decodes what it just produced and compares both graphs strictly. That round-trip replaces an exhaustive per-constraint option schema: anything the grammar cannot carry is refused at write time rather than dropped silently. The comparison is structural, not `==`, which would treat 5 and '5' as equal where the DSL keeps them apart; - `groups` other than the default and non-null `payload` are refused explicitly, being excluded from the rendered form by design; - reading stays fail-safe: a corrupt root constraint is dropped and logged with its registry context, its valid siblings stay active, and a front-office render never dies on a tampered row. Writing fails closed, before any DDL runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Hi, thanks for this contribution! I found some issues with the Pull Request description:
Would you mind having a look at it? This will help us understand how interesting your contribution is, thank you very much! (Note: this is an automated message, but answering it will reach a real human) |
|
This pull request seems to contain new translation strings. I have summarized them below to ease up review:
(Note: this is an automated message, but answering it will reach a real human) |
| * - reading fails safe — decodeTolerant() drops the offending root constraint, reports it, and keeps | ||
| * its valid siblings, because a corrupt row must never break a front-office render. | ||
| */ | ||
| final class ExtraPropertyConstraintCodec |
There was a problem hiding this comment.
| final class ExtraPropertyConstraintCodec | |
| final class ExtraPropertyConstraintEncoder |
Codec is a bit weird, since it holds decode/encode I'd use the same naming as in some symfony components with an Encoder suffix
There was a problem hiding this comment.
Another suggestion about this class, I wonder if it should be a service instead of using static function, and maybe use the symfony interfaces EncoderInterface DecoderInterface ?
The class can be used as a service (and not instanciated all the time) fetched via DI when possible, and for small inline usages where no container is available we can still create one on demand since the constructor will likely remain without parameters.
The drawback is that we could end up build many encoders in "static callers" so this one is just a suggestion to analyze to see if it's worth it.
Because I admit that the integration via static call in ExtraPropertyDefinition is quite easier this way, but we could also prepare and parse the constraints outside of the object, usually the repository is the right place for more advanced interpretation (like data from DB analysis), so it could be responsible for using the Encoder service. And it also means the decoding would be cached.
There was a problem hiding this comment.
OK, renamed to ExtraPropertyConstraintEncoder, and it's now a service injected into the registry and the repository. ExtraPropertyDefinition::fromRow() builds one on demand, being a static factory.
| * | ||
| * @param array<string, mixed> $row | ||
| */ | ||
| protected function hydrateDefinition(array $row): ExtraPropertyDefinition |
There was a problem hiding this comment.
I think this function should be built like the existing enriching functions enrichRowsWithShopAssociations enrichRowsWithColumnMetadata
It has the list of rows as input and returns an array of rows as output; the construction of the ExtraPropertyDefinition::fromRow is left outside of it
| * | ||
| * @return list<Constraint>|null | ||
| */ | ||
| public static function decodeTolerant(mixed $raw, ?callable $onRejected = null): ?array |
There was a problem hiding this comment.
I'm not sure about the dual function decodeStrict / decodeTolerant Maybe we should have a single decode method with a second parametrer that toggles the strictness? (if we follow SF interface this could also we an array $context)
I'm also not a fan of the passed allback function that is mainly used for:
- testing, but the way we test shouldn't force the implementation
- warning logging, but a callback for this also seems overkill
Although I do understand and I do agree that having a "tolerant" decoding that tries to return as many constraints as possible is nice. But the parts that return the errors should not work via a passed callbak function.
Maybe instead we should split this in two:
- the
decodeTolerant(or decode with parameter) simply returns the stripped list of constraints without logging and callback on the filtered ones - a new method could be added to return the invalid constraints, it only returns this info not the decoded constraints (actually assertEncodable almost already handles this but it fails on the first error, maybe this new method could be used in the existing assertEncodable and we'd still need two distrinct methods)
So the logging can still be done without messing with the decode function contract, and it can also be tested and used individually.
There was a problem hiding this comment.
Ah my bad assertEncodable is here to check the list of constraints, not precheck the string that is about to be decoded, two distinct use cases
There was a problem hiding this comment.
I uggested the encoder/decoder interfaces But maybe the DenormalizerInterface / NormalizerInterface would be a better fit Esepcially since they also implement the supportsNormalization and supportsDenormalization method that match our two use case (denormalize data from DB or input string, normalize constraints into persistable string), and they even integrate the $context we mentioned that could include some options
There was a problem hiding this comment.
Kept one function, and it now returns a DecodedConstraints object, containing the Constraints and the rejections list. Something like this:
final class DecodedConstraints
{
public function getConstraints(): ?array // list<Constraint>|null
public function getRejections(): array // list<array{index: int|string|null, reason: string}>
public function hasRejections(): bool
}
No need to identify a strict/tolerant mode, for now (unused). So I remove
| public function __construct( | ||
| protected readonly Connection $connection, | ||
| protected readonly string $prefix, | ||
| protected readonly ?LoggerInterface $logger = null, |
There was a problem hiding this comment.
I made a similar comment in a recent PR, but does the LoggerInterface need to be nullable? It should always be present, IMHO
If the reason is because of the FO container that has no LogerInterface then we should create one, we have fallback/adapters using the legacy Logger that could be used. Or maybe we don't have it yet, but creating a facade should not be complicated and it would solve the problem for good.
There was a problem hiding this comment.
You're right. done
Fixes a real defect found in review, and reshapes the API along the lines the reviewers asked for. The defect (spotted by @cnavarro-prestashop): the registry guard did not refuse everything save() refuses. assertEncodable() walked the object graph, while encode() also renders and round-trips, so a constraint such as Choice(['choices' => ['a' => 1]]) passed step 1 and threw at step 8 — after the DDL had already created its storage column. The guard is now a genuine dry-run of encode(), so the two cannot diverge again. API changes: - ExtraPropertyConstraintCodec becomes ExtraPropertyConstraintEncoder, following the Symfony naming where an Encoder also decodes; - it becomes a service with no constructor dependency, injected into the registry and the repository. ExtraPropertyDefinition::fromRow() builds one on demand, being a static factory that cannot receive injection; - the rejection callback is gone. decodeTolerant() returns a DecodedConstraints value carrying both the constraints and the rejections, so the reporting no longer travels through the method contract and stays testable on its own. The two decoding modes keep distinct names rather than a runtime flag: their failure contracts differ — decode() throws, decodeTolerant() never does — and a method name is checked when reading the call site; - the length bound is applied by a single private guard shared by both decoding paths, instead of sitting at different levels on each side. Repository and mapper: - hydrateDefinition() becomes enrichRowsWithDecodedConstraints(), taking rows and returning rows like the other enriching steps, with the construction left at the call sites; - the logger is no longer nullable: the dedicated LegacyLogger service is reachable from every container, including the hand-built front-office one, which an integration test now pins; - maxRawLength()/maxNestingDepth() become public constants; - fromNames() goes through tokenize(), which now carries the token-count bound so every caller splitting a definition gets it. ConstraintRowPresenter guards its own calls: displaying a definition must degrade to an empty builder rather than break the page. Kept as is, with the reasoning recorded: rows always carry composite_options, because the form type binds the field on every row regardless, and the registry still logs before throwing, like its five sibling steps. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up to the review discussion on the decoding API. Two findings settled it. First, the strict decode had no production caller at all: the back-office form and its data handler call ExtraPropertyConstraintMapper::fromNames() directly, so only the tolerant path was ever used. The "one method or two" question therefore answers itself — there is a single decoding, and it returns a single type. Second, Symfony 6.4's NormalizerInterface and DenormalizerInterface declare no return types, and getSupportedTypes() is not required yet. Implementing them costs two supports*() methods and an unused $type argument, and lets us keep our own precise return types (?string and DecodedConstraints) rather than widening them to mixed. So the encoder now implements both contracts, whose semantics match what it does: denormalize the value read from the registry into constraints, normalize constraints into a persistable string. encode()/decodeTolerant() become normalize()/denormalize(), and assertEncodable() becomes assertNormalizable(). The service is deliberately not tagged serializer.normalizer — nothing routes through the Serializer here, the callers are direct, and neither container autoconfigures these files. The private helper backing the strict structural comparison is renamed toComparableForm() to free the normalize() name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- The class becomes ExtraPropertyConstraintNormalizer, matching the interfaces it implements. Second rename in this PR: the first followed the earlier Encoder suggestion, before the normalizer contracts were settled on. - ExtraPropertyDefinition no longer decodes anything. fromRow() receives constraints already decoded and carries them through, so the value object stops instantiating a normalizer of its own. Nothing is lost: the repository is fromRow()'s only production caller and already decodes upstream. - assertNormalizable() is gone. The registry calls normalize() directly and discards the result, with a comment saying why: the guard has to stay a full dry-run of what save() performs. - supportsNormalization()/supportsDenormalization() now inspect the data rather than only the type, which is what the interfaces pass it for. The denormalization side examines the stored string against the grammar — bounds, allowlisted names, forbidden options — and rejects an unknown name on its name alone, building nothing to decide. They stay deliberately shallower than normalizing, and the registry keeps its dry-run rather than relying on them. A constraint can pass every name-level check and still be unrenderable — Choice(['choices' => ['a' => 1]]) is the case that motivated the earlier fix — and a boolean carries no reason, where the registry needs the message for its exception, the back-office flash and the module install error. A test pins that asymmetry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
| $logger: '@prestashop.extra_property.constraint_decode_logger' | ||
| $constraintNormalizer: '@PrestaShop\PrestaShop\Core\ExtraProperty\Validation\ExtraPropertyConstraintNormalizer' | ||
|
|
||
| prestashop.extra_property.constraint_decode_logger: |
There was a problem hiding this comment.
This one is not bound to extra property only It makes sense that we need a logger in common services, so it should be moved in src/PrestaShopBundle/Resources/config/services/adapter/common.yml
And while we're at it, the adapter service itself should be private but we can add a public alias for the interface It will work better with the autowiring for future developments, and we can calways switch the default alias later if needed
There was a problem hiding this comment.
ok, as we saw:
- adapter moved to
adapter/common.yml - public alias in
config/services/front/services_prod.yml(to have it only in FO -- BO already have a defaultLoggerInterface(mononolog)
…composites Restores tests this PR had no business touching. Replacing the constraint tests in ExtraPropertyDefinitionFromRowTest used an anchor-delimited edit that swept away five unrelated methods and a data provider, among them the default-value hydration test whose docblock pins a historical bug — a naive (string)/(bool) round-trip that lost a false default on reload. They are back verbatim from 9.2.x and pass unchanged. The LegacyLogger service moves out of the extra property config into adapter/common.yml, where it belongs: nothing about it is extra-property specific, and that file is loaded by every container. It is private, and the front office aliases LoggerInterface onto it in config/services/front/ services_prod.yml — a file the Symfony kernels never load, so their own logger binding is untouched while front-office autowiring gains the interface. Test coverage for the composite options tail, which was propagated through existing expectations without a single case exercising it: the row presenter now covers a Collection and a list composite carrying their own options, checking the split between the children tail and the options tail, and the form test covers one composite surviving a submit. Also on the normalizer tests: the round-trip over every public constraint now asserts no rejection was reported, which makes the dedicated clean-value test redundant; the Collection wrappers test states both halves of its rule, accepted as Collection fields and refused anywhere else; and the tautological registry guard test is removed, a leftover from when assertNormalizable() existed — it asserted normalize() throws, then asserted normalize() throws. The keyed-map limitation stays, with the test comment now describing it as a known limitation to revisit rather than a desirable rule. Follow-up tracked in epic PrestaShop#41422: Symfony ignores the keys of Choice::$choices, so normalizing such a map to its values would change no validation behaviour — what it changes is the strict round-trip contract, which deserves its own decision. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jolelievre
left a comment
There was a problem hiding this comment.
Thanks @Jeremie-Kiwik
|
Non-blocking notes after a full read of the diff. None of these should hold the merge, they are informational. 1. Rows written by 9.2.0-beta.1 are not migrated.
2. The flag is assigned before the 3. Two
For the rest the approach holds: the allowlist is core-owned and never taken from the stored value, the option denylist is applied independently at parse, encode and catalog level so no path bypasses it, the bounds cover a tampered row, and |
…ration Hydrating a definition whose stored constraints cannot be decoded logs the rejection. With the legacy logger that log is a PrestaShopLogger row, and ObjectModel::add() loaded every definition to validate extra properties before writing it, which decoded the same row and logged it again, without end: one corrupt row, or any row still holding a PHP-serialized blob from a 9.2 beta install, took every back-office page to a 500 and every front-office page to max_execution_time. Three independent guards: - ObjectModel::validateExtraProperties() and persistExtraProperties() return at once when no extra property value was ever set on the instance, so saving an entity without extra properties never touches the definitions repository. - ExtraPropertyDefinitionRepository reports a rejection once per row content and per instance, marking it before calling the logger, so a logger that hydrates the definitions itself finds it already reported. This also keeps the Logs page to one line per bad row. - The repository takes Psr\Log\LoggerInterface instead of the legacy adapter service: the Symfony kernels resolve it to Monolog (no database write), only the hand-built front-office container aliases it to the legacy logger.
| $allExploded = true; | ||
| foreach (ExtraPropertyConstraintMapper::tokenize($matches[1]) as [$childToken, $childLine]) { | ||
| try { | ||
| $children = ExtraPropertyConstraintMapper::tokenize($matches[1]); | ||
| } catch (ExtraPropertyException) { | ||
| continue; | ||
| } |
There was a problem hiding this comment.
Minor, and only reachable on a hand-edited database value (detailed in my conversation comment above): $allExploded is set before the try, so if tokenizing the children throws, the continue skips the row but the flag stays true. A later top-level All[...] is then presented as an opaque set-level row instead of feeding the per-language zone.
Setting the flag only once the children were actually read keeps the zone available for the next All[...]:
| $allExploded = true; | |
| foreach (ExtraPropertyConstraintMapper::tokenize($matches[1]) as [$childToken, $childLine]) { | |
| try { | |
| $children = ExtraPropertyConstraintMapper::tokenize($matches[1]); | |
| } catch (ExtraPropertyException) { | |
| continue; | |
| } | |
| try { | |
| $children = ExtraPropertyConstraintMapper::tokenize($matches[1]); | |
| } catch (ExtraPropertyException) { | |
| continue; | |
| } | |
| $allExploded = true; |
e01a6ff
|
QA of the "How to test" section on PrestaShop 9.2.0 / PHP 8.1.34 (debug mode, hummingbird), code before = merge base 68ca49a, code after = 984a78d, database reset from the same dump before each run, Steps 1 to 5 and 7 behave as described, and none of them did before the PR:
Step 6 failed, and it is the one step that matters for shops upgrading from a 9.2 beta. The loopSetting the The rejection is logged from inside the cache callback, before the item is stored. The legacy logger persists a No corruption is needed to trigger it: the PHP-serialized rows a 9.2 beta install already holds are rejected by the new decoder and therefore logged. Restoring a dump with 12 such rows (the demo module on 9.2.x) gave the same 30 s per front-office page and an unreachable back office, so the recovery path described in the PR (re-save the definition, or let the module re-register) was not reachable. The fix (e01a6ff, pushed to this branch)Three independent guards, so the loop cannot form again through any single change:
Covered by Checked after the fix: the corrupt-row scenario gives front-office pages in under 2 s and back-office pages in 2 to 3 s, one Monolog line when the back office rebuilds the cache and one Two pre-existing details seen on both codes, not part of this PR: |
… after a dropped All - prestashop.adapter.legacy.logger moves from adapter/services.yml to adapter/common.yml, so the hand-built front-office container gets the same service the Symfony kernels already had instead of a second LegacyLogger instance (prestashop.adapter.legacy.logger.shared is removed). The front-office alias of Psr\Log\LoggerInterface points at it. - ConstraintRowPresenter::rows() marks the first All[...] as exploded only once its children tokenized: an All with unrepresentable children (a hand-edited row) is dropped and the next top-level All still feeds the per-language zone instead of presenting as an opaque set-level row.
|
Thanks at @jolelievre @mattgoud @cnavarro-prestashop too, for this really deep review! |
The webservice runs on its own hand-built container (webservice/dispatcher.php), built from config/services/webservice/*.yml, which imports config/services/common.yml like the front one does. The Psr\Log\LoggerInterface alias lived in the front-office file only, so the extra property repository could not be built there and every webservice request failed. The alias now sits in config/services/common.yml, loaded by the hand-built containers only; the Symfony kernels keep Monolog. The container test covers the webservice container as well.
fab8255

ExtraPropertyDefinition::decodeConstraints()calledunserialize()withallowed_classes => true, so any autoloadable class in the shop could be instantiated from theps_extra_property_definition.constraintscolumn before theinstanceof Constraintfilter ran — its__wakeup()/__destruct()had already fired by then. Rather than hardeningunserialize(), this PR removes it from the path entirely: the column now holds the same DSL the back-office "Validation" field already displays, so reading a row means parsing a grammar whose class names come from a core-owned allowlist and never from the stored value. This also unifies the two mechanisms that used to coexist — the DSL knew how to represent a constraint and owned the whitelist, while persistence had its own, disjoint conversion with no shared rules. Details below the table.What changes
One grammar authority
ExtraPropertyConstraintMapperbecomes the single owner of the format. It gains:/^(\w+)\s*\[(.*)\]$/;ReflectionClass::getShortName(), so a class outside the grammar is structurally unrenderable rather than emitted as a name the parser would later reject;One storage contract on top of it
ExtraPropertyConstraintCodeccontains no parsing, no rendering and no class resolution — it orchestrates the mapper:encode()validates the object graph handed over by module code, renders it, decodes what it just produced and compares both graphs strictly. That round-trip is the fidelity contract: anything the grammar cannot carry is refused at write time instead of being dropped silently. The comparison is structural rather than==, which would treat5and'5'as equal where the DSL keeps them apart on purpose;decodeStrict()is used wherever a human is waiting for an answer;decodeTolerant()is used when reading a row.ConstraintRowSerializerandConstraintRowPresenterkeep their role as back-office form adapters. The presenter now delegates token splitting to the mapper instead of maintaining its own regex — that was the last duplicated grammar fragment on the PHP side.Composites can carry their own options
The grammar is extended so a composite carries its options ahead of its children:
RequiredandOptionalbecome internal aliases: the grammar needs them (Symfony generates them inside aCollection), they are accepted only asCollectionfields, and they never appear in the builder catalog.Failure policy
ExtraPropertyRegistry::register()validates constraints as its first step, before any DDL, so a rejected definition cannot leave an orphan storage column behind. A newINVALID_CONSTRAINTSreason code carries a dedicated back-office message.errorlevel with its registry context (definition id, entity, module, property, reason); its valid siblings stay active. A composite is always dropped as a whole — a partially decoded composite would silently weaken the validation it describes. Definitions are hydrated on front-office requests too, so a corrupt or tampered row must never take a page down.Removing
unserialize()is necessary but not sufficientDropping
unserialize()removes the CWE-502 family: no__wakeup()/__destruct()is triggered by stored data, and classes come from the core list rather than from the text.A text format is not safe on its own, though, and this is the part worth reviewing carefully. Several allowlisted constraints expose options Symfony invokes or resolves at validation time:
normalizer(onNotBlank,Email,Url,Uuid,Ip,Length,Regex),Choice::$callback, and the*PropertyPathfamily on 16 constraints. Those are plain strings, so they survive a DSL round-trip exactly as they survived a whitelistedunserialize(), and theis_callable()guard Symfony applies in the constructor does not run when a constraint is rebuilt from stored data.They are therefore rejected explicitly, both when parsing and when encoding. Two guarantees, deliberately independent:
normalizer,callback,*PropertyPathThe round-trip says nothing about danger: a constraint carrying
normalizerpasses it perfectly. This is the design trap to keep in mind if the code evolves.A regression test asserts the exact list of callable-typed options across the allowlisted constraints, so a future Symfony version introducing a new one fails the build instead of silently widening the surface.
A back-office bug fixed along the way
Independent of the security topic: opening a definition that carries a
Collectionconstraint and clicking Save without changing anything currently fails.toNames()emitsCollection[ name: Required[ NotBlank ] ]whilefromNames()rejectsRequired, which is absent from the list. The extended grammar handles those wrappers, and the full chain (data provider → presenter → row serializer → data handler) now round-trips.Behavior changes to be aware of
The feature is in beta and not covered by the backward compatibility promise, but these are worth stating:
unserialize()into the stable release. Re-saving a definition, or letting a module re-register its property, restores its constraints.Constraintsubclass outside the mapper's list) is rejected atModule::registerExtraProperty()with an explicit message, instead of being accepted and silently dropped later.normalizer,callbackand*PropertyPathoptions are rejected, as are non-defaultgroupsand a non-nullpayload. None of them has a meaningful use for an extra property, which validates a single scalar value.DateTime/DateTimeImmutablebounds are rejected. This is not only about representability:ExtraPropertyValueCasterhandsDATEvalues over asY-m-d H:i:sstrings, and comparing such a string against an object bound produces no violation at all — a silent false negative. Date bounds are expressed as strings.An extension point letting modules declare their own constraint classes was deliberately left out to keep this reviewable before the release. It is purely additive: nothing written today becomes invalid when it lands.
How to test
1. Supported constraints behave as before
Advanced Parameters > Extra Properties, create or edit a property and fill the "Validation" field with, for example:constraintscolumn now holds readable text rather than a serialized blob.2. A composite keeps its own options (the new grammar)
Collection(allowExtraFields: true)[ name: NotBlank ]to the "Validation" field.3. The Collection regression is gone
Collectionconstraint and click Save without editing anything: it must save successfully (this fails on9.2.xtoday).4. Unsupported input is refused with a clear message
Length(normalizer: 'trim')to the "Validation" field and save: the form must show an explicit error and leave the definition unchanged.5. No orphan column on rejection
Constraintsubclass.ExtraPropertyException, and the storage column must not exist in the corresponding*_extratable.6. A corrupt row degrades instead of crashing
constraintsvalue of a row inps_extra_property_definitionto something unparsable.errorlog line naming the definition id, entity, module and property.7. Automated tests
Notes for reviewers
ExtraPropertyConstraintMapper(grammar),ExtraPropertyConstraintCodec(new, storage contract),ExtraPropertyDefinitionRepository(hydration + logging),ExtraPropertyDefinition(decodeConstraints()reduced to one line). The rest is propagation.parseTail()handles a generic tail and the new field is one. Onlyconstraint-builder.tswas touched, to give a composite two editors — a typed one for its own options, a raw one for its children.INVALID_CONSTRAINTSis10:8and9were taken by Handle non-conventional entity naming, default value typing and the cart entity in extra properties #42538.serialize()/unserialize()call remains anywhere on this path.