RDF export names the rule behind a derived fact but not what the rule means · Issue #902 · deeplethe/utopia · GitHub
Skip to content

RDF export names the rule behind a derived fact but not what the rule means #902

Closed

Description

@ansonnmm

Problem

The supported RDF export (ADR 0020, clarified in #550) links every derived
fact to the rule that produced it. Verified on 9f50e3c
(emit_derived in crates/utopia-server/src/rdf.rs, fed by derived_page
in crates/utopia-store/src/export.rs), a derived fact serializes as:

…:derived:{id}  a rdf:Statement ;
    rdf:subject         …:entity:{id} ;
    rdf:predicate       <relation IRI> ;
    rdf:object          …:entity:{id} | "literal" ;
    utopia:derived      true ;
    schema:validFrom    … ;  schema:validThrough … ;
    prov:generatedAtTime … ;  prov:invalidatedAtTime … ;
    utopia:confidence   … ;
    prov:wasGeneratedBy …:rule:{rule_id} ;
    prov:used           …:fact:{premise_id} , …:derived:{premise_id} , …

…:rule:{rule_id}  a prov:Activity ;
    rdfs:label  "Gas-bearing well"     # attribute_rules.name
    # or        "transitive"           # rules.kind

The rule identity is not dangling: it is minted, typed, and labelled, and it
is stable across exports. What it does not carry is any machine-readable
account of what the rule does. A prov:Activity plus a human label lets an
auditor cite that something produced a conclusion, but not what operation
produced it — which is the question a derived conclusion actually raises.
(The rule:{uuid} mint itself is also absent from the documented IRI list in
web/src/docs/mcp.md, which names only entity/fact/derived/document.)

What is missing differs by rule family (the same rule:{uuid} mint serves
both, so the body is also the only reliable way to tell which family a rule
came from — a business rule can perfectly well be named "transitive"):

Axiom rules (rules) — one row per (kb_id, predicate_id, kind), kinds
transitive | symmetric | inverse | sub_property:

  • The kind survives only as rdfs:label text; a consumer must parse a label
    to recover a closed enum.
  • The bound predicate (rules.predicate_id) is not emitted. For transitive
    and symmetric it coincides with the conclusion's rdf:predicate and can
    be inferred. For inverse and sub_property the rule is bound to the
    declaring predicate (Derived.via: ceo_of ⊑ works_at concludes on
    works_at), which differs from the conclusion's predicate — recovering it
    means re-deriving the engine's compile convention from the exported
    owl:inverseOf / rdfs:subPropertyOf declarations, and is ambiguous when
    several predicates declare toward the same target.

Attribute rules (attribute_rules + attribute_rule_conditions) —
name, subject class, a conclusion (typing | attribute | computed), and
grouped conditions (predicate, op, operand):

  • Only name reaches the export, as the label.
  • Nothing of subject_type_id, conclusion, conclude_type_id,
    conclude_predicate_id, conclude_value, conclude_expr, or the
    conditions is emitted.
  • Some of it is visible on the derived fact — an attribute conclusion's
    value is the fact's rdf:object, a typing conclusion's class sits inside
    the object's {"class": …} literal — but the rule's declared criterion is
    nowhere: prov:used says which readings fired, never the thresholds, ops,
    or sets the rule was declared with. For a computed conclusion the
    expression itself is unrecoverable; only its result survives.

An auditor can therefore watch a reading "depth = 3500" become a typing
"Gas-bearing" under a rule labelled "Gas-bearing well", but cannot tell that
the rule declared depth > 3000 on entities of class Well. The criteria
are not secret — MCP exposes list_rules / rule_matches precisely so an
agent can read a rule and its thresholds — but they reach an agent as
formatted prose, not the export's structured data. An auditor holding only
the file, which is the premise ADR 0020 was written for, has no way to
recover the operation the name abbreviates.

Why this belongs in the RDF contract

ADR 0020 made this mapping the supported machine-readable read contract so a
reader can "walk from a conclusion to the sentences underneath it without our
API", and web/src/docs/mcp.md already documents derivations as "linked to
their rules and premise statements". prov:wasGeneratedBy commits the
contract to exposing which rule produced a conclusion — that identity is
public lineage today. What is not yet true is that the identity resolves to
something an auditor can understand without database access: today it
resolves to a label, not to the operation it names.

This is not a request to export the whole ledger. It asks only that a
resource the contract already exposes as a generator carry enough structure
to be understood.

Scope of a first cut

Only the two rule families the current derived-fact export already
references, and only the minimum structured body that lets an auditor
understand the declared operation:

  • rules — kind as structured data, plus the bound predicate.
  • attribute_rules — subject class, conclusion kind, and what it concludes;
    conditions to the extent they can be represented without committing to an
    unstable operand/expression vocabulary.

Plus the small structural regression described below. A natural home is the
existing derived_page read and emit_derived; no new machinery is implied.

Explicitly out of scope:

  • implication_rules, implied_fact_sources, phrase_readings, and
    implied-fact provenance generally — that lineage integrity is unresolved
    in Implied-fact lineage is outside the same-KB provenance invariant #901 and this issue neither fixes nor depends on it;
  • conflict / review state — Expose conflict and review state in the supported RDF read contract #564 owns that contract gap;
  • premise ordering (prov:used identifies premises, not their sequence —
    already declared in ADR 0020);
  • open-statement completeness, evidence/chunk/document-version expansion,
    entity descriptions, relation declared qualifiers, or any other ledger
    surface;
  • schema or migration changes, SPARQL, import-of-export;
  • a generic metadata framework or a second serializer.

Design questions for maintainers

  1. What is the minimum semantic body an exported rule identity should carry?
    For axiom rules, kind + bound predicate seems to suffice. For attribute
    rules the candidates are subject class, conclusion kind, the concluded
    target, and the conditions — which of these are needed to understand the
    rule
    , versus merely restating what each derived fact already shows?
  2. Is this maintenance of ADR 0020 — a dated revision landing with the
    implementation — or does the rule-semantic vocabulary constitute a new
    decision record in its own right?
  3. Which fields are stable enough to enter the permanent RDF vocabulary?
    op is a closed eight-value enum and condition grouping is one level,
    but operand and conclude_expr embed expression trees whose attr
    leaves are stored UUIDs — exporting them commits the contract to a public
    expression vocabulary (or to IRI resolution inside literals). Migration
    0070 already anticipated this seam: it exempts attr leaves from
    column-level reference enforcement and assigns them to export-side
    validation instead — a validation layer that does not yet exist because
    the export never reads expressions today.
  4. Should expression bodies be represented structurally now, or should the
    first cut cover only the semantics expressible without a public
    expression vocabulary — kind, bound predicate, subject class, conclusion
    kind/target, non-expression operands?

Suggested acceptance criteria

  • For a representative derived fact, an auditor can follow
    derived fact → generating rule → enough structured semantics to understand that rule without database access, internal API calls, or
    parsing a label.
  • Identical coverage in Turtle and JSON-LD.
  • Additive only: no existing triple removed or renamed.
  • No implication-rule or implied-fact support, no conflict state, no schema
    or migration changes.
  • A small structural regression passes: every Utopia-local IRI appearing in
    object position in the emitted set is either described by a resource body
    or explicitly classified as intentionally opaque — and for rule resources,
    the agreed minimum body is asserted, not merely non-emptiness.

Related

Activity

  1. Maya-Kid commented on Sep 24, 2026

    @Maya-Kid
    Contributor

    Proposed scope for #902 — awaiting maintainer decisions

    The current export already describes each generating rule with its stable IRI,
    prov:Activity type and human-readable label. A documentation-only correction can
    list that IRI and make the missing semantics explicit without adding RDF terms.
    The rule-body extension remains a proposal; the existing implementation is a local
    prototype, not evidence that its vocabulary has been accepted.

    Decisions requested

    Question from #902 Recommendation Alternative / consequence
    Minimum semantic body Start with axiom kind and bound predicate only. Keep business-rule resources at their existing identity/type/label until temporal semantics are agreed. Include business-rule subject and conclusion metadata, but explicitly distinguish current definitions from historical derivation evidence. This requires additional vocabulary decisions.
    ADR revision or new ADR Land a separate ADR after discussion because the extension introduces public vocabulary and temporal meaning. A dated ADR 0020 revision is possible if maintainers prefer it; it must be accepted before implementation.
    Stable public fields Decide the exact axiom terms first. Use “business rule” in public prose; any future class spelling, including BusinessRule, is only a candidate. Reusing the storage name AttributeRule exposes an implementation name and needs an explicit rationale.
    Expression scope Defer all business-rule conditions and expression trees in the first implementation. A broader cut needs a public expression grammar, reference validation and an explicit representation of omitted information.

    Details a broader proposal must settle

    • Range bounds must have distinguishable roles. Candidate lower/upper predicates
      or an explicitly ordered structure preserve those roles; two unordered operand
      values do not. Equal endpoints must remain representable as both endpoints.
    • Do not introduce bodyComplete as an unexplained Boolean. Define completeness
      relative to supported rule families, expressions, omitted fields and time, or
      omit that concept from the initial vocabulary entirely.
    • A business rule is updated in place. If the threshold changes from 3000 to 3500,
      an old derivation's rule IRI does not identify a preserved 3000 definition.
      Exporting 3500 on the generator cannot establish why the old conclusion held.
      Either model current definitions distinctly or defer this extension; historical
      rule versions require separate storage/provenance work.
    • An axiom's kind and bound predicate alone need not reconstruct every historical
      inverse/sub-property declaration. Do not claim historical replay or a complete
      proof snapshot for the smaller cut either.

    Implementation and validation after agreement

    Keep the existing IRI, type, label and provenance triples. Add only the agreed
    terms, with the same snapshot and same-KB reference validation as the existing
    export. Schema changes, rule versioning, implied-fact lineage and conflict state
    remain separate work.

    Use focused serializer tests for exact term/value mapping; store tests for missing
    or foreign references; and one representative database-backed route scenario run
    in Turtle and JSON-LD. Avoid repeating the full semantic matrix at all three
    layers. Check malformed references fail the export rather than appearing as a
    successful, complete result. Require the database guard when running SQL tests.

    Sequence: discuss these choices in #902, land the accepted ADR, then implement the
    agreed slice. The documentation correction does not close #902.

  2. WaylandYang commented on Sep 24, 2026

    @WaylandYang
    Contributor

    Thanks for narrowing this. Decisions:

    1. First cut is axiom rules only, as structured data: utopia:axiomKind with the closed enum (transitive, symmetric, inverse, sub_property) and utopia:declaredOn pointing at the declaring predicate's IRI, which is Derived.via and differs from the conclusion's predicate for inverse and sub_property. Plus a family type on every rule resource, utopia:AxiomRule or utopia:BusinessRule, so a reader no longer parses a label to know which table a rule came from.
    2. A dated revision of 0020, not a new record. This adds two terms and a type to a resource the contract already exposes; it does not move the contract's footing.
    3. No storage names in the vocabulary. AttributeRule stays internal; the public word is business rule.
    4. Business-rule conditions and expressions wait for rule versioning. A rule is updated in place, so the generator IRI cannot vouch for the definition an old conclusion was drawn under; exporting today's threshold on it would tell an auditor something false about yesterday's conclusion. That is a deferral, not a refusal: once a rule's definitions are versioned and a derivation points at the version it used, the body can be exported per version. Anyone who wants that can open the versioning issue; it is storage and provenance work of its own.

    I will draft the 0020 revision and land the slice behind it.

  3. WaylandYang commented on Sep 24, 2026

    @WaylandYang
    Contributor

    Landed in #910 as the first cut decided above: every rule resource is typed utopia:AxiomRule or utopia:BusinessRule, and an axiom rule carries utopia:axiomKind and utopia:declaredOn (the declaring predicate, which for inverse and sub_property is not the conclusion's). Recorded as the 2026-09-25 revision of 0020; the MCP guide says the same. Business-rule bodies are deferred for the reason 0020 now states under What is not here: a rule is edited in place, so its IRI cannot vouch for the definition an older conclusion used. If someone wants that, please open an issue for rule versioning; the export of conditions and expressions follows it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions