Skip to content

PyCharter — Reference Model Unification (PAC-4 + PAC-5)

Status: implemented (PAC-4 #299, PAC-5 #298) · Created: 2026-06-09 · Audience: maintainers · Resolves: PAC-4 + PAC-5 in python-api-consolidation.md

Why

The same "address a versioned artifact by name + version" concept is spelled four ways, with two different separators and one inconsistent attribute name:

Type Where Shape Serialization
StoreRef shared/refs.py (public, PAC-1) (name, version?) dataclass name@version
ContractStoreRef semantic/pipelines/models.py (YAML) {type:store, name, version?} Pydantic structured YAML
ArtifactRef semantic/refs/resolver.py (resolver) (kind, name, version?) dataclass name:version
ContractRef streaming/_types.py (streaming) (contract_id, version) dataclass n/a

Plus ContractFileRef ({type:file, path}) — a genuinely different concept (a filesystem path, not a store address) that must survive.

StoreRef is the public, canonical address (pycharter.StoreRef, used by Validator.from_store / Pipeline.from_store / StreamWorker.from_store). The goal: one internal address type — StoreRef — for every store reference, with the Pydantic YAML models kept only as the parse/wire layer.

Decisions

  1. Canonical type = StoreRef (name, version | None). Every store reference resolves through a StoreRef internally.
  2. Separator = @ (StoreRef's). The resolver's ArtifactRef and runtime._serialize_ref currently use : (name:version); standardize on @. This string form is internal (a serialize→resolve round-trip) plus a few error messages — it is not in YAML (structured) or on any wire. @ already matches the user-facing CLI convention. (Risk: a handful of tests assert the : error text; update them.)
  3. File ref stays. ContractFileRef ({type:file, path}) is unchanged. The YAML discriminated union stays ContractFileRef | ContractStoreRef (Pydantic, type-discriminated) — these remain the parse layer; ContractStoreRef gains .to_store_ref() -> StoreRef and all resolution goes through StoreRef. (StoreRef is a frozen dataclass and cannot sit in a Pydantic discriminated union directly, so the thin ContractStoreRef schema model is retained.)
  4. Fold ArtifactRef into StoreRef. Its kind field is redundant — the resolver already has kind-specific methods/registries (resolve_contract, resolve_linkml_schema). Resolver methods take a StoreRef (+ implicit kind); parse_ref/as_string collapse into StoreRef.parse/__str__.
  5. Streaming ContractRef → StoreRef (PAC-4). The contract_id attribute becomes name; version stays. ~14 streaming files.

Hard constraints (must NOT change — verified by round-trip tests)

  • YAML: {type:store, name, version} / {type:file, path} byte-identical.
  • Kafka wire: header name CONTRACT_HEADER = "pycharter-contract" + the separate version header — unchanged. Only Python attribute access changes.
  • DLQ / file-sink envelope JSON: the keys "contract_id" and "version" stay literal (_file_sinks.py, _observability.py logging extras). Only the right-hand side contract.contract_id → contract.name changes.

Execution (one coherent hard-cut, two PRs)

  • PR A (PAC-5): ContractStoreRef.to_store_ref(); route the pipeline runtime + resolver onto StoreRef; fold ArtifactRef → StoreRef (@ separator); keep the Pydantic YAML models + ContractFileRef. Update the :→@ error-text tests.
  • PR B (PAC-4): replace streaming ContractRef with StoreRef (contract_id→name); pin the Kafka header + DLQ/log JSON keys with a round-trip test asserting the envelope bytes are unchanged.

Both land with the existing suites green; no YAML/wire byte changes.