Offline pipeline plans and version identities¶
pycharter.pipeline_plan inspects a supplied canonical pipeline without reading
sources, loading referenced contracts, resolving environment variables, creating
connectors, or importing configured Python. It produces an incomplete structural
plan for review. Extraction previews are different: they can read a source.
Which version means what?¶
| Value | Meaning |
|---|---|
api_version: pycharter.io/v2 |
The document format understood by the reader. It is independent of a pipeline release. |
Pipeline metadata.version |
The authored version of the complete pipeline definition, including inline steps and its declared references. |
Step contract.ref.version |
The independently selected version of that data contract. Publishing a newer contract does not change an explicit reference. |
plan_summary_digest |
A hash of the exported inspection summary, with the explicit scope offline_structure_v1. |
| Resolved bundle digest | Future identity of captured effective content and dependencies. This slice always leaves bundle_digest null. |
| Deployment identity and binding revision | Optional public context identifying an environment/deployment. This slice records labels; it does not resolve or enforce bindings. |
| Invocation/run identity | Optional public context identifying one run and its input names. This slice records no parameter values and enforces no parameter policy. |
Inline steps belong to their pipeline's version. Independently versioned reusable
step artifacts require interface and parameter rules and are tracked separately.
Legacy spec.versions has no execution consumer: this API reports its presence
without activating, reinterpreting, exporting, or deleting its contents.
This is a full authored pipeline example. Its pipeline version 1.2.0 and
contract version 1.1.0 refer to different artifacts. Loading it for inspection
does not require the fixture, database, referenced contract, or environment
variables to exist.
api_version: pycharter.io/v2
kind: Pipeline
metadata:
namespace: market
name: daily-prices
version: "1.2.0"
spec:
variables:
DESTINATION_URL: "${PRICES_DATABASE_URL}"
steps:
- id: read-prices
type: extract
source:
type: file
path: fixtures/prices.json
- id: shape-prices
type: transform
input: read-prices
operations:
- select: [symbol, date, close]
- id: save-prices
type: load
input: shape-prices
target:
type: postgres
connection_string: "${DESTINATION_URL}"
table: daily_prices
contract:
ref:
type: store
name: market/daily-prices
version: "1.1.0"
Inspect supplied configuration¶
from pycharter.pipeline_artifact import load_pipeline_from_dict
from pycharter.pipeline_plan import (
compile_pipeline_plan,
dump_pipeline_plan_to_dict,
load_pipeline_plan_from_dict,
)
# document is the already supplied YAML/JSON mapping above.
definition = load_pipeline_from_dict(document)
plan = compile_pipeline_plan(
definition,
deployment={"id": "staging", "binding_revision": "revision-2"},
invocation={"run_id": "run-42", "input_names": ["WINDOW_START", "SYMBOL"]},
)
exported = dump_pipeline_plan_to_dict(plan)
restored = load_pipeline_plan_from_dict(exported)
assert restored == plan
assert plan.spec.bundle_digest is None
assert not plan.spec.frozen and not plan.spec.verified and not plan.spec.executable
Use the canonical artifact loader for supplied manifests. The runtime parser can resolve environment-dependent configuration; it is not an offline inspection boundary. Reading a declared configuration file is an explicit outer operation; the compiler and plan codec themselves perform no file, store, or network I/O.
The plan retains authored step order, input edges, operation names, direct
structural dependency declarations, and safe issue codes. An exact declared
contract pin still has resolution: unresolved: the compiler has not captured
or verified its content. An omitted contract version is floating. Files,
configured Python/expressions, engine versions, and runtime bindings remain
unresolved. Discovery is direct and structural, not a complete transitive
dependency inventory. Native settings.state_store and settings.plugins
remain inactive and are reported when present.
Privacy and digest limits¶
Only public identifiers and allowlisted structural facts are exported. Treat
pipeline/step/dependency identifiers, deployment/run labels, and input names as
public; do not put secrets in them. Connector options, URLs, paths, variable
defaults, expression bodies, callable names/kwargs, policies, cataloguing
metadata, and legacy ledger contents are withheld and are not hashed.
Use dump_pipeline_plan_to_dict as the export boundary: it revalidates nested
values before serialization, including models changed through unchecked
Pydantic copying/construction. Direct serialization of arbitrary unchecked
Python objects is outside this guarantee. Public labels accept letters, digits,
underscores, dots and hyphens in slash-delimited segments; artifact identity
uses the existing stricter slug and semantic-version rules.
The digest covers the exported structural summary, excluding the digest itself and deployment/invocation context. It preserves ordered graph and operation names. Changing a destination, secret, rename mapping, policy, or other withheld option can leave the digest unchanged. Equal summary digests do not establish equal execution behavior. It is neither an execution lock nor a signature or proof of origin. The codec rejects unsupported claims and inconsistent summaries; an author can still construct a different valid summary with its own digest.
Retained pipeline components adds a separate supplied-input API that saves admitted native operations and contract rules with scoped content checksums. It leaves complete bundle and execution claims false. Frozen execution, immutable publication, parameter enforcement, deployment validation, and replay evidence are later slices. Configuration reproducibility alone cannot guarantee identical outputs from a changing source, clock, randomness, or custom code. This API does not execute configured Python; connector substitution and offline planning do not sandbox arbitrary Python objects or application code.