Flywheel walkthrough — discover, suggest, accept, bake¶

This guide walks through the pycharter contribution flywheel: the loop that turns an unbound field in a data contract into a governed, baked-in binding, entirely from evidence observed in the data itself — no manual mapping required.

The walkthrough runs the self-contained example at examples/flywheel_demo/. Every command below was verified against the develop branch on 2026-07-16 (GV-101..GV-118 merged, GV-117 example).

Audience:

Contributors who want to see the flywheel work end to end before reading the GV-10x/GV-11x implementation.
Reviewers evaluating whether a pipeline run's structural extraction is wired correctly (grounding, matching, suggestion, bake).

Cross references:

tests/unit/test_flywheel_end_to_end.py — the un-mocked proof this walkthrough narrates; the example mirrors its exact mechanics.
Vocabulary — concepts, schemes, and tiers.
Binding any resource to a concept — the general field-binding model this flywheel produces bindings into.
Seed data and sample files — how this example relates to data/seed/examples/.

What the flywheel does¶

A data contract can bind a field to a concept explicitly (field_bindings.priority.primary.concept). Fields with no explicit binding are not lost: every automatic pipeline run profiles each unbound field's observed value domain (distinct values, cardinality, type) and compares it against the value domains already grounded for concepts in the target scheme. A strong match (similarity > 0.8) produces a bind_field suggestion — a candidate binding a reviewer can accept or dismiss, never an automatic write to the contract.

The loop, in one pass:

run 1: ground a concept's value domain from a bound field
  -> run 2: an unbound field with the same value domain matches it
     -> bind_field suggestion
        -> accept: governed APPROVED binding + mapping-projection row
           -> bake: forked contract version carries the binding

Prerequisites¶

git clone https://github.com/optophi/pycharter.git
cd pycharter
pip install -e ".[dev]"

No database server, no LLM, and no network access are required — the example runs against a throwaway temp-file SQLite database and uses only the pipeline runtime's own record-content extraction.

Run it¶

cd examples/flywheel_demo
python run_flywheel.py

The script prints a banner per stage. tests/unit/test_flywheel_demo.py runs the same core function (run_flywheel()) as a pytest smoke test — run pytest tests/unit/test_flywheel_demo.py -q to verify without reading output by eye.

Files¶

flywheel_demo/
├── concept_scheme.yaml     # the seed vocabulary (kind: ConceptScheme)
├── contracts/ticket.yaml   # priority -> TicketPriority bound; priority_level left unbound
├── pipeline.yaml           # kind: Pipeline; extract bundled CSV -> load w/ contract ref
├── bundled/tickets.csv     # 60 rows; priority + priority_level share a 4-value domain
├── run_flywheel.py         # orchestration script (prints each stage)
└── README.md

Stage 1: Seed the concept scheme¶

run_flywheel.py loads concept_scheme.yaml through the real scheme parser (pycharter.semantic.rdf.vocabulary.scheme_from_document) and inserts each concept, including TicketPriority — the concept run 1 grounds and run 2 matches against.

What you see:

1. Seed the ticketing concept scheme
scheme_id: <uuid>

Stage 2: Seed the base contract¶

The base contract (tickets@1.0.0) binds priority to TicketPriority and leaves priority_level unbound — the same field the two columns share in bundled/tickets.csv.

What you see:

2. Seed the base contract (tickets 1.0.0)
saved tickets@1.0.0: priority -> TicketPriority bound, priority_level left unbound

Stage 3: Run 1 — ground TicketPriority¶

pipeline.yaml extracts the bundled CSV and loads it through a step carrying contract.ref (with annotate: true). Because the contract's extraction_policy.trigger is automatic, the pipeline attaches a live extraction sink for this run. The priority field is bound, so its observed value domain (low, medium, high, urgent, 15 rows each) rolls up into TicketPriority's observed profile — a ConceptObservedProfileModel row keyed on (scheme_id, concept_key).

What you see:

3. Run 1: ground TicketPriority
observed profile rows for TicketPriority: 1
  record_count=60 top_values=[{'value': 'str:high', 'count': 15}, ...]

This diagnostic shows the stored roll-up used for matching. Agent and API reads apply profile suppression before serialization: restricted profiles, profiles from 1–10 records, and profiles with at most 10 distinct values return policy_gated: true. Their exact counts, rates, ranges, samples, timestamps, and contributor provenance are withheld. Consumers should show "Withheld" for nullable metrics rather than treating them as zero.

Stage 4: Run 2 — discover priority_level¶

Run the same pipeline again. priority_level still has no binding in the contract, but its observed value domain is now compared against TicketPriority's grounded profile from run 1. The domains match closely enough (well past the 0.8 similarity threshold) to produce a bind_field suggestion — a SuggestionModel row whose provenance.proposed_binding carries the field, the target concept key, and the owning contract's identity (contract_id, contract_version).

What you see:

4. Run 2: discover priority_level -> bind_field suggestion
suggestion <uuid>: bind priority_level -> TicketPriority (contract tickets@1.0.0)

Stage 5: Accept the suggestion¶

accept_suggestion() turns the suggestion into a governed decision: a BindingLifecycle event transitioning to approved, keyed on the concept key (TicketPriority), and an approved primary row in the mapping-projection table for (tickets@1.0.0, priority_level). Nothing in the contract artifact itself has changed yet — acceptance is a governance decision, not a write to field_bindings.

The HTTP API always persists lifecycle projection updates in SQL, alongside the binding event log. PYCHARTER_MAPPING_PROJECTION_BACKEND continues to select the standalone repository factory's backend; it does not switch hosted approvals to memory. Apply the database migrations before starting the API. When enabled, projection-backed lineage reads the same SQL rows. Its feature flag still chooses between that projection and the JSON-scan view.

What you see:

5. Accept the suggestion into a governed binding
binding event: approved (priority_level -> TicketPriority)
approved projection row: field=priority_level concept=TicketPriority scheme=ticketing:1.0.0

Stage 6: Bake into a new contract version¶

bake_approved_bindings() reads every currently-approved primary binding for tickets@1.0.0, merges them into a copy of the base field_bindings envelope, and forks a new version (1.1.0) carrying the merged artifact. The base version is never mutated.

The baked version inherits approval for each exact approved primary target and scheme, preserving the base field's pii and restricted flags. Its projection retains the source approval event ID. Ordinary forks inherit approval only when the full normalized field binding is unchanged, including role, tags, privacy flags and annotations. Changes require review again. Only currently approved rows qualify: a revoked or rejected base decision cannot be inherited.

Inherited approval is a decision on the new version. A later version-scoped revocation or rejection withdraws that version's agent-facing meaning without changing its immutable artifact or the base version's projection. Agent meaning remains advisory, and ambiguous namespace twins still withhold semantic context; approval inheritance also refuses an ambiguous source or destination identity.

What you see:

6. Bake the approved binding into tickets 1.1.0
baked 1 binding(s) into tickets@1.1.0
  priority_level: {'primary': {..., 'concept': {'concept': 'TicketPriority'}, 'concept_scheme': 'ticketing:1.0.0', ...}}
base tickets@1.0.0 field_bindings: ['priority'] (priority_level absent, as expected)

FLYWHEEL COMPLETE
discover -> suggest -> accept -> bake, all proven end to end.

tickets@1.1.0 now binds both fields; tickets@1.0.0 is byte-for-byte what it was before the flywheel ran.

Things worth understanding¶

Two pipeline runs are required, not one. Grounding (materialize(mode="suggestion")) and profiling (materialize(mode="profile")) both run inside a single pipeline run's finalise step, in that order — so the grounded profile a later run matches against only exists after an earlier run has completed and committed. A single run cannot ground and match in the same pass; that is why run_flywheel() calls the pipeline twice with distinct run_ids before a suggestion appears.
Nothing writes to the contract automatically. Run 2 produces a suggestion, not a binding. A human (or an automated policy, if one is configured) must call accept_suggestion() before anything governed exists, and bake_approved_bindings() before the contract artifact itself changes.
The scheme reference is name:version, not a UUID. The discovered binding's concept_scheme resolves to ticketing:1.0.0 — the same shape as the hand-authored priority binding — so every scheme-matching consumer treats a flywheel-discovered binding identically to an authored one.
The concept scheme is seeded directly, not via Vocabulary.schemes.save(). As of this writing, Vocabulary.schemes.save() does not populate ConceptModel.concept_key and derives tier from hierarchy depth rather than honouring an explicit value — both of which the flywheel's bind_field matching queries rely on. run_flywheel.py parses concept_scheme.yaml with the same real scheme parser (scheme_from_document) but inserts the resulting concepts directly, matching tests/unit/test_flywheel_end_to_end.py.
Low-cardinality categorical fields make the clearest demo. The 4-value domain over 60 rows clears the enum-candidate threshold (distinct ratio ≈ 0.067) and drives the Tier-2 value-domain overlap well past 0.8. A high-cardinality field (free text, UUIDs) will not produce a bind_field suggestion this way.
SQLite for zero setup. The identical script runs against Postgres by pointing the SQLAlchemy URLs it builds at a Postgres connection string instead of a temp SQLite file.