Skip to content

Superseded by docs/proposals/vocabulary-consolidation.md (2026-07-10).

Closed record, not a backlog. Statuses below are as they stood when the program closed and are not maintained. Open work lives only in BACKLOG.md.

Historical document. Current product terminology is Vocabulary (third pillar), concept scheme, and field binding. Retired names (ontology, KB, field mapping) appear here as program-era references only. See Canonical Terminology.

Vocabulary Implementation Plan (historical)

Objective

Implement ontology governance as a practical layer in PyCharter that:

  1. normalizes and validates ontology bindings in contracts,
  2. enforces proposal/review governance with merge gates, and
  3. operationalizes ontology for validation, routing, and impact analysis.

This plan is derived from:

  • docs/guides/ontology-governance-rfc.md

Scope Summary

In scope:

  1. Canonical contract ontology schema (field_bindings) with backward compatibility.
  2. Semantic health APIs and merge gate validation APIs.
  3. Policy-aware approvals by tier and change type.
  4. UI support in ontology proposal flow for ontology change governance.
  5. Initial ontology-driven contract checks (coverage, references, derived_from).

Out of scope for this implementation cycle:

  1. Advanced graph reasoning and automated ontology inference.
  2. Full ontology recommender/copilot.
  3. Hard global enforcement on all legacy contracts from day one.

Workstreams

WS1: Contract Ontology Canonicalization

Goal: make ontology contract payload strict enough for reliable processing.

Tasks:

  1. Define canonical schema:
  2. ontology.version
  3. ontology.field_bindings.{field_path}.concept_id
  4. optional confidence, relationships, lineage, tags
  5. Implement strict Pydantic models in db/schemas/data_contract.py.
  6. Add parser normalization:
  7. legacy ontology.fields -> canonical ontology.field_bindings
  8. Add structured validation errors and migration warnings.
  9. Add tests:
  10. parse legacy and canonical
  11. invalid relationship type rejection
  12. missing concept_id rejection

Primary files:

  • src/pycharter/db/schemas/data_contract.py
  • src/pycharter/contract_parser/parser.py
  • tests/test_contract_parser.py
  • tests/wiki/test_knowledge_json_schema.py (ontology enums: ConceptType, RelationshipType)
  • tests/wiki/test_schema_loader.py (loads vocabulary_metaschema.yaml)

Acceptance criteria:

  1. Existing contracts still parse via normalization path.
  2. New canonical schema is validated deterministically.
  3. Validation error messages identify field path and fix hint.

WS2: Semantic Health Service

Goal: expose measurable ontology quality for each contract.

Tasks:

  1. Add semantic health service:
  2. mapping coverage (% mapped fields)
  3. invalid concept references
  4. deprecated concept usage
  5. invalid relationship types
  6. Add API endpoint:
  7. GET /api/v1/contracts/{contract_id}/field-mapping
  8. Add tests for service + route.

Primary files:

  • src/pycharter/wiki/knowledge/* (new helper/service module)
  • src/pycharter/wiki/api/routes/v1/*
  • src/pycharter/wiki/api/models/*

Acceptance criteria:

  1. Endpoint returns deterministic health payload.
  2. Health score works for both legacy and canonical ontology payloads.

WS3: Proposal Governance + Merge Gates

Goal: make ontology changes policy-driven and safely mergeable.

Tasks:

  1. Extend proposal model:
  2. proposal_type
  3. required_approvals
  4. gate_status JSON
  5. Add policy evaluator:
  6. derive required approvals by tier + change type
  7. Add proposal validation endpoint:
  8. POST /api/v1/proposals/{id}/validate
  9. Implement gates:
  10. concept existence
  11. relationship-type validity
  12. semantic coverage delta
  13. optional SHACL gate (feature flag)
  14. Block merge unless gates pass (for governed tiers).

Primary files:

  • src/pycharter/db/models/semantic/ (proposal-related models)
  • src/pycharter/wiki/collaboration/proposals.py
  • src/pycharter/wiki/collaboration/review.py
  • src/pycharter/wiki/collaboration/permissions.py
  • src/pycharter/wiki/api/routes/v1/proposals.py

Acceptance criteria:

  1. core changes require configured approvers and passing gates.
  2. Gate outputs are persisted and queryable.
  3. Failed gates provide actionable diagnostics.

WS4: Ontology UI Enhancements for Ontology Governance

Goal: make governance decisions visible and auditable in the UI.

Tasks:

  1. Proposal form adds proposal_type and impacted contract summary.
  2. Proposal detail page shows:
  3. required approvers
  4. gate status (pass/fail/pending)
  5. impact summary (contracts/fields/concepts touched)
  6. Review panel shows policy progress to merge readiness.

Primary files:

  • src/pycharter/ui/src/app/ontology/proposals/page.tsx
  • src/pycharter/ui/src/app/ontology/proposals/detail/page.tsx
  • src/pycharter/ui/src/components/ontology/CreateProposalModal.tsx
  • src/pycharter/ui/src/components/ontology/ReviewList.tsx

Acceptance criteria:

  1. User can see exactly what is blocking merge.
  2. Policy and gate outcomes are aligned with backend source of truth.

WS5: Ontology-Driven Contract Checks

Goal: turn ontology into useful validation behavior.

Tasks:

  1. Add optional semantic coverage threshold checks.
  2. Implement references integrity checks (quality checks, not DB FKs).
  3. Implement derived_from lineage consistency checks.
  4. Route incidents to concept + contract owners.

Primary files:

  • src/pycharter/validator/*
  • src/pycharter/api/services/contract_service.py
  • metadata ownership/governance integration points

Acceptance criteria:

  1. Contract validation report includes ontology-derived checks.
  2. Failures identify impacted owners/stakeholders.

Sequencing and Milestones

Milestone 1 (2 weeks)

  1. WS1 complete.
  2. WS2 partial (service + API v1 without full scoring weights).

Deliverable:

  • canonical schema + health endpoint + parser compatibility.

Milestone 2 (2 weeks)

  1. WS3 backend complete.
  2. WS4 partial UI integration for gate and approvals display.

Deliverable:

  • governed proposal workflow with merge block behavior.

Milestone 3 (2 weeks)

  1. WS4 complete.
  2. WS5 baseline checks complete.

Deliverable:

  • ontology-driven validation and end-to-end governance UX.

Testing Strategy

  1. Unit:
  2. schema normalizer
  3. policy evaluator
  4. gate engine
  5. Integration:
  6. proposal creation -> review -> validate -> merge
  7. contract parse -> semantic health -> validation report
  8. Regression:
  9. legacy contract parsing
  10. existing ontology flows without ontology changes

Migration and Backward Compatibility

  1. Accept both legacy ontology.fields and canonical field_bindings.
  2. Normalize at parse time; keep write-path to canonical for new saves.
  3. Add warnings in API responses for legacy payload usage.
  4. Provide migration command (future task):
  5. dry-run summary + optional rewrite output.

Delivery Backlog (Ticket-Ready)

Backend

  1. BE-ONT-001: Add canonical ontology binding models to DataContractSchema.
  2. BE-ONT-002: Implement parser normalizer fields -> field_bindings.
  3. BE-ONT-003: Build semantic health calculator service.
  4. BE-ONT-004: Add semantic health API route + response model.
  5. BE-ONT-005: Extend proposal DB model for governance metadata.
  6. BE-ONT-006: Implement policy evaluator by tier/change type.
  7. BE-ONT-007: Implement proposal gate runner endpoint.
  8. BE-ONT-008: Merge guard enforcement with gate + approval checks.
  9. BE-ONT-009: Add ontology-derived validation checks (references, derived_from).

Frontend

  1. FE-ONT-001: Add proposal type selector in create proposal modal.
  2. FE-ONT-002: Add gate status panel in proposal detail.
  3. FE-ONT-003: Add approval requirement/progress display.
  4. FE-ONT-004: Add impact summary block for changed concepts/fields/contracts.

QA/DevEx

  1. QA-ONT-001: Contract fixture suite for legacy + canonical ontology payloads.
  2. QA-ONT-002: End-to-end test for proposal governance lifecycle.
  3. DX-ONT-001: Author migration guide section + API examples.

Risks and Controls

  1. Risk: concept ID inconsistency across environments.
  2. Control: explicit concept registry validation + seed synchronization check.
  3. Risk: governance friction for domain teams.
  4. Control: tiered policy with configurable thresholds.
  5. Risk: UI/backend policy mismatch.
  6. Control: backend-provided policy status model; UI only renders server state.

Decision Log (To Fill)

  1. Canonical field path format.
  2. Decision: JSONPath in v1.
  3. Rationale: user preference and better support for nested/array structures.
  4. Field-to-concept cardinality.
  5. Decision: many-to-many in v1.
  6. Concept identifier strategy.
  7. Decision: use an immutable concept_key (separate from display name).
  8. Coverage denominator.
  9. Decision: leaf fields only.
  10. Coverage enforcement behavior.
  11. Decision: warn-only in v1 (no hard fail).
  12. Approval policy for core in v1.
  13. Decision: any authenticated user can approve for now.
  14. Merge blocking scope by tier.
  15. Decision: block merges for core only in v1; domain/free are warn-only with audit.
  16. Legacy payload handling (ontology.fields).
  17. Decision: always normalize in memory; persist canonical field_bindings on create/update saves.
  18. Note: no forced bulk rewrite migration for untouched historical contracts.
  19. SHACL gate default (off/on) for first production rollout.
  20. Decision: feature-flagged OFF by default in v1, report-only when enabled.

WS3 API Examples

Create proposal with governance metadata:

POST /api/v1/proposals/
{
  "contract_id": "9fae5b8a-2a50-4f77-86fc-7f96d130c001",
  "title": "Map customer fields to core identity concepts",
  "author": "alice",
  "proposal_type": "mapping_change",
  "affected_tiers": ["core"]
}

Run proposal validation gates:

POST /api/v1/proposals/{proposal_id}/validate
{
  "proposed_ontology": {
    "version": "1.1.0",
    "field_bindings": {
      "$.customer_id": [
        {"concept_id": "CustomerIdentifier"}
      ]
    }
  }
}

Semantic health by contract name/version:

GET /api/v1/contracts/by-name/{contract_name}/{contract_version}/field-mapping

Implementation Start Recommendation

Start with WS1 + WS2 in a single branch to establish canonical data shape and observability first. This minimizes downstream churn before governance and UI work.