Superseded by
docs/proposals/vocabulary-consolidation.md(2026-07-10).Closed record, not a backlog. Statuses below are as they stood when the program closed and are not maintained. Open work lives only in
BACKLOG.md.Historical document. Current product terminology is Vocabulary (third pillar), concept scheme, and field binding. Retired names (
ontology,KB,field mapping) appear here as program-era references only. See Canonical Terminology.
PyCharter KB/Ontology Workflow Kanban¶
This board supports the implementation plan in
pycharter/docs/guides/kb-ontology-workflow-proposal.md.
Use it as the shared coordination artifact for AI agents and human reviewers. Every card should preserve a narrow write scope, explicit acceptance criteria, and recorded review/test evidence.
Recommended Claude skill: /orchestrate
(pycharter/.claude/skills/orchestrate/SKILL.md).
Board Rules¶
- One owner per card.
- One reviewer per card before QA.
- No card moves to Done without evidence.
- No agent edits outside the card's write scope without updating the card first.
- Blockers must name the owner, reason, and unblock condition.
- UI work must remain consistent with the existing editor shell, semantic components, store patterns, and API client patterns.
- Backend work must keep workflow state durable and traceable.
- Only the Orchestrator edits this kanban file. Worker agents write their findings to the card's named output file; the Orchestrator pulls evidence into the card.
Program Decisions (locked 2026-05-18)¶
| Decision | Resolution | Effect |
|---|---|---|
| Branch strategy | One umbrella branch per phase. Phase 0 lives on feat/kb-phase-0. |
All Phase 0 cards land here; merged to develop via one PR after Phase 0 summary. |
| Domain workspace model | UI label over existing concept schemes. No new persisted model in Phase 1. | Phase 0 audits and Phase 1 schema work assume no new workspace entity. |
| Sample domain | Revenue / billing (matches proposal's "Revenue Agent Readiness" worked example). | End-to-end demo, fixtures, and Phase 1+ test data target revenue/billing. |
| API compatibility | Preserve existing semantic endpoints until replacements are stable (per mission rule 5). | Phase 0 API audit must classify each route as keep / replace / deprecate but mark no breaking changes during Phase 1. |
Open decisions still deferred (capture on the relevant cards when raised):
- Mapping write-back: direct to contracts vs. reviewed overlay.
- Roles for first release.
- MCP direct vs. via PyGubernator.
- Minimum evidence envelope for agent context.
Agent Role Bindings¶
The harness exposes the agent types general-purpose, Explore, Plan, claude, claude-code-guide, statusline-setup. The proposal's logical roles map onto these as follows; the Orchestrator briefs each spawn with a role-scoped prompt.
| Logical role | Harness agent type | Notes |
|---|---|---|
| Backend Agent | general-purpose |
Read codebase + write to scoped doc/code paths. |
| Frontend Agent | general-purpose |
Same; one parallel spawn per card to keep write scopes disjoint. |
| QA Agent | general-purpose |
Test matrix + verification checklists. |
| Docs / DevRel Agent | general-purpose |
Vocabulary, user-facing labels, guides. |
| Reviewer / Integrator | general-purpose (separate session) |
Reviews completed work before QA column. |
| Codebase exploration (read-only lookup) | Explore |
Used inside other agents' briefs when they need a fast file search. |
Card Template¶
### CARD-ID: Short action-oriented title
- Phase:
- Area:
- Owner:
- Reviewer:
- Status:
- Priority:
- Risk:
- Source:
- Dependencies:
- Write scope:
- Acceptance criteria:
- Test evidence:
- Review notes:
- Residual risks:
Backlog¶
P2-001: Implement asset import summary model¶
- Phase: Phase 2
- Area: Backend/API
- Owner: Backend Agent (
general-purpose, agentIda519646f17f877ca0) - Reviewer: Reviewer/Integrator Agent
- Status: Done
- Priority: High
- Risk: Medium
- Source: KB/Ontology Workflow Proposal §"Phase 2"; P1-002 + P1-003 foundation.
- Dependencies: P1-002 (Done), P1-003 (Done), P1-003-FU-B (Done).
- Write scope: As declared. Disjoint from P2-002 (suggestion evidence). ✅
- Acceptance criteria:
- Import runs return counts (assets_extracted, extracted_candidates, errors, warnings, suggestions_emitted). ✅
- Import results include source references and stable IDs (asset_id, source_uri, fingerprint). ✅
- API and Python facade expose the same shape — route handler is a one-line wrapper over
build_import_run_summary. ✅ - Test evidence:
- Files added (2):
src/pycharter/semantic/asset_lineage/import_summary.py(268 lines) — Pydantic modelsSourceReference,ImportRunCounts,ImportRunSummary+build_import_run_summaryfacade.tests/unit/test_import_summary.py(273 lines, 9 tests). - Files modified (3):
asset_lineage.pyroute + newGET /api/v1/semantic/ingest-runs/{run_id}/summary(393 lines total;LookupError → 404via audited helper).asset_lineage/__init__.py(re-exports).test_asset_lineage_route_factory.py(+6 parametrised tests). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean; 15 new tests + asset-lineage regression all pass; full unit — 2474 passed.
- Review notes:
- APPROVE (
kb-phase-2-foundation-review.md): facade is a thin wrapper over the route;LookupError → 404via audited helper; auth applied via existing_routers.pymount; 35 tests pass; ruff + mypy clean. Reviewer recommended P2-001-FU-A to pivotextracted_candidatesto query suggestions once P2-002 lands. - Residual risks:
extracted_candidatesislen(list_assets_by_ingest_run(...))— diverges from the proposal's narrative "candidate business terms". Tracked in P2-001-FU-A (Backlog).- No snapshot for mid-flight runs; counts shift between calls.
- No per-asset suggestion breakdown in this card.
P2-002: Add structured evidence to suggestion generation¶
- Phase: Phase 2
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda0dd099b1e346a3e8) - Reviewer: Reviewer/Integrator Agent
- Status: Done
- Priority: High
- Risk: High → landed cleanly (opt-in validation keeps legacy callers byte-identical).
- Source: KB/Ontology Workflow Proposal §"3. Strengthen Suggestion Evidence" (Data Model Recommendations).
- Dependencies: P1-003 (Done), P1-005 (Done — asset/ingest-run linkage).
- Write scope: As declared. Disjoint from P2-001. ✅
- Acceptance criteria:
- Every suggestion CAN include source asset ID (existing, from P1-005), source span/locator (new), confidence (existing), rationale (new), proposed action (new). ✅
- Suggestions without minimum evidence are rejected (chosen via
IncompleteSuggestionEvidenceError). ✅ - Tests cover complete / incomplete / conflicting cases. ✅ (24 new tests.)
- Test evidence:
- Design decisions:
- Columns over JSON — first-class queryable columns matching P1-002 / P1-003 / P1-005 pattern.
- Reject over mark-incomplete — clean failure boundary; bad data doesn't land in the table.
- Opt-in validation — triggers on
require_evidence=Trueor when any evidence kwarg is supplied. Legacy callers stay byte-identical.
- Files added (3): migration
suggestion_evidence_columns(99 lines;down_revision="mapping_projections", reversible).src/pycharter/persistence/collaboration/_suggestion_evidence.py(259 lines) —SuggestionEvidencedataclass + validators + 2 domain exceptions.tests/unit/test_suggestion_evidence_validation.py(368 lines, 24 tests). - Files modified (3):
suggestion.pymodel (3 new columns:source_span String(512),rationale Text,proposed_action String(64) indexed).suggestions.pyservice (create/create_batchaccept new kwargs +require_evidenceflag; new_maybe_validate_evidencehelper;_to_dictexposes new fields).semantic_suggestions.pyPydantic (SuggestionResponseexposes new fields withField(..., description=...)). - Domain exceptions:
SuggestionEvidenceError(PyCharterSemanticError)→IncompleteSuggestionEvidenceError,ConflictingSuggestionEvidenceError. Carry structured context (.missing,.conflicts). - Existing test fixtures NOT touched — opt-in design meant zero churn in
test_suggestion_manager.py,test_suggestion_source_traceability.py,test_semantic_suggestions_api.py,test_suggestion_adapter.py,test_extraction_materializer.py. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean (55 source files); 24 new tests pass; 39-test suggestion regression clean; 1 integration restart-survival pass; full unit — 2474 passed (baseline 2435 + 39 new from both P2 cards).
- Review notes:
- APPROVE (
kb-phase-2-foundation-review.md): opt-in validation keeps 39-test legacy suite byte-identical (zero fixture changes); migration idempotent + reversible; 24 new tests cover complete / incomplete / conflicting genuinely. Reviewer flagged_suggestion_evidenceas private-named but de-facto public failure contract — promote before first external consumer. - Residual risks:
- Extraction adapter (
_suggestion_adapter.py,_materializer.py) still emits drafts WITHOUT new evidence fields. Opt-in design keeps them working; tracked in P2-002-FU-A (Backlog). - No filter param on
GET /api/v1/semantic/suggestionsforproposed_actionyet; tracked in P2-002-FU-B (Backlog). Migration adds the index in anticipation. - Conflict catalogue (
_LOW_CONFIDENCE_ACTIONS,_HIGH_CONFIDENCE_ACTIONS) is intentionally short; widen once real-worldproposed_actionvalues are seen. - Domain exceptions live in
_suggestion_evidence.py(private-named) — promote when first cross-module caller appears.
P3-001: Redesign suggestions inbox around source / confidence / type / tier / reviewer filters¶
- Phase: Phase 3
- Area: Frontend
- Owner: Frontend Agent (
general-purpose, agentIdaf59ee7866ca0184b) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 3" task P3-001 + §"Suggestions Inbox" UI proposal.
- Dependencies: P2-002 (evidence shape, Done), P2-005 (dedup, Done), P2-006 (confidence bands, Done), P2-002-FU-B (
proposed_actionfilter, Done). - Write scope:
- Frontend only — locate the existing suggestions inbox page (per UX0-001's audit:
/ontology/suggestionsand/orcomponents/editor/panels/SuggestionPanel.tsxvs.components/semantic/SuggestionsQueue.tsx). Reconcile per UX0-001 (the two parallel UIs were flagged for unification). - Add filter controls: source asset / source type / confidence band / suggestion kind / governance tier / reviewer (when known). Each filter maps to a
GET /api/v1/semantic/suggestionsquery param. - Display columns include: title, source asset, evidence preview, confidence band (P2-006), proposed action (P2-002), duplicate indicator (P2-005), reviewer status.
- Row-click opens P3-002's suggestion detail panel.
- Acceptance criteria:
- Inbox uses filter controls bound to the backend query params.
- The two parallel suggestion UIs (per UX0-001 residual risk) are reconciled to a single canonical surface.
- Empty / loading / error states.
- Tests + manual browser-check protocol.
- Test evidence:
- Review notes:
- Residual risks:
P3-002: Suggestion detail panel with evidence, duplicate candidates, and impact preview¶
- Phase: Phase 3
- Area: Frontend
- Owner: Frontend Agent (
general-purpose, agentIda60f04ab07e70cea8) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 3" task P3-002.
- Dependencies: P2-002 (evidence), P2-005 (dedup), P2-006 (bands).
- Write scope:
- New
<SuggestionDetailPanel>component (Drawer or Modal — match existing primitive). Renders the full suggestion: payload, evidence (source_span,rationale,proposed_action,confidence_band), duplicate-of link (whenis_duplicate_ofis set), and an impact preview (what gets created if accepted). - Standalone — can be rendered by P3-001's inbox on row-click AND by a deep-link route.
- Action buttons: accept / dismiss / request-changes (real handlers wire to backend; P3-003 builds the batch version).
- Acceptance criteria:
- Renders every evidence field with revenue/billing example.
- Renders duplicate link when
is_duplicate_ofis set. - Action buttons present and call typed API methods.
- Empty / loading / error states.
- Test evidence:
- Review notes:
- Residual risks:
P3-003: Batch accept / dismiss / request-changes workflows¶
- Phase: Phase 3
- Area: Backend + Frontend
- Owner: Full-stack Agent (
general-purpose, agentIdaabdeea892c858599) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium-High (touches accept path + new request-changes endpoint per P0-003 gap)
- Source: Proposal §"Phase 3" task P3-003. P0-003 API audit flagged missing
POST /suggestions/{id}/request-changes. - Dependencies: P3-001 (inbox checkbox / selection UI), P3-004 (decision events).
- Write scope:
- Backend: new
POST /api/v1/semantic/suggestions/batch-accept,batch-dismiss,batch-request-changesendpoints. Also the missing single-itemPOST /{id}/request-changesroute. - Frontend: bulk-action toolbar in the inbox; per-row checkboxes; confirm-dialog before applying.
- Acceptance criteria:
- All three batch actions land via single backend transaction per batch. ✅
- Per-row results surfaced (partial-success allowed). ✅
- Frontend honours busy / error / partial-success states. ✅
- Test evidence:
- Single-item
POST /api/v1/semantic/suggestions/{id}/request-changesroute added — closes the P0-003 audit gap (P3-002's detail panel had been calling a 404 endpoint until now). - Batch route
POST /api/v1/semantic/suggestions/request-changes-batchadded (companion to pre-existingaccept-batch/dismiss-batch). Naming uses repo's existing kebab pattern. - New helper module
src/pycharter/persistence/collaboration/_suggestion_batch.pyhostsSuggestionBatchResult/SuggestionBatchFailuredataclasses +run_batch_lifecycle()dispatcher — keepssuggestions.pyfrom drifting further past 400 lines. SuggestionManager.dismiss_batch/request_changes_batchadded alongside existing single-item methods. Per-id failures (404/409/422/500) recorded without aborting the batch. Caller owns commit; each successful item flushes status + P3-004 review-event in the same session.- Pydantic models extended:
RequestChangesSuggestionRequest,BatchRequestChangesRequest;SuggestionResponse.statusLiteral widened to includerequest_changesso the new lifecycle status serializes cleanly. - Frontend:
SuggestionsQueue.tsxwires all three bulk actions through a single confirmation-dialog gated dispatcher (ConfirmBulkActionDialog). New "Request changes" bulk button withMessageSquareWarningicon. Per-row error icon (AlertCircle+data-erroredstyling) keyed byrowErrorsmap for partial-failure UX. Toast +<details>collapsible enumerate[code] detailper failed id. api.suggestions.requestChangesBatch(ids, actor, comment?)added;requestChangesreturn type tightened toSuggestion.- Backend tests (14 new):
TestRequestChanges(4),TestBatchRequestChanges(4),TestSuggestionManagerBatchLifecycle(6). - Frontend tests (3 new + 1 updated): confirm dialog walkthrough, cancel preserves selection, request-changes-N path, partial-failure row error rendering.
- Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean;
pytest tests/unit/ -q --timeout=60— 2607 passed, 1 skipped, 8 deselected;npm run lint— 0 errors;npm run type-check— clean;npm run test— 519 passed;npm run build— production build succeeds (all editor + ontology routes prerender static). - Review notes:
- Approved on orchestrator independent re-run. Partial-success contract is internally consistent (per-id 404/409/422/500 captured without poisoning the batch transaction). New
_suggestion_batch.pyhelper is the right factoring — keepssuggestions.pyunder the soft cap. - Residual risks:
- Endpoint naming uses the existing
accept-batch/dismiss-batch/request-changes-batch(kebab-suffix) pattern. The card text proposedbatch-acceptetc.; the suffix variant is what shipped for parity with pre-existing routes. dismiss_batch/request_changes_batchflush per item but route owns the commit. A defensiveExceptioninside the helper leaves prior flushes pending until the route commits; FastAPI's default request-cycle then rolls the entire batch back on driver-level errors. No half-commit risk in normal operation.- Idempotency on retry: previously-resolved rows surface as 409 in a re-issued batch (UI treats as "skipped"). A future follow-up could surface "n/m already resolved" copy distinctly.
- i18n: per-action labels live in a local
BULK_LABELSmap inSuggestionsQueue.tsx— natural extraction point when localisation is added.
P3-004: Review decision events (actor / decision / comment / timestamp)¶
- Phase: Phase 3
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda9fb7018a1344d720) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 3" task P3-004.
- Dependencies: None (new table; backward-compatible).
- Write scope:
- New
SemanticReviewEventModel(tablepycharter.semantic_review_events) with columns:idUUID,suggestion_id(soft string ref),actorString(255),decisionString(32) —accepted/dismissed/request_changes,commentText nullable,timestampDateTime(tz). Indexed on(suggestion_id, timestamp). - Service module under
src/pycharter/persistence/collaboration/for the event log. - Wire
SuggestionManager.acceptandSuggestionManager.dismiss(and the newrequest_changesaction from P3-003) to emit an event row in the same transaction. - Migration revising the current head.
- Tests covering: every decision shape; restart-survival; multi-event timeline per suggestion.
- Acceptance criteria:
- Every suggestion accept/dismiss/request_changes emits a durable event row.
- Existing accept/dismiss APIs continue to work (events are additive).
- Migration is reversible.
- Test evidence:
- Review notes:
- Residual risks:
P3-005: Define direct-accept rules for free-tier suggestions¶
- Phase: Phase 3
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIdace6b1e37f01485be) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: Proposal §"Phase 3" task P3-005.
- Dependencies: P3-004 (events).
- Write scope:
- Define the canonical "free-tier" rules: per the proposal —
confidence >= 0.9AND no duplicate ANDproposed_action == "add_concept"AND tier is free. - Apply at
SuggestionManager.accepttime: if rules pass, materialise the concept immediately; otherwise route to review. - Tests covering each rule axis.
- Acceptance criteria:
- Direct-accept rules documented + configurable (constants OK; document where to change). ✅
- Rule violations route to manual review unchanged from today. ✅
- Tests cover each rule axis. ✅
- Test evidence:
- New module
src/pycharter/persistence/collaboration/_direct_accept_rules.py(218 lines). Module-level constants_FREE_TIER_MIN_CONFIDENCE=0.9,_FREE_TIER_ALLOWED_ACTIONS={add_concept, add_relationship}. Frozen-slotsDirectAcceptDecision(eligible: bool,failing_rules: list[str],rationale: str). Pureevaluate_direct_accept(suggestion) -> DirectAcceptDecision— no DB access, no raises. - Rule set:
status == "pending",tier == "free"(read frompayload["tier"]with"free"fallback — mirrors_materialise_conceptconvention),confidence_score >= 0.9,is_duplicate_of is None,proposed_action in {add_concept, add_relationship}. SuggestionManager.acceptconsultsevaluate_direct_accept, accepts newbypass_rules: bool = Falsekwarg, returnsDirectAcceptDecision, tags the P3-004 audit-eventcommentper outcome (_compose_accept_comment).- Enforcement mode: silent pass-through with
bypass_rules=Trueopt-out — preserves backward compatibility. The HTTP route is opted in via_acceptance.py(accept_suggestionpassesbypass_rules=Truein both materialization branches) because concept/relation has already been materialized by that point. Once the auto-accept gate is wired pre-materialization in a follow-up, the HTTP path will become rules-gated. - Tier field: no dedicated column on
SuggestionModel; canonical convention ispayload["tier"]with"free"fallback (already used by_materialise_conceptin_acceptance.py:187). Rules mirror that fallback. - Tests:
tests/unit/test_direct_accept_rules.py— 28 tests (1 baseline + 5 rule-axis classes + decision-shape class).tests/unit/test_suggestion_manager.py::TestSuggestionManagerDirectAcceptRules— 4 integration tests (rules-pass, low-confidence pass-through, bypass override, domain-tier pass-through). Existing fixtures extended with highconfidence_score/proposed_action/ evidence to exercise rule-passing path; onebypass_rules=Trueannotation intest_suggestion_dedup.py. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean;
pytest tests/unit/ -q --timeout=60— 2607 passed, 1 skipped, 8 deselected (delta +49 vs. 2558 baseline reflects this card's 28 tests + parallel P3-003 / P3-007). - Review notes:
- Approved on orchestrator independent re-run. Silent pass-through is the right default — keeps today's HTTP behaviour byte-identical while opening the door to a future pre-materialization gate. Rule axis coverage is complete and pure (no side effects), so unit tests are deterministic.
- Residual risks:
- P3-006 coordination.
SuggestionManager.acceptreturn signature changed fromNonetoDirectAcceptDecision. P3-006 (domain/core routing) will need to either keep this return or coordinate a wider signature change. Thebypass_ruleskwarg is the natural layering point for the tier-based routing gate. bypass_rulesopt-in across the HTTP route._acceptance.py::accept_suggestionpassesbypass_rules=Truein both branches because the concept/relation has already been materialized by the timemanager.acceptis reached. Today'sPOST /accepttherefore remains reviewer-driven and is NOT auto-gated by these rules. A future card should pre-evaluate the rules in_acceptance.pybefore materialization to activate the auto-accept gate over HTTP (or move materialization behind the rules check).- Configurability is static. Thresholds live as module constants; runtime tuning would need a settings module.
_acceptance.pywas outside the literal write scope but is the wrapper called by the HTTP route. Two addedbypass_rules=Trueannotations + a comment — minimal and necessary to preserve today's behaviour.
P3-006: Route domain/core suggestions into proposal/review workflow¶
- Phase: Phase 3
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIdacc7e198142bc4d5b) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: Proposal §"Phase 3" task P3-006.
- Dependencies: P3-004 (events). Coordinates with P3-005 (rule precedence — domain/core OVERRIDES direct-accept rules).
- Write scope:
- When a suggestion's tier is
domainorcore, ALWAYS route to thesemantic_proposalsworkflow regardless of confidence. - Hook into
SuggestionManager.acceptafter P3-005's rules (P3-005 short-circuits free-tier; this card adds the parallel routing for higher tiers). - Tests covering tier-based routing.
- Acceptance criteria:
- Domain/core suggestions cannot bypass the proposal/review workflow. ✅
- Free-tier suggestions continue to allow direct-accept per P3-005. ✅
- Tests pin the tier-based routing. ✅
- Test evidence:
- New module
src/pycharter/persistence/collaboration/_proposal_gate.py(~225 lines). Pureevaluate_proposal_gate(suggestion, *, proposal_lookup_fn) -> ProposalGateDecision;ProposalLookupFnProtocol;ProposalRequiredError(PublicFacingError)domain exception;_PROPOSAL_REQUIRED_TIERS = frozenset({"domain", "core"});PROPOSAL_GATE_AUDIT_COMMENTconstant. Pure / read-only / never writes. - Gate placement:
accept_suggestion(in_acceptance.py) callsevaluate_proposal_gatebetween the "row.status != pending" check and the materialization branch (lines 149–165). Gated by newbypass_proposal_gate: bool = Falsekwarg + optionalproposal_lookup_fn. Default lookup:_default_proposal_lookup_factory(session)readspayload["proposal_id"](UUID string), parses, queriesProposalModel.id.SuggestionManager.acceptbody untouched. - HTTP error:
acceptroute catchesProposalRequiredErrorand surfaces 409 viaraise_logged_conflict_from_exception. Structured body:{"detail": {"message": "...", "code": "ProposalRequiredError"}}.accept_batchrecords per-row 409 in the failures list. - Audit trail:
_emit_proposal_gate_auditwrites arequest_changesSemanticReviewEventModelrow viamake_semantic_review_event_repository()withPROPOSAL_GATE_AUDIT_COMMENTon the deny path. Wrapped in defensive try/except — audit emit failures log but do not mask the gate error. Suggestion stayspending(the gate is a routing decision). - Tests:
tests/unit/test_proposal_gate.py— 15 tests across 5 classes (free-tier pass-through, domain/core deny + allow, non-pending defer, frozen-dataclass guard, error-context preservation, lookup-invocation spying).tests/unit/test_semantic_suggestions_api.py::TestProposalGateAcceptRouting— 2 tests (domain-tier accept returns 409 + suggestion stays pending; free-tier accept still 200 + flips accepted). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean;
pytest tests/unit/ -q --timeout=60— 2634 passed, 1 skipped, 8 deselected (delta +27 vs. 2607 baseline). - Review notes:
- Approved on orchestrator independent re-run. Gate is pure / injectable / well-tested. Audit emission is correctly best-effort — the gate decision is the load-bearing behaviour. HTTP 409 mapping uses the audited helper.
- Residual risks:
- Proposal wiring is MVP-only. The
proposalstable has nosuggestion_idFK, so today the default lookup requires the suggestion'spayload["proposal_id"]to be set. In practice no extractor sets this key, so EVERY domain/core suggestion currently fails the gate with 409 — the conservative-and-correct closure of the bypass. Follow-up card should add (a) asuggestion_idcolumn toproposals, (b) an HTTP endpoint to link a suggestion to a proposal, © update the default lookup to use the join. - Composition with P3-005's
bypass_rules. Flow is now:accept_suggestionruns my gate (blocks domain/core without proposal) → materialises → callsmanager.accept(..., bypass_rules=True). The two gates are orthogonal and compose correctly. A newbypass_proposal_gatekwarg exists symmetrically for legacy callers but the HTTP route never sets it. - Audit repository is constructed fresh inside
_emit_proposal_gate_audit. HonoursPYCHARTER_SEMANTIC_REVIEW_EVENT_BACKEND. A future card that wires proper proposal lookups should consider passing areview_event_repositoryin too.
P3-007: Build mapping workbench MVP inside existing editor¶
- Phase: Phase 3
- Area: Frontend
- Owner: Frontend Agent (
general-purpose, agentIda158fd761f5b01822) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: High
- Source: Surgical UI/UX Enhancement Plan; Proposal §"Phase 3" task P3-007.
- Dependencies: UX0-001, UX0-002, P1-001, P1-007-IMPL (mapping projection).
- Write scope:
- New mapping workbench component inside the unified editor. Two-column layout: contract fields on the left, candidate concepts on the right.
- Reads from
MappingProjectionRepositoryvia a typed API client method (the route exists from P1-007-FU-B behind a feature flag — surface it directly here). - Per-field row: current binding state, candidate concepts ranked by confidence, accept/remap actions.
- Reuses existing editor layout, toolbar, panels, tabs, badges, MappingDot, BindingHistoryDrawer.
- Acceptance criteria:
- Users can view contract fields and candidate concepts side by side. ✅
- Users can approve, reject, or revise a field binding. ✅ (Accept functional; Remap/Annotate stubbed pending P3-008.)
- UI uses existing editor layout/toolbar/panels/tabs. ✅
- Empty / loading / error / conflict states. ✅
- Tests + browser-check protocol. ✅
- Test evidence:
- Mount point:
/editor/contracts/[name]/[version]/mapping-workbench(example:/editor/contracts/revenue_invoices/1.0.0/mapping-workbench). - Component tree (10 new files under
src/pycharter/ui/src/components/editor/MappingWorkbench/):MappingWorkbench.tsx(348 lines) — top-level two-column composition, data wiring, Accept action.ContractFieldsList.tsx(106 lines) — left column withMappingDot+BindingChip.CandidateConceptList.tsx(204 lines) — right column with ranked candidates + Accept / Remap / Annotate buttons.CoverageStrip.tsx(62 lines) — header<bound>/<total>progress bar.ConflictBanner.tsx(47 lines) — alert when ≥2 candidates exceedCONFLICT_THRESHOLD=0.75.MappingDot.tsx(67 lines) — local mapping-state dot (palette ported from legacyAssetTreeper Phase 0 vocabulary; legacy KB-store coupling not imported)._LoadingSkeleton.tsx,_helpers.ts,types.ts,index.ts.
- Route entries:
src/pycharter/ui/src/app/editor/contracts/[name]/[version]/mapping-workbench/page.tsx+MappingWorkbenchRouteClient.tsx. - API surface added:
editorApi.getMappingProjection(contractId)→GET /api/v1/editor/contracts/{id}/mapping-projection(MVP wrapper returnsnullon any failure; documents the backend gap so the UI doesn't regress when the endpoint is missing). - MVP read path reuses existing endpoints:
contractsApi.listVersions→contractsApi.get→contractsApi.getFieldBindings→api.semantic.searchConcepts(synthetic confidence ramp until a true ranker is wired). Write path usescontractsApi.patchFieldBindingswith optimistic concurrency (expected_revision). - Action buttons: Accept is functional (patches
field_bindings; toasts success/error). Remap / Annotate are stubs with a "Action wiring lands in P3-008" toast — explicitly documented for the next card. - States rendered: empty (no selection / no fields / no candidates), loading (initial skeleton + per-candidate skeleton with
aria-busy), error (initial alert + retry; inline candidate error + retry), conflict (banner above candidate list), populated (highlighted accepted candidate). - A11y:
role="listbox"+role="option"+aria-selectedon left column. Native<button>rows for keyboard.role="progressbar"on coverage strip.role="alert"on conflict banner + initial-error alert.role="img"+aria-labelonMappingDot. Vocabulary followskb-phase-0-vocabulary.md. - Layout: flow + Flexbox + CSS Grid (
grid grid-cols-1 md:grid-cols-2). Zeroposition: absolutein primary structure. - Reused primitives:
BindingChip(fromcomponents/editor/bindings/),listSchemaFieldshelper,Button,contractsApi,useToastStore,getApiErrorMessage,ApiError. - Tests (19 new Vitest):
MappingWorkbench(3 — populated/select/error+retry/Accept routing),CandidateConceptList(7),ContractFieldsList(4),CoverageStrip(3),ConflictBanner(2). - Quality gates (orchestrator independent re-run on 2026-05-19):
npm run lint— 0 errors, 26 pre-existing warnings (no new ones in workbench);npm run type-check— clean;npm run test— 519 passed;npm run build— production build succeeds; all editor routes prerender static including the new mapping-workbench route. - Review notes:
- Approved on orchestrator independent re-run. Composition matches the surgical-enhancement constraint (no new design language, reuses existing primitives). Stub buttons are clearly labelled and route to P3-008. Synthetic confidence ramp is a sensible MVP affordance — labelled in the residual risks for replacement once a real ranker lands.
- Residual risks:
- P3-008 dependency. Remap / Annotate are toast stubs. Accept currently calls
patchFieldBindingsdirectly (bypasses any lifecycle FSM) — once P3-008 lands, Accept should be re-pointed at the lifecycle action endpoint and the optimistic UI re-audited. - P3-009 dependency. No history affordance on the workbench.
BindingHistoryDrawerexists but is mounted only insideConceptInspectortoday; integration into the workbench is P3-009's responsibility. - Feature-flag relevance. MVP read path does not depend on
PYCHARTER_USE_MAPPING_PROJECTION_FOR_LINEAGE— it readsfield_bindingsdirectly. The flag matters once a forward per-contractmapping-projectionendpoint is added andeditorApi.getMappingProjectionswitches over. - Backend gap:
editorApi.getMappingProjectiontargets/api/v1/editor/contracts/{id}/mapping-projectionwhich does not yet exist. Wrapper returnsnullon failure so the UI does not regress; backend follow-up can add the endpoint without UI changes. - Synthetic confidence.
searchConceptsdoes not return scores; workbench synthesises a 1.0…0.0 ramp from result order for conflict-banner signal. ReplacesyntheticConfidencewith the real score when the projection or a true ranker is wired in. - Mapping state simplification. The left-column dot shows only
unmappedvsmappedtoday;suggested/approved/technical_onlydistinctions come from the binding lifecycle / projection (P3-008/P3-009).
P3-008: Mapping actions (accept / remap / mark technical-only / add annotation)¶
- Phase: Phase 3
- Area: Backend + Frontend
- Owner: Full-stack Agent (
general-purpose, agentIdaae90cb61e4748d36) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 3" task P3-008.
- Dependencies: P3-007 (workbench UI), P1-008 (lifecycle hooks).
- Write scope:
- Backend: action endpoints under
/api/v1/semantic/bindings/that call intoBindingLifecycle.transitionwith the appropriateBindingState. Each action emits aconcept_binding_eventsrow (P1-008 wired the projection updater). - Frontend: action buttons / menus in the workbench rows.
- "mark technical-only" is a binding-state nuance — likely a status flag rather than a concept assignment.
- "add annotation" attaches a freeform note to the binding (new column or annotation table).
- Acceptance criteria:
- Each action surfaces in the workbench UI and reaches the backend. ✅
- Backend emits a
concept_binding_eventsrow per transition. ✅ - Projection updates via P1-007-IMPL's lifecycle hook. ✅
- Test evidence:
- New endpoints (under
src/pycharter/api/routes/v1/semantic_bindings.py):POST /api/v1/semantic/bindings/accept— body{contract_id, field_path, concept_id, actor, comment?, contract_version?}.POST /api/v1/semantic/bindings/remap— body{contract_id, field_path, current_concept_id, new_concept_id, actor, comment?, contract_version?}. Returns{detach_event, attach_event}.POST /api/v1/semantic/bindings/mark-technical-only— body{contract_id, field_path, concept_id, actor, comment?, contract_version?}.POST /api/v1/semantic/bindings/annotate— body{contract_id, field_path, concept_id, actor, comment, contract_version?}(comment min_length=1).- Body shape uses the canonical triple instead of
{binding_id}because P1-008's FSM does not expose a flat identifier — documented in the route module docstring.
- Lifecycle action coverage (P1-008 mapping):
- Accept →
transition(to_state=APPROVED)— fully supported. - Remap → two transitions:
(old_concept) → UNMAPPEDthen(new_concept) → MAPPED— composed from existing FSM (NOT atomic; documented as follow-up if a first-classremapaction is added). - Mark technical-only → no
technical_onlystate in P1-008 today; recorded as a same-state event with[technical_only]prefix inreason. Marker constantTECHNICAL_ONLY_REASON_MARKERexported for P3-009. - Annotate → no annotation state today; recorded as a same-state event with
[annotation] <comment>prefix inreason. Marker constantANNOTATION_REASON_MARKERexported for P3-009.
- Accept →
- Frontend:
- Added
src/pycharter/ui/src/lib/api/semanticBindings.ts— typed wrappers:accept,remap,markTechnicalOnly,annotate. Registered inlib/api/index.tsassemanticBindings. - Added
MappingWorkbench/RemapDialog.tsx,AnnotateDialog.tsx,MarkTechnicalOnlyDialog.tsx— Radix Dialog confirm dialogs with from→to summaries / required textarea (annotate) / dialog copy noting reversibility (technical-only). - Modified
MappingWorkbench.tsx: Accept now callsapi.semanticBindings.accept(drops legacypatchFieldBindingsdirect-write). On success, the workbench refetches bindings. Remap / Annotate / Mark-technical-only buttons trigger the new dialogs. - Modified
CandidateConceptList.tsx: addedonMarkTechnicalOnlyprop + per-row button. Actor sourced fromgetCurrentUserId()JWT identity (toasts error if missing).
- Added
- Tests:
- Backend: 10 new tests across the 4 endpoints in
tests/unit/test_semantic_bindings_route.py(happy + error paths). - Frontend: 12 new tests —
RemapDialog.test.tsx,AnnotateDialog.test.tsx,MarkTechnicalOnlyDialog.test.tsx,lib/semanticBindings.test.ts, plus 2 updatedMappingWorkbench.test.tsxintegration tests (Remap dialog opens, Mark-technical-only confirms → API call).
- Backend: 10 new tests across the 4 endpoints in
- Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean (144 files);
pytest tests/unit/ -q --timeout=60— 2634 passed, 1 skipped, 8 deselected;npm run lint— 0 errors;npm run type-check— clean;npm run test— 547 passed (61 files);npm run build— clean prod build. - Review notes:
- Approved on orchestrator independent re-run. The 4 endpoints are uniformly shaped; lifecycle composition is correct where P1-008 supports it; FSM gaps (
technical_only,annotatestates) are surfaced as reason-prefix markers with exported constants for P3-009 — a clean handoff. - Residual risks:
- Invented transitions: none. Mark-technical-only and Annotate are recorded as same-state events with reason-prefix markers, not new FSM states. Follow-up: add
TECHNICAL_ONLYstate + dedicated annotation row to P1-008 so consumers don't have to parsereasonprefixes. - Mark-technical-only is reversible: doesn't lock the field; any later
accept/remapmoves through the normal lifecycle. Communicated in the dialog copy ("audit event is reversible"). - Remap atomicity: the route emits two events sequentially. If the attach step fails after the detach commits, the field ends up
UNMAPPED. Documented in the route docstring as a follow-up when the FSM grows a first-classremapaction. - File size:
semantic_bindings.pynow 480 lines (soft cap 400). Follow-up: extract the action endpoints + request models intosemantic_bindings_actions.py. - P3-009 coordination: the history drawer reads
BindingEvent.reason. P3-009 formats[technical_only]and[annotation]markers via the exported constants.
P3-009: Mapping lifecycle event display¶
- Phase: Phase 3
- Area: Frontend
- Owner: Frontend Agent (
general-purpose, agentIdaaffbd16a40039294) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Low
- Source: Proposal §"Phase 3" task P3-009.
- Dependencies: P1-008 (lifecycle events exist), P3-008 (events written during workbench actions).
- Write scope:
- New
<MappingHistoryDrawer>or extension toBindingHistoryDrawer(the existing primitive flagged in UX0-002). - Reads from a new API endpoint that returns the lifecycle event log for a given mapping.
- Renders chronological list: actor, action, state transition, comment, timestamp.
- Acceptance criteria:
- Drawer renders events with actor + decision + timestamp. ✅
- Empty state when no events. ✅
- Tests. ✅
- Test evidence:
- Mount strategy: URL-param subscriber. The workbench top bar has a single
HistoryTriggerButtonthat sets?historyForField=<path>viauseRouter().replace. A self-mountedBindingHistorySubscriberwatches the param and renders a Radix Dialog overlay. Closing the dialog clears the param. Merge surface inMappingWorkbench.tsxis a single 10-line additive block bracketed by{/* P3-009: history trigger */}markers between<CoverageStrip>and the grid body — clean coexistence with P3-008's rewire. - Files added:
src/pycharter/ui/src/components/editor/MappingWorkbench/HistoryTriggerButton.tsx(65 lines) — trigger button.src/pycharter/ui/src/components/editor/MappingWorkbench/BindingHistorySubscriber.tsx(138 lines) — URL-param subscriber + Radix Dialog overlay using the extracted view.src/pycharter/ui/src/lib/api/bindingEvents.ts(75 lines) — typed wrappers for binding + review events.- 3 Vitest spec files (
HistoryTriggerButton.test.tsx,BindingHistorySubscriber.test.tsx,bindingEvents.test.ts).
- Files modified:
MappingWorkbench.tsx— additive block + 2 alphabetically-placed import lines.MappingWorkbench/index.ts— barrel re-export.editor/shell/BindingHistoryDrawer.tsx— extracted a presentationalBindingHistoryDrawerView(status / error / events / emptyMessage props). The legacy concept-keyed drawer delegates inner rendering to that view —ConceptInspectorkeeps working unchanged.MappingWorkbench.test.tsx— addedvi.mock('next/navigation', ...)stub (required because the subscriber pulls in the router).
- API methods:
getBindingEvents({contractId, fieldPath})→GET /api/v1/semantic/bindings/events?contract_id=…&field_path=…(existing P1-008 endpoint). Response reversed so the drawer shows newest first.getReviewEvents(suggestionId)→ stub returning[]. No matching backend route exists today (verified insemantic_suggestions.py); module docstring documents the gap, signature pinned so the swap is one line when the backend ships.
- States rendered: empty ("No history for this field yet."), loading (animated skeleton rows in the view), error (
role="alert"banner + Retry button). - A11y: Radix Dialog provides focus-trap + ESC + ARIA modal landmark.
DialogTitleandDialogDescriptionpopulated. Trigger button hasaria-label. Error banner usesrole="alert". Loading skeleton setsaria-busy="true". - Reuse:
BindingHistoryDrawerrow markup fully reused via the new exportedBindingHistoryDrawerView— no duplication. - Tests (11 new):
HistoryTriggerButton(3),BindingHistorySubscriber(5),bindingEvents(3). - Quality gates (orchestrator independent re-run on 2026-05-19):
npm run lint— 0 errors;npm run type-check— clean;npm run test— 547 passed (61 files);npm run build— clean prod build, prerenders all 47 pages. - Review notes:
- Approved on orchestrator independent re-run. URL-param subscriber is the right merge-safety choice — keeps
MappingWorkbench.tsxblast radius minimal.BindingHistoryDrawerViewextraction is the right factoring (presentational view + container preserved). Stub forgetReviewEventsis the correct closure: typed and labelled with an explicit gap note rather than guessing the endpoint shape. - Residual risks:
- Review-events endpoint gap: the spec hints at
GET /suggestions/{id}/review-eventsbut no such route exists.getReviewEventsis a documented stub returning[]; the drawer currently only renders lifecycle events. Backend follow-up needed to surface P3-004 review events alongside lifecycle events. - Drawer extraction:
BindingHistoryDrawer.tsxnow exportsBindingHistoryDrawerView+BindingHistoryStatustype. LegacyBindingHistoryDrawerobservable behaviour preserved; new loading skeleton surfaces on first open of the inline panel insideConceptInspector— minor improvement, not a regression. - P3-008 reason-prefix markers: the drawer surfaces
[technical_only]and[annotation]reason prefixes as raw text today. Follow-up: pretty-format these into dedicated badges usingTECHNICAL_ONLY_REASON_MARKER/ANNOTATION_REASON_MARKERconstants exported by P3-008.
P3-010: Tests for suggestion review → concept/materialization → mapping update¶
- Phase: Phase 3
- Area: QA / Backend
- Owner: Test Engineer Agent (
general-purpose, agentIdac52bc9673f544cd5) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Low
- Source: Proposal §"Phase 3" task P3-010.
- Dependencies: P3-003 (batch endpoints), P3-004 (events), P3-005 (rules), P3-006 (routing), P3-008 (mapping actions).
- Write scope:
tests/integration/— new test(s) that drive the full review→materialise→bind flow.- Acceptance criteria:
- At least one end-to-end test covering the full review → materialise → bind chain. ✅
- Assertions on event log + projection state, not just counts. ✅
- Test evidence:
- New file
tests/integration/test_review_lifecycle_flow.py(630 lines incl. module docstring + helpers), one test classTestReviewLifecycleFlowwith 6 test methods. - Test list / acceptance-criterion mapping:
test_accept_path_free_tier_high_confidence_emits_event— green path. Drivesaccept_suggestionend-to-end against a 0.95-confidence free-tieradd_conceptsuggestion; asserts status →accepted,accepted_concept_idstamped, exactly oneSemanticReviewEventModelrow withdecision="accepted", correct actor, suggestion_id soft-ref, UTC-bounded timestamp.test_dismiss_path_emits_event— status →dismissed, reason mirrored toprovenance, event emitted with comment.test_request_changes_path_emits_event— status →request_changes, comment preserved.test_batch_mixed_outcomes— 3 pending + 1 prior-dismissed + 1 missing UUID → 3 successes, 2 failures (404 + 409), exactly one new event per success, no event growth on failure rows.test_review_events_survive_session_restart— accepts + dismisses + commits + closes engine + reopens against the same SQLite file; asserts status + events round-trip with stable timestamps.test_multi_event_timeline_per_suggestion(bonus) — request_changes → ORM-reset → dismiss → ordered, append-only 2-row timeline.
- Fixture pattern: Reused from P2-009 (
_alias_sqlite_jsonb(),tmp_pathon-disk SQLite +Base.metadata.create_all+sessionmaker+dispose_alllifecycle,pycharter.db.models.semanticregistration). New:_install_sqlalchemy_review_repo()monkeypatchespycharter.persistence.collaboration.suggestions.make_semantic_review_event_repositorysoSuggestionManagerinsideaccept_suggestionwrites to the same DB the test queries._seed_suggestion()factory produces rule-compliant rows._events_for()and_as_naive_utc()assertion helpers. - Quality gates (orchestrator independent re-run on 2026-05-19):
pytest tests/integration/test_review_lifecycle_flow.py -m integration -v— 6 passed;pytest tests/integration/ -m integration -v— 14 passed, 2 skipped, 3 pre-existingtest_api_pipeline_runs_route.pyfailures (require Postgrespycharterschema; verified pre-existing); ruff clean. - Unit regression vs. 2607 baseline:
pytest tests/unit/ -q --timeout=60— 2634 passed, 1 skipped, 8 deselected (delta +27 from this batch). - Review notes:
- Approved on orchestrator independent re-run. Adapter-driven approach mirrors P2-009. Real SQLAlchemy session round-trip (not mocks) provides genuine restart-survival coverage. The factory monkeypatch is the right seam given the factory is constructed inside
SuggestionManager. - Residual risks:
- SQLite vs. Postgres divergence: test runs against on-disk SQLite (
JSONB → JSONaliased,DateTime(timezone=True)returns offset-naive). Theix_semantic_review_events_suggestion_timestampJSONB index and timezone-aware comparison semantics are not exercised here._as_naive_utcnormalisation keeps assertions portable, but Postgres-specific behaviour stays CI-only. - No P3-006 / P3-008 / P3-009 integration coverage: tier-routing, mapping actions, lifecycle event UI ship concurrently and are not yet exercised in integration tests. Follow-up: a Phase-3 integration tail card covering tier-routed accept gate + mapping action persistence + BindingEventStore once they settle on
develop. accept_suggestionfactory override: the green-path test monkeypatchesmake_semantic_review_event_repositoryin the suggestions module. Ifaccept_suggestionmigrates to argument-injection, the patch site needs updating; if the factory accepts an env-var-only path, the override becomes unnecessary. Documented in the module docstring.
P1-001-FU-B: Evaluate JSON vs JSONB for new KB tables on Postgres¶
- Phase: Phase 1
- Area: Backend / DB
- Owner: Docs / Backend Agent (
general-purpose, agentIdafdc0357d6da88bc8) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Low
- Risk: Low
- Source: P1-001 reviewer report (
kb-phase-1-review.md) + P1-001/002/003 residual risks. - Dependencies: P1-001 (Done).
- Write scope:
docs/guides/kb-phase-1-p1-001-fu-b-jsonb-decision.mdonly (decision record). ✅ - Acceptance criteria:
- Decision record with stated rationale. ✅ — STAY (keep
JSON). - Codebase survey for current
JSON/JSONBusage. ✅ — ZeroJSONB()columns in~40ORM models; zero JSON path operators against the five new KB columns. - Test evidence:
- 2026-05-18 —
docs/guides/kb-phase-1-p1-001-fu-b-jsonb-decision.md(143 lines). - Decision: STAY. No follow-up migration card needed. Residuals retired from
kb-phase-1-review.md,kb-phase-1-p1-002-review.md,kb-phase-1-p1-003-review.md. - Review notes:
- Approved via orchestrator independent re-run. Recommended one-sentence annotation on affected model docstrings the next time a card legitimately edits them; not blocking.
- Residual risks:
- Stale
JSONBcomment incontract_store/_sqlalchemy_connection.py:106-107— one-line cleanup, out of scope.
ENV-001: Tighten upper bounds on framework deps in pyproject.toml¶
- Phase: Dev-env debt (outside KB workflow scope; tracked here for visibility)
- Area: Build / DX
- Owner: Build Agent (
general-purpose, agentIdae64add714a2e447c) - Reviewer: Orchestrator (independent re-run)
- Status: Done (committed
b586e885onchore/dev-env-debt-cleanup, PR #171; cherry-picked ontofeat/kb-phase-0as285b63d2) - Priority: High
- Risk: Low
- Source: Surfaced during P1-001-FU-A push attempt on 2026-05-18.
- Dependencies: Optional — PR #170 (band-aid) may be closed in favour of #171.
- Write scope:
pyproject.toml[project.dependencies]+[project.optional-dependencies]only. ✅ - Acceptance criteria:
- Upper bounds added on
fastapi (<0.130),alembic (<2.0),sqlalchemy (<3.0),pydantic (<3.0),pydantic-settings (<3.0),httpx (<1.0),uvicorn (<1.0),PyJWT (<3.0). ✅ - Policy comment block added explaining when to lift caps. ✅
pip install --dry-run -e .[dev]resolves cleanly. ✅- Test evidence:
- 2026-05-18 —
pyproject.toml(+31, -12). 8 deps capped. python -c "import tomllib; tomllib.loads(...)"— valid.ruff check pyproject.toml— clean.pip install --dry-run -e .[dev]— resolves; in-scope versions: fastapi 0.129.0, pydantic 2.12.4, sqlalchemy 2.0.43, alembic 1.18.3, httpx 0.28.1, PyJWT 2.12.1, uvicorn 0.40.0.- Review notes:
- Approved via orchestrator independent re-run. Sibling packages (pystator/pyoptima) may want the same pattern for fleet parity — flag for follow-up if not already in their kanbans.
- Residual risks:
fastapi <0.130intentionally freezes the installed-today version; lifting requires a dedicated PR with release-notes review.
ENV-002: Reclassify Postgres-dependent tests out of tests/unit/¶
- Phase: Dev-env debt
- Area: Tests / DX
- Owner: Tests Agent (
general-purpose, agentIda345ceac2f037343d) - Reviewer: Orchestrator (independent re-run + push-gate validation)
- Status: Done (committed
b586e885onchore/dev-env-debt-cleanup, PR #171; cherry-picked ontofeat/kb-phase-0as285b63d2) - Priority: High
- Risk: Medium
- Source: Surfaced during P1-001-FU-A push attempt —
tests/unit/test_api_pipeline_runs_route.pyrequires Postgres but lives in the unit tree. - Dependencies: None.
- Write scope:
git mvof one test file +pytestmarkaddition. ✅ - Acceptance criteria:
pytest tests/unit/passes without Postgres up. ✅ (Validated bygit pushitself running the pre-push hook clean —Local CI parity (pre-push) ... Passed.)- Existing
[tool.pytest.ini_options]already hadaddopts = ["-m", "not integration and not slow"]— no config change needed. ✅ - Integration coverage preserved (test still runs under
pytest -m integration). ✅ - CONTRIBUTING.md documents the unit/integration split (delivered by ENV-004). ✅
- Test evidence:
- 2026-05-18 — Renamed
tests/unit/test_api_pipeline_runs_route.py→tests/integration/test_api_pipeline_runs_route.py(93% similarity preserved); addedpytestmark = pytest.mark.integration. - Audit summary: 156 modules in
tests/unit/. Only 1 was genuinely Postgres-bound; 155 correctly useapp.dependency_overrides[get_db_session]or monkeypatched session resolvers (SQLite-backed). pytest tests/unit/with fakePYCHARTER_DATABASE_URL— 2279 passed, 1 skipped, 8 deselected in 16.47s.- Push-gate validation:
git push origin chore/dev-env-debt-cleanupran the pre-push hook and it passed — first successful push gate without--no-verifysince the issue surfaced. - Review notes:
- Approved via orchestrator independent re-run + the push-gate-passing-itself as the strongest possible regression test.
- Residual risks:
- Postgres schema setup is a separate problem: the moved tests require
pycharter db upgradeagainst the target Postgres before they pass. Today the auto-migration inget_db_sessiononly repairsconcepts.scheme_id, not the broader schema. Flagged for a future card (extend auto-migration, or add a session-scoped fixture undertests/integration/that runs Alembic head before any router test fires).
ENV-003: Add a reproducible lockfile (uv or pip-compile) — pycharter pilot¶
- Phase: Dev-env debt (outside KB workflow scope; tracked here for visibility)
- Area: Build / DX
- Owner: Build Agent (
general-purpose, agentIdad68c77b57f39e078) - Reviewer: Orchestrator (independent re-run)
- Status: Done (pycharter pilot — fleet-wide propagation to pystator / pygubernator / pyactuator is a separate per-repo decision)
- Priority: Medium
- Risk: Medium
- Source: Surfaced during P1-001-FU-A push attempt — open-bounded version pins and no lockfile mean every
pip install -e .[dev]produces a different env depending on the day. - Dependencies: ENV-001 (upper bounds — Done).
- Tool choice:
uv(Astral). Same vendor asruffwhich the repo already uses. Adopted as a pycharter pilot; fleet-wide convention will be decided once the pilot soaks. Alternatives considered:pip-compile(more conservative, slower),poetry(heavier, opinionated). - Write scope:
- Add
uv.lock(generated, reproducible). - Add
uvto the[dev]extra inpyproject.toml. - Add a new
test-lockfileCI job alongside the existingtestmatrix (additive — existing pip-install path stays the canonical fast pre-push gate). - Add
lockandsynctargets toMakefile. - Document the dep-bump workflow in
CONTRIBUTING.mdalongside the existing pre-push contract. - Do NOT break the existing
pip install -e .[dev]workflow. It must continue to work. - Acceptance criteria:
uv.lockgenerated, reproducible (runninguv locktwice produces byte-identical files). ✅ (Verified.)- CI exercises both install paths (existing pip + new uv
sync --frozen). ✅ - Dependency-bump workflow documented in
CONTRIBUTING.md. ✅ - Pre-push hook still passes without change. ✅
- Test evidence:
- Files added (1):
uv.lock(5,842 lines, 1,003 KiB, 256 packages resolved). Reproducible:diff -qclean across twouv lockruns. - Files modified (4):
pyproject.toml(adds"uv>=0.5.0,<1.0"to[dev]extra, same pin style as siblings),.github/workflows/ci.yml(newtest-lockfilejob; existing matrix untouched),Makefile(newlock+synctargets),CONTRIBUTING.md(new "Dependency management withuv" section documenting both install paths and the 3-step dep-bump workflow). - CI workflow design: new
test-lockfilejob as a top-level entry (not a matrix entry). Steps: checkout → setup-python 3.12 →pip install uv→uv lock --locked(drift gate; fails if lockfile is stale) →uv sync --frozen --all-extras→ ruff + pytest. The existingtestmatrix (3.11/3.12/3.13) is byte-identical — no regression risk on a required status check. - Quality gates (orchestrator independent re-run on 2026-05-19):
uv --version→ 0.11.14;uv lockreproducible;uv lock --lockedclean;uv sync --frozen --dry-runresolves;pip install --dry-run -e .[dev]resolves;pre-commit run --all-filespasses;python -c "import yaml; yaml.safe_load(open('.github/workflows/ci.yml'))"valid. - Review notes:
- Approved via orchestrator re-run. Pycharter is the first optophi package to adopt
uv. Existing pip workflow preserved; pilot can be unwound by removing the lockfile and thetest-lockfilejob if needed. - Residual risks:
- Lockfile diff size on bumps:
uv.lockis ~1 MB. Routine transitive bumps produce ~20-line diffs;uv lock --upgradeproduces thousand-line diffs. Mitigation: bump in dedicated commits. - No Renovate / Dependabot for
uv.lockyet: existing dependabot covers pip + github_actions ecosystems. Dependabot supportsuv(GA 2025); adding it here is a follow-up. - CI lockfile job uses Python 3.12 only: existing test matrix covers 3.11/3.12/3.13. Locking against all three Python versions is an easy future enhancement (matrix on the lockfile job).
- Fleet alignment is open: pystator / pygubernator / pyactuator have not adopted
uvyet. This pilot doesn't force them to; future fleet decision can converge or diverge. make syncrequiresuvon PATH: first-time contributors who runmake syncbeforepip install -e .[dev]will get "uv: command not found". Documented inCONTRIBUTING.md.
ENV-004: Document the pre-push contract and local dev prerequisites¶
- Phase: Dev-env debt
- Area: Docs / DX
- Owner: Docs Agent (
general-purpose, agentIda67938caf090a03da) - Reviewer: Orchestrator
- Status: Done (committed
b586e885onchore/dev-env-debt-cleanup, PR #171; cherry-picked ontofeat/kb-phase-0as285b63d2) - Priority: Medium
- Risk: Low
- Source: Surfaced during P1-001-FU-A push attempt.
- Dependencies: ENV-002 (so the documented "what to run before push" matches the new unit-vs-integration split). ✅
- Write scope:
CONTRIBUTING.md(append) + one-line cross-reference inAGENTS.md. ✅ - Acceptance criteria:
- "Before you push" section in CONTRIBUTING.md. ✅
- "When tests fail at push time" diagnosis tree. ✅
- AGENTS.md cross-reference. ✅
- Real commands documented (
./scripts/ci.sh --no-ui,docker compose up -d postgres,pip install -e ".[dev]",pre-commit install --hook-type pre-push). - Test evidence:
- 2026-05-18 —
CONTRIBUTING.md(+70 lines),AGENTS.md(+2 lines). - All documented commands grep-verified to exist (
scripts/ci.sh,docker-compose.ymlpostgres service,Makefile install-dev,pre-commit install --hook-type pre-pushin.pre-commit-config.yamlcomments). - Review notes:
- Approved.
- Residual risks:
- The cross-referenced kanban file (
docs/guides/kb-ontology-workflow-kanban.md) does not yet exist ondevelop(it lives onfeat/kb-phase-0). The link will resolve oncefeat/kb-phase-0merges. Document this caveat if it lingers.
P1-005: Link suggestions to source asset and ingest run IDs¶
- Phase: Phase 1
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIdae358d1de645b78bc) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIdaaf876ed9ba474845) - Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 1" task P1-005.
- Dependencies: P1-002 (Done), P1-003 (Done).
- Write scope: As declared — observed clean.
- Acceptance criteria:
- Suggestions can carry
source_asset_idandingest_run_id. ✅ - Existing suggestion API and DB shape stays backward compatible — both columns nullable, defaults preserved. ✅
- Tests prove the linkage round-trips and that lookups via the new columns work. ✅
- Migration is reversible. ✅
- Test evidence:
- Canonical suggestion model:
src/pycharter/db/models/semantic/suggestion.py→SuggestionModel(tablesuggestions, schemapycharter). Confirmed canonical via Phase 0 audit. - Files modified (2):
src/pycharter/db/models/semantic/suggestion.py(+2 columns + docstring);src/pycharter/persistence/collaboration/suggestions.py(SuggestionManager.create / create_batch / list / _to_dictaccept optionalsource_asset_id/ingest_run_id; defaults preserved). - Files added (3): migration
20260518000003_add_suggestion_source_traceability.py(100 lines, revisionsuggestion_source_traceability,down_revision = asset_ingest_run_link, idempotent_has_columnguards, reversible downgrade);tests/unit/test_suggestion_source_traceability.py(264 lines, 7 tests);tests/integration/test_suggestion_source_traceability_restart.py(113 lines,@pytest.mark.integration). - Out-of-scope check: No API route, UI, asset-lineage, or KB-management source touched. Sibling files (
semantic_activity_api.py,semantic_provenance_api.py, etc.) that constructSuggestionModel(...)for fixture setup pass unchanged (defaults remainNone). - Quality gates (orchestrator independent re-run on 2026-05-18): ruff clean; mypy clean (30 files); new unit (7 of combined 27) pass; new integration pass; full unit suite — 2366 passed, 1 skipped, 8 deselected (zero regression vs. P1-003 baseline).
- Review notes:
- APPROVE (
kb-phase-1-traceability-review.md): all 4 acceptance bullets satisfied; soft-reference columns confirmed atsuggestion.py:81-82; migration chain verified by direct module load; 7 unit + 1 integration test green; 46-test regression sweep clean; CLAUDE.md hygiene complete. Reviewer recommended a tiny follow-upP1-005-FU-Ato surface the new columns onSuggestionResponseover HTTP. - Residual risks:
- Loose UUID format on the new soft-reference strings — same as
SemanticAssetModel.ingest_run_id; no UUID validation at write time. Program-consistent. - No referential integrity at DB layer — a suggestion can carry IDs that don't resolve. Joins must tolerate dangling refs.
- Open question (orchestrator/PM): should a future Phase 2 card add
validate_lineage=Trueopt-in service flag mirroringUnknownConceptSchemeError? - Open question: should the missing
POST /suggestions/{id}/request-changesroute (Phase 0 audit gap) also persistsource_asset_id/ingest_run_idon the change-reviewer's audit trail? - Open question: should
api/models/semantic_suggestions.py:SuggestionResponsesurface the two new fields? Out of P1-005's scope; would be a tiny follow-up (P1-005-FU-A).
P1-007: Durable mapping projection for field-to-concept bindings (scoping)¶
- Phase: Phase 1
- Area: Backend (scoping first)
- Owner: Docs / Backend Agent (
general-purpose, agentIdaa261750e291aa60a) - Reviewer: Orchestrator (independent re-run)
- Status: Done (scoping pass — implementation tracked in P1-007-IMPL below)
- Priority: Medium
- Risk: Medium
- Source: Proposal §"Phase 1" task P1-007 ("if not already sufficiently represented").
- Dependencies: None for scoping pass.
- Write scope:
docs/guides/kb-phase-1-p1-007-mapping-projection-decision.mdonly. ✅ - Acceptance criteria:
- Decision record exists with implement-vs-retire conclusion + cited evidence. ✅ — IMPLEMENT.
- Test evidence:
- 2026-05-18 —
docs/guides/kb-phase-1-p1-007-mapping-projection-decision.md(342 lines). - Gaps documented vs. proposal:
field_mappings.mapping_datais per-contract JSON blob (not per-binding);field_conceptsis lossy ingest-path-only;concept_binding_eventsis history log, not current-state projection;editor_lineage.py:163-182flags an O(contracts × bindings) JSON scan that needs a pre-computed reverse index. - Review notes:
- Decision: IMPLEMENT. Promoted P1-007-IMPL to Backlog (below).
- Residual risks:
- Open question (PM): versioned rows vs. current-only — proposal's "mapping history visible" argues for versioned +
is_currentflag. - Open question (PM): store unresolved bindings —
UnresolvedBindingsemantics argue yes. - Open question (PM): write-through vs. overlay — proposal §4 says "or"; recommend write-through in Phase 1, defer overlay to Phase 3.
P1-007-IMPL: Implement durable mapping projection (Protocol + factory + SQLAlchemy + in-memory)¶
- Phase: Phase 1
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIdaf94d78a6f94b6383) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIda41e0689189645f69) - Status: Done
- Priority: Medium-High
- Risk: Medium-High (touches existing field-mapping write paths) — landed cleanly
- Source: P1-007 scoping decision (
kb-phase-1-p1-007-mapping-projection-decision.md). - Dependencies: P1-007 (Done — scoping). P1-001 / P1-003 (Done — Protocol+factory+SQLAlchemy+in-memory pattern is the template).
- Write scope: As declared — observed clean. No API route, no UI, no protected source (
knowledge_base/,asset_lineage/, existingfield_mapping.pymodel,concept_binding_eventstable) touched. The two wiring changes (_contract_field_bindings.py+lifecycle.py) are the smallest possible additive integrations. - Acceptance criteria:
MappingProjectionModel+mapping_projectionsmigration created. ✅ (Migrationmapping_projections,down_revision = suggestion_source_traceability.)src/pycharter/semantic/mapping_projection/package with Protocol + factory + SQLAlchemy + in-memory + lifecycle hook + value objects + errors. ✅- Default backend
"memory"; env varPYCHARTER_MAPPING_PROJECTION_BACKEND. ✅ - Backfill CLI script with
--dry-run,--force, idempotency. ✅ - Wiring into
update_field_bindings_for_contract(write-through, same transaction) +BindingLifecycle.on_approve(exceptions logged + swallowed). ✅ - Versioned rows +
is_currentflag with flip-then-insert atomicity. ✅ - All P1-001/002/003/FU-A/P1-005 regression suites pass. ✅
ruff check,ruff format,mypy, pytest pass. ✅- Test evidence:
- Files added (15): 1 SQLAlchemy model (
mapping_projection.py, 123 lines) + 1 Alembic migration (83 lines) + new packagemapping_projection/(8 files:__init__.py64,_types.py136,models.py90,errors.py40,_in_memory.py237,_sqlalchemy_mapping_projection.py336,_row_mappers.py41,factory.py122,lifecycle_hooks.py103) + backfill CLI (backfill utility script, 223 lines) + 3 test files (test_mapping_projection_repository.py468,test_mapping_projection_wiring.py295,test_mapping_projection_restart.py115). - Files modified (4):
db/models/semantic/__init__.py(export),db/migrations/env.py(import for autogenerate),api/services/_contract_field_bindings.py(write-through wiring),api/dependencies/lifecycle.py(on_approvecomposition). - Public surface (7 methods):
record_projection,update_state,mark_resolved,list_current_for_contract,list_history_for_field,list_contracts_using_concept,get_projection. is_currentflip-then-insert atomic in both backends (_flip_prior_currenthelper); validated byTestRecordProjection::test_record_flips_prior_current_rowand wiring testtest_second_save_flips_prior_projection_row.- Quality gates (orchestrator + reviewer independent re-runs on 2026-05-18):
- ruff check / format on all touched paths — clean.
mypy src/pycharter/semantic/mapping_projection/ src/pycharter/db/models/semantic/—Success: no issues found in 30 source files.pytest tests/unit/test_mapping_projection_repository.py -q— 20 passed.pytest tests/unit/test_mapping_projection_wiring.py -q— 6 passed.pytest tests/integration/test_mapping_projection_restart.py -q -m integration— 1 passed.pytest tests/unit/ -q --timeout=60— 2396 passed, 1 skipped, 8 deselected (was ~2370, +26 from new tests; zero regression).
- Review notes:
- APPROVE (
kb-phase-1-p1-007-impl-review.md): all locked decisions honoured; transaction-boundary contract correctly implemented (live session usessession.flush()only;_commit_and_closeno-ops when caller owns the session); migration chain wires cleanly; backfill CLI honours--dry-run/--force/PYCHARTER_DATABASE_URL; wiring is the smallest possible additive change. - One partial:
last_event_idis intentionally alwaysNonefrom theon_approvehook becauseBindingEventdoesn't yet carry the persistedconcept_binding_events.id. Filling the link is deferred to P1-008. - Residual risks:
linkml_slotprimary bindings becomestate="unresolved"—_record_projection_rowsderivesconcept_keyonly fromprimary.concept.concept; whenlinkml_slotis set instead, the projection recordsconcept_key=None, state="unresolved"despite the field being genuinely bound. Tracked in P1-007-FU-A (Backlog).concept_id→concept_keymatching inon_approveis unchecked — the lifecycle hook treats itsconcept_idarg as aconcept_keywhen callinglist_contracts_using_concept. Works today by convention, not by contract.- In-memory default in production loses state on restart — both the contract-PATCH service and lifecycle wiring default to in-memory when the env var is unset. Acceptable for the introduction card but becomes wrong the moment a read route depends on the projection. Tracked in P1-007-FU-B (Backlog).
- Store commits its own session before the projection write — if the projection write raises, the field-mapping blob has already landed in a separate transaction. The wiring test documents this; tightening would require a
ContractStoreClientrefactor (out of scope here). MappingProjectionApprovalUpdater.on_approvefans out to every active contract version —BindingEventlackscontract_version. P1-008 should extend the event payload.
P1-008: Mapping lifecycle events¶
- Phase: Phase 1
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIdab3f2e2ec48c75248) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIdafeba002d4c69008c) - Status: Done
- Priority: Medium
- Risk: Medium
- Source: Proposal §"Phase 1" task P1-008; P1-007-IMPL residual risks.
- Dependencies: P1-007-IMPL (Done). Scope adjusted from "new lifecycle table" to "extend existing
BindingLifecycle+ projection updater" because P1-007-IMPL already established the lifecycle composition pattern and theconcept_binding_eventstable already exists. - Write scope: Lifecycle event payload + hooks + projection updater extension. Disjoint from P1-007-FU-A (
_contract_field_bindings.py) and P1-007-FU-B (editor_lineage.pyroute). - Acceptance criteria:
- Every mapping state transition fires the full projection updater (not just
on_approve). ✅ (on_approve,on_reject,on_remap, pluson_transitiondispatcher.) BindingEventcarriescontract_version(optional; defaults to None for backward compat). ✅BindingEventStore.appendreturns the persisted event id solast_event_idcan be threaded into the projection. ✅- Existing
BindingLifecycle.transitioncallers stay compatible. ✅ (All new fields default to None / no-op hook.) - Test evidence:
- Files modified (5):
src/pycharter/semantic/lifecycle/_types.py—BindingEvent.contract_version: str | None = None.src/pycharter/semantic/lifecycle/_store.py—BindingEventStore.append() -> str | None.InMemoryBindingEventStoresynthesises UUID;SqlBindingEventStoreflushes then commits and returnsstr(row.id).src/pycharter/semantic/lifecycle/_events.py—OnTransitionHookProtocol +BindingLifecycle.on_transitionfield.transition()capturesevent_idfrom store and fireson_transition(event, event_id)after commit.src/pycharter/semantic/mapping_projection/lifecycle_hooks.py—MappingProjectionApprovalUpdater→MappingProjectionLifecycleUpdater(with backward-compat alias). Newon_transitiondispatcher routes toon_approve/on_reject/on_remap;_applyfilters byevent.contract_versionwhen set, falls back to fan-out when None.src/pycharter/api/dependencies/lifecycle.py— wiresBindingLifecycle(on_approve=asserter.on_approve, on_transition=projection_updater.on_transition).
- Files added (1):
tests/unit/test_mapping_projection_lifecycle.py(~360 lines, 13 tests). - State transition map: APPROVED →
"approved"; UNMAPPED →"unresolved"; MAPPED →"mapped"; SUGGESTED →"suggested". - Quality gates (orchestrator + reviewer independent re-runs on 2026-05-18): ruff clean; mypy clean (54 source files); 13 new lifecycle tests pass; full unit suite — 2433 passed, 1 skipped, 8 deselected.
- Review notes:
- APPROVE with follow-ups (
kb-phase-1-completion-review.md): full lifecycle catalogue wired; backward-compat alias preserved; protocol widening clean; end-to-endlast_event_idtest passes; in-memory event store now allocates UUIDs (subtle contract change, no test broken). One follow-up filed: P1-008-FU-A to threadcontract_versioninto the public TransitionRequest payload and internal callers (see Backlog). - Residual risks:
contract_versionplumbing complete in lifecycle layer but no API caller passes it yet —POST /api/v1/bindings/transitionsandupdate_field_bindings_for_contractboth skip it. The new field is a no-op in production until P1-008-FU-A wires it.- In-memory
BindingEventStore.appendnow allocates UUIDs per call — minor contract change; no existing test relies on the previous id-less contract. SqlBindingEventStore.appendnow flushes before commit — required to surfacerow.id; behavior unchanged for callers.concept_idvsconcept_keymatching — preserved from P1-007-IMPL's residual risk; flagged separately as a future fix.
P1-003-FU-A: Tighten link_asset_to_ingest_run miss contract¶
- Phase: Phase 1
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIdadc0a54d7ab387a9a) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Low
- Risk: Low
- Source: P1-003 reviewer report (
kb-phase-1-p1-003-review.md). - Dependencies: P1-003 (Done).
- Write scope:
src/pycharter/semantic/asset_lineage/errors.py+__init__.py+_types.py+_in_memory.py+_sqlalchemy_asset_lineage.py+tests/unit/test_asset_lineage_repository.py. ✅ - Acceptance criteria:
- Decision: raise on missing asset (chosen for consistency with
UnknownConceptSchemeErrorandupdate_ingest_run_status'sLookupError). link_asset_to_ingest_run(asset_id, run_id)raisesUnknownAssetError(asset_id)on missing asset in both backends. ✅- At least one new test pins the miss path for each backend. ✅ (4 new tests.)
- All existing P1-003 tests still pass. ✅
- Test evidence:
- Files modified (6):
errors.py(+UnknownAssetError),__init__.py(export),_types.py(Protocol docstring updated),_in_memory.py+_sqlalchemy_asset_lineage.py(raise on miss),tests/unit/test_asset_lineage_repository.py(4 new tests). - New exception:
UnknownAssetError(AssetLineageError)—AssetLineageError → PyCharterSemanticError. Instance attrasset_id; message includes the offending id. - SQLAlchemy side-effect: malformed UUID input also raises
UnknownAssetError(was a silent no-op before). One-axis 404-mappable contract. - Quality gates (orchestrator independent re-run on 2026-05-18): ruff clean; mypy clean (8 files);
pytest tests/unit/test_asset_lineage_repository.py— 24 passed (+4 vs. P1-003 baseline of 20); full unit suite — 2343 passed, 1 skipped, 8 deselected (was 2339 baseline + 4 new tests). - Review notes:
- Approved via orchestrator independent re-run. P1-003-FU-B's route (
POST /semantic/assets/{id}/link-ingest-runon develop post-#173) will inherit the new behaviour; once_http_errors.pyaudited helper is wired,UnknownAssetError → 404mapping is one line. - Residual risks:
- SQLAlchemy folds malformed UUID into
UnknownAssetError(was previously a separate silent no-op). Reviewer can split into a separateValueError/422 path later if desired; kept single-axis for 404-mappability.
P1-003-FU-B: Wire asset-lineage repository through a FastAPI dependency¶
- Phase: Phase 1
- Area: Backend / API
- Owner: Backend Agent (
general-purpose, agentIdaea6e6c092f37fc75) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIdaaf876ed9ba474845) - Status: Done
- Priority: Medium
- Risk: Medium
- Source: P1-003 reviewer report (
kb-phase-1-p1-003-review.md); mirrors P1-001 → P1-001-FU-A. - Dependencies: P1-003 (Done).
- Write scope: As declared — observed clean.
- Acceptance criteria:
- Default backend remains
"memory". ✅ - SQLAlchemy mode reachable via
PYCHARTER_ASSET_LINEAGE_BACKEND=sqlalchemy. ✅ - In-memory backend has
known_schemesseeded fromconcept_schemeslazily on first request. ✅ - Tests cover both backends through the route layer. ✅ (20 parametrised tests.)
- All P1-001/002/FU-A/P1-003 regression suites still pass. ✅
- Test evidence:
- Routes added (10):
GET /api/v1/semantic/assets(withscheme/asset_type/ingest_runfilters),GET /api/v1/semantic/assets/{id},POST /api/v1/semantic/assets(400 onUnknownConceptSchemeErrorviaraise_logged_bad_request_from_exception),POST /api/v1/semantic/assets/{id}/link-ingest-run,GET /api/v1/semantic/ingest-runs,GET /api/v1/semantic/ingest-runs/{id},POST /api/v1/semantic/ingest-runs,PATCH /api/v1/semantic/ingest-runs/{id}/status(404 onLookupError). - Files added (2):
src/pycharter/api/routes/v1/asset_lineage.py(369 lines);tests/unit/test_asset_lineage_route_factory.py(482 lines, 20 tests). - Files modified (1):
src/pycharter/api/_routers.py— registered new router under/api/v1with_auth_dep, mirroringeditor_knowledge_base.router. - Dependency wiring:
_get_asset_lineage_repositoryresolves viaresolve_backend(). Memory mode yields a module-level singleton with lazy_seed_memory_known_schemesfromConceptSchemeModel.scheme_keyon first request (failures logged DEBUG and swallowed). SQLAlchemy mode constructs request-scoped repo overget_db_session; dependency commits on success, rolls back on exception. Live-session contract honoured (repo usessession.flush()only — confirmed viatest_live_session_does_not_commitin P1-003 regression). - Out-of-scope check: No protected source touched (asset-lineage repo, KB-management repo, suggestion files, UI all clean).
- Quality gates (orchestrator independent re-run on 2026-05-18): ruff clean; mypy clean (30 files); new tests — 20 pass;
test_editor_knowledge_base_route_factory.pyregression — 16 pass;test_asset_lineage_repository.pyregression — 16 pass;test_public_api_surface.py— 17 pass (no surface drift); full unit suite — 2366 passed. - Review notes:
- APPROVE (
kb-phase-1-traceability-review.md): all 5 acceptance bullets satisfied;_DEFAULT_BACKEND = "memory"unchanged; 20 parametrised tests green across both backends;UnknownConceptSchemeError → 400correctly usesraise_logged_bad_request_from_exceptionwithextra_safe_types; rawHTTPExceptioncalls use fixed strings only; public-API surface unchanged (17/17 pass). One non-blocking note:asset_lineage.pyis 405 lines (5 over the 400-line CLAUDE.md guideline) — reviewer opened a P1-003-FU-B-FU-A follow-up to split. - Residual risks:
- Auth model: new router uses
_auth_dep = [Depends(get_current_user)]mirroring sibling editor routes. No rate limits configured (consistent). - Error envelope:
UnknownConceptSchemeErrorsurfaced via auditedraise_logged_bad_request_from_exception(extra_safe_types includes the error class). Not-found paths use plainHTTPException(404, detail=…)with fixed strings. - Memory-mode singleton lifetime:
concept_schemesmutations mid-process aren't picked up (rebuilds on next restart). Productivity convenience — production should flip to SQLAlchemy where validation happens per-call. list_assetswithout filter returns[]— Phase 2 consolidated-API review may want pagination.POST /semantic/assets/{id}/link-ingest-rundoes not validateingest_run_idexists — same soft-reference pattern. P1-003-FU-A (in Backlog) tightens the underlyinglink_asset_to_ingest_runcontract; once that lands, this route can re-raise as 404.
P1-005-FU-A: Surface source_asset_id and ingest_run_id on SuggestionResponse¶
- Phase: Phase 1
- Area: Backend / API
- Owner: Backend Agent (
general-purpose, agentIdaf1e737ee611663db) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Low
- Risk: Low
- Source: P1-005 reviewer report (
kb-phase-1-traceability-review.md). - Dependencies: P1-005 (Done).
- Write scope:
src/pycharter/api/models/semantic_suggestions.py+tests/unit/test_semantic_suggestions_api.pyonly. ✅ - Acceptance criteria:
SuggestionResponseexposessource_asset_id: str | Noneandingest_run_id: str | None. ✅- Existing clients ignoring unknown fields stay compatible. ✅ (Pydantic defaults to
None; OpenAPI surface annotated viaField(..., description=...).) - At least one new test confirms the two fields round-trip through the HTTP API. ✅ (2 new tests in
TestSuggestionResponseTraceability.) - Test evidence:
- 2026-05-18 —
SuggestionResponse+2 fields (Field(default=None, description=...)style).SuggestionListResponseinherits viaitems: list[SuggestionResponse]— no separate edit. - 2 new tests:
test_get_surfaces_source_asset_id_and_ingest_run_id(round-trip viaGET /{id}and list endpoint),test_get_returns_none_when_traceability_absent(null round-trip). - Quality gates: ruff clean; mypy clean;
pytest tests/unit/test_semantic_suggestions_api.py— passes; full unit suite — 2368 passed. - Review notes:
- Approved via orchestrator re-run.
Field(..., description=...)used so FastAPI's OpenAPI export carries field docs. - Residual risks:
- Legacy fields on
SuggestionResponseuse bare type annotations (noField(...)descriptions). Not retrofit per "surgical changes" rule; a future card can lift OpenAPI quality across the whole model.
P1-003-FU-B-FU-A: Split asset_lineage.py route file under 400 lines¶
- Phase: Phase 1
- Area: Backend / API
- Owner: Backend Agent (
general-purpose, agentIda1d2d0e69861fe0f8) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Low
- Risk: Low
- Source: P1-003-FU-B reviewer report (
kb-phase-1-traceability-review.md). - Dependencies: P1-003-FU-B (Done).
- Write scope:
src/pycharter/api/routes/v1/asset_lineage.py+ newasset_lineage_schemas.py. ✅ Test filetest_asset_lineage_route_factory.pyNOT touched. - Acceptance criteria:
- All resulting modules under 400 lines. ✅ (
asset_lineage.py362 lines;asset_lineage_schemas.py61 lines.) - Public router export
asset_lineage.routerpreserved. ✅_routers.pyunchanged. tests/unit/test_asset_lineage_route_factory.pypasses without modification. ✅ (20/20 pass unchanged.)- Test evidence:
- Strategy chosen: B (sibling-schemas extraction). Strategy A (directory split) would have silently broken the test fixture's
asset_lineage._memory_seeded = Falsemutation; Strategy B is surgical. - 4 inline Pydantic request models extracted to
asset_lineage_schemas.py(61 lines, new).asset_lineage.py405 → 362 (-43). - Public surface preserved:
router,_memory_repository,_memory_seeded,_get_asset_lineage_repository,_seed_memory_known_schemesall still on theasset_lineagemodule. - Quality gates: ruff clean; mypy clean (94 source files); 71-test combined regression pass; full unit suite — 2368 passed, 1 skipped, 8 deselected (no public-API drift).
- Review notes:
- Approved via orchestrator re-run. Strategy B is the right call given the existing test contract.
- Residual risks:
editor_knowledge_base.pyis the natural twin; if it grows past 400 lines a similar split applies.- Strategy A remains an option if a future card needs to add another ~80+ lines of route or schema.
P1-007-FU-A: Tighten linkml_slot binding semantics in projection write-through¶
- Phase: Phase 1
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda16f3891398625d0b) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIdafeba002d4c69008c) - Status: Done
- Priority: Medium
- Risk: Low
- Source: P1-007-IMPL reviewer report (
kb-phase-1-p1-007-impl-review.md). - Dependencies: P1-007-IMPL (Done).
- Write scope: As declared — observed clean. No protected source touched.
- Acceptance criteria:
- A
linkml_slotbinding produces a row withstate="mapped"and aconcept_keyderived from the slot identifier. ✅ - Existing
concept-bound behaviour unchanged. ✅ - Backfill script handles both cases. ✅ (Imports the shared
_primary_concept_keyhelper.) - At least one new test covers each path. ✅ (4 new tests total — 2 wiring + 2 backfill.)
- Test evidence:
- Encoding chosen:
linkml://<class>.<slot>— deterministic, fitsString(255), collision-free vs. real concept keys. - Files modified (4):
_contract_field_bindings.py(new_LINKML_SLOT_KEY_PREFIX,_encode_linkml_slot_key,_primary_concept_keyhelpers);backfill utility script(imports_primary_concept_keyso encoding cannot drift);mapping_projection.py(column docstring documents the convention);test_mapping_projection_wiring.py(2 new tests). - Files added (1):
backfill utility test module(110 lines, 2 tests). - Quality gates (orchestrator + reviewer independent re-runs on 2026-05-18): ruff clean; mypy clean; 4 new + 6 existing wiring tests pass; full unit suite — 2433 passed.
- Review notes:
- APPROVE (
kb-phase-1-completion-review.md): encoding convention is collision-free and well-documented; backfill imports the shared helper so the two paths cannot diverge. - Residual risks:
- Reverse-lookup callers must use the encoded form (
linkml://...); naïve calls with a bare slot won't match. Model docstring documents this. - Pre-fix backfill runs persisted
state="unresolved"for slot-bound primaries. Operators must re-run with--forceto fix in place.
P1-007-FU-B: Wire mapping projection through editor lineage behind a feature flag¶
- Phase: Phase 1
- Area: Backend / API
- Owner: Backend Agent (
general-purpose, agentIdae9aa50ca4b42577d) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIdafeba002d4c69008c) - Status: Done
- Priority: Medium
- Risk: Medium → landed cleanly (default-off feature flag).
- Source: P1-007 scoping decision + P1-007-IMPL reviewer report.
- Dependencies: P1-007-IMPL (Done).
- Write scope: As declared. Production backend flip kept as an operational follow-up, not in this card.
- Acceptance criteria:
- Both code paths return the same result for the same input. ✅ (Parity test pinned.)
- Default behaviour unchanged. ✅ (Flag defaults off; route uses existing JSON-scan path.)
- Tests pin both paths. ✅ (18 new tests across flag-off, flag-on memory, flag-on sqlalchemy, parity, active-contract filter, annotation relationship surface.)
- Test evidence:
- Feature flag:
PYCHARTER_USE_MAPPING_PROJECTION_FOR_LINEAGE. Default unset/falsy → JSON-scan path. Truthy set ("1","true","yes","on", case-insensitive) → projection path. Read per-request via_use_projection_path()so flips need no restart. - Files modified (1):
src/pycharter/api/routes/v1/editor_lineage.py(+185/−25). New_get_mapping_projection_repositorydependency mirrorseditor_knowledge_base._get_kb_management_service; read-only route so no commit/rollback wrap. New_projection_lineagealternate path. Legacy logic refactored into_scan_json_lineagefor a clean dispatcher handle. - Files added (1):
tests/unit/api/test_editor_lineage_projection_flag.py(557 lines, 18 tests). - Quality gates (orchestrator + reviewer independent re-runs on 2026-05-18): ruff clean; mypy clean; 18 new tests pass; existing
test_editor_lineage.pyregression — 10 passed; full unit suite — 2433 passed. - Review notes:
- APPROVE (
kb-phase-1-completion-review.md): flag defaults off; parity pinned between paths including active-contract filter and annotation relationship surface; both backend paths covered. - Residual risks:
- Two-query approach for active-contract filter — projection lacks a
statuscolumn, so the projection path doeslist_contracts_using_concept+_active_contract_keys. Both indexed; consider denormalising in a follow-up if perf matters. - Mid-process flag flip without backfill returns empty results — if
PYCHARTER_USE_MAPPING_PROJECTION_FOR_LINEAGE=truebutPYCHARTER_MAPPING_PROJECTION_BACKEND=sqlalchemyis not flipped or backfilled, the in-memory backend returns empty. Rollout sequence must be: flip backend + backfill, then flip lineage flag. - Operational flip + rollout doc — release-notes section was NOT added in this PR (
docs/guides/kb-phase-1-p1-007-fu-b-lineage-rollout.md). Tracked in P1-007-FU-B-DOCS (Backlog).
P1-008-FU-A: Thread contract_version from API and service callers into BindingLifecycle¶
- Phase: Phase 1
- Area: Backend / API
- Owner: Backend Agent (
general-purpose, agentIda678918580f826bf5) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Low
- Source: P1-008 reviewer report (
kb-phase-1-completion-review.md). - Dependencies: P1-008 (Done).
- Write scope:
src/pycharter/api/routes/v1/semantic_bindings.py+tests/unit/test_semantic_bindings_route.pyonly. ✅ - Acceptance criteria:
- Public API accepts optional
contract_version. ✅ (TransitionRequestextended.) - Existing payloads stay valid. ✅ (Field defaults to None.)
- End-to-end test pins per-version filter activates. ✅ (2 new tests.)
BindingLifecycle.transitionsignature already acceptedcontract_version(added by P1-008); no signature change needed.- Test evidence:
- Files modified (2):
src/pycharter/api/routes/v1/semantic_bindings.py— addedcontract_version: str | None = NonetoTransitionRequestandBindingEventResponse; threaded throughlifecycle.transition(...)andBindingEventResponse.from_event(...).tests/unit/test_semantic_bindings_route.py— addedTestContractVersionEndToEndclass with 2 tests. - Service caller finding:
update_field_bindings_for_contractdoes NOT callBindingLifecycle.transition— it writes projections directly viarepo.record_projection(...). The only in-tree lifecycle caller is the route handler; no other plumbing needed. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean (100 source files);
pytest tests/unit/test_semantic_bindings_route.py— 6 passed; full unit suite — 2435 passed, 1 skipped, 8 deselected (baseline 2433 + 2 new). - Review notes:
- APPROVE via orchestrator re-run. End-to-end per-version filter pinned:
test_contract_version_in_payload_filters_projection_writesconfirms only v1.0.0 row flips;test_omitted_contract_version_preserves_fan_outconfirms backward compat. - Residual risks:
- SQLAlchemy backend not covered by the new end-to-end test (uses in-memory); the version-filter logic is backend-agnostic in
MappingProjectionLifecycleUpdater._applyper P1-008 tests.
P1-007-FU-B-DOCS: Document mapping projection rollout sequence + vocabulary update¶
- Phase: Phase 1
- Area: Docs
- Owner: Docs Agent (
general-purpose, agentIda352fc0d33fd6536c) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Low
- Risk: Low
- Source: P1-007-FU-B + completion-batch reviewer report.
- Dependencies: P1-007-FU-B (Done), P1-008 (Done).
- Write scope:
docs/guides/kb-phase-1-p1-007-fu-b-lineage-rollout.md(new) +docs/guides/kb-phase-0-vocabulary.md(modify). ✅ - Acceptance criteria:
- Rollout doc exists with numbered steps + rollback + troubleshooting. ✅
- Vocabulary entry for
"unresolved"projection state added with rationale. ✅ - Test evidence:
- Files added (1):
docs/guides/kb-phase-1-p1-007-fu-b-lineage-rollout.md(232 lines). 8 numbered steps with copy-pasteable commands + rollback path + troubleshooting tree. - Files modified (1):
docs/guides/kb-phase-0-vocabulary.md— new row for"unresolved"(projection-only) in the lifecycle status table; "Note onunresolvedvs.unmapped" paragraph explaining the divergence (projection rolls upon_reject+concept_key NULLinto one state;BindingStatekeeps finer transitions); cross-references tomapping_projection/models.pyandlifecycle/_types.py; new row in the Backend → UI Status Map. - Both backend tokens share the same UI label ("Unmapped") and badge colour (gray-300);
unresolvedis operator-facing only. - Review notes:
- Approved via orchestrator re-run. Rollout sequence preserves the reversibility story for each step.
- Residual risks:
- Soak-window durations in steps 5 and 7 are stated as "at least one release cycle" — PM may want a concrete minimum (e.g. 7 days).
- Canary vs. fleet-wide rollout strategy (env var on a single host first vs. everywhere) is not pinned — left as a PM call.
P2-001-FU-A: Pivot extracted_candidates to query suggestions table¶
- Phase: Phase 2
- Area: Backend / API
- Owner: Backend Agent (
general-purpose, agentIda377457141aa6eea4) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Low
- Source: P2-001 reviewer report (
kb-phase-2-foundation-review.md). - Dependencies: P2-001 (Done), P2-002 (Done).
- Write scope:
import_summary.py+ new route handler dep + import-summary tests + asset-lineage-route tests. Disjoint. ✅ - Acceptance criteria:
extracted_candidatesnow reflects suggestion-row count, not asset-row count. ✅- Existing P2-001 tests updated to reflect the new semantic. ✅
- Documentation note clarifying the change in column docstring. ✅
- Test evidence:
- Approach: Option 1 (Protocol injection).
build_import_run_summarygainedsuggestion_counter: SuggestionCounterProtocol | None = Nonekw-only parameter. Countslen(suggestion_counter.list(ingest_run_id=run_id)). Route constructsSuggestionManager(session=db_session)and threads it. No coupling toSuggestionManager.listinternals. - Files modified (4):
import_summary.py,asset_lineage.py(only the new summary route handler),test_import_summary.py(TestSuggestionDerivedCandidates with 4 new tests),test_asset_lineage_route_factory.py(+1 SQL backend test). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff/mypy clean; new tests pass; full unit — 2501 passed.
- Review notes:
- Approved. Protocol-driven DI keeps
asset_lineage/free of a hardSuggestionManagerdependency; tests pass a tiny stub. Mirrors existing package pattern. - Residual risks:
SuggestionManager(session=None)would compile-check cleanly via Protocol; runtimeSuggestionError("No database session configured")if called. Route always supplies a live session; safe in production.extracted_candidatesis now0for runs with assets but no suggestions (previously matched asset count). Intended semantic shift per proposal; downstream UI must update.- New route test is SQLAlchemy-only (in-memory
SuggestionManagerrequires a session).
P2-002-FU-A: Wire extraction adapter to populate suggestion evidence¶
- Phase: Phase 2
- Area: Backend / extraction
- Owner: Backend Agent (
general-purpose, agentIdaf0580fbc49662307) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium → landed cleanly (zero churn on existing extraction tests).
- Source: P2-002 reviewer report.
- Dependencies: P2-002 (Done).
- Write scope:
_suggestion_adapter.py,_materializer.py, + 17 new test additions. Disjoint. ✅ - Acceptance criteria:
- All extracted suggestions land with new evidence fields populated. ✅
- Existing extraction tests continue to pass (zero modifications to existing test methods). ✅
- Test evidence:
- Design choices: (A)
source_spanhierarchical fallback —json_pointer/xpath/cell→line=N,col=A:B→bytes=start:end→ coarsesource_type=<type>. (B)rationaleshort string referencing concept-key + source. (C)proposed_action="add_concept"for concept-kind,"add_relationship"for relation-kind. Both outside the validator's high/low-confidence sets so 0.6 default never conflicts. (D)require_evidence=Truein the materializer's_materialize_as_suggestionspath. - Files modified (4):
_suggestion_adapter.py(new helpers_resolve_source_asset_id,_resolve_default_confidence,_source_span_for_entry,_structured_locator,_concept_rationale,_relation_rationale),_materializer.py(one-linerequire_evidence=True),test_suggestion_adapter.py(+16 tests in newTestExtractedSuggestionEvidence),test_extraction_materializer.py(+1 test). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff/mypy clean; 17 new tests pass; zero churn on existing extraction tests; full unit — 2501 passed.
- Review notes:
- Approved. Adapter populates every evidence field; default-0.6 confidence keeps the validator's conflict catalogue happy.
require_evidence=Trueenforces the contract at the write boundary going forward. - Residual risks:
- Synthetic
source_asset_id— whenresult.metadata["source_asset_id"]is absent, adapter mints a deterministic UUID5 placeholder. Satisfies validator but is NOT a realSemanticAssetModel.id. Follow-up needed to plumb real asset ids throughkb.ingest. proposed_actionset is narrow (add_concept/add_relationship); P2-005 dedup work will add verbs likemerge_with_existing/reject_duplicate.require_evidence=Trueis a one-way commitment — any new adapter forgetting evidence will hard-fail at write time. Mitigated bytest_evidence_validates_via_structured_validator.- Default confidence 0.6 is arbitrary; future tuning per-origin (e.g. tree-sitter higher than docs).
P2-002-FU-B: Add proposed_action filter on GET /api/v1/semantic/suggestions¶
- Phase: Phase 2
- Area: Backend / API
- Owner: Backend Agent (
general-purpose, agentIdacfdbfb3010e1e7cd) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Low
- Risk: Low
- Source: P2-002 reviewer report.
- Dependencies: P2-002 (Done) — index on
proposed_actionalready in place. - Write scope:
semantic_suggestions.pyroute handler +SuggestionManager.list(one kwarg only) + new test class. Disjoint. ✅ - Acceptance criteria:
- Route accepts
?proposed_action=...query param. ✅ - Existing callers without the param see unchanged behaviour. ✅
- At least one test pins the filter. ✅ (4 new tests.)
- Test evidence:
- Route + handler:
src/pycharter/api/routes/v1/semantic_suggestions.py, handlerlist_suggestionson@router.get("")mounted at/api/v1/semantic/suggestions. - Files modified (3): route file (new query param +
Q(default=None, description=...)),suggestions.py(one new kwarg onlist()only;create/create_batch/_to_dict/lifecycle untouched),test_semantic_suggestions_api.py(+4 tests inTestSuggestionsListProposedActionFilter). - Index used:
ix_suggestions_proposed_actionfrom the P2-002 migration. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff/mypy clean; 4 new tests pass; full unit — 2501 passed.
- Review notes:
- Approved. Filter style matches existing string filters on this handler (
status,suggestion_kind,originare also unvalidated). Reviewer flagged value-validation against canonical vocabulary as a future enhancement. - Residual risks:
- No vocabulary validation — unknown verbs pass to SQL and quietly return
[]. Consistent with existing string filters on this handler. - Single-value only — multi-value (
?proposed_action=add_concept&proposed_action=merge) would needlist[str] | Nonewith.in_(...). - NULL semantics — when filter is set, legacy pre-P2-002 NULL rows are excluded. Documented in the query description.
P2-005: Duplicate candidate detection in suggestion generation¶
- Phase: Phase 2
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIdac058b496dce76bc2) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: Proposal §"Phase 2" task P2-005.
- Dependencies: P2-002 (Done).
- Write scope:
SuggestionModel(new column),SuggestionManager.create/create_batch(dedup helper + single_to_dictline for the new column), new migration,SuggestionResponse(one new field), dedup tests. Coordinated with parallel P2-006 (no overlap). ✅ - Acceptance criteria:
- Newly generated suggestions checked against existing concepts and pending suggestions. ✅
- Duplicates flagged (not silently dropped). ✅
- Threshold for "near-duplicate" documented — fuzzy axis deferred to P2-005-FU-A (exact-match dedup has no threshold to tune; documented in residual risks).
- Test evidence:
- Detection axes: Axis 0 (approved-concept) wins over Axis 1 (pending-suggestion same
concept_key) wins over Axis 2 (pending-suggestion samesource_asset_id+source_span). is_duplicate_ofconvention:String(128), nullable=True, indexed. Values:concept:<concept_uuid>orsuggestion:<suggestion_uuid>. Soft reference (no FK), matches the program-wide pattern.- Files modified (3):
suggestion.py(new column + docstring),suggestions.py(_detect_duplicatehelper + threaded throughcreate/create_batch; single_to_dictline for the new field),semantic_suggestions.py(onlyis_duplicate_offield added; coordinated with P2-006'sconfidence_band). - Files added (2): Migration
suggestion_is_duplicate_of(74 lines;down_revision=suggestion_evidence_columns; reversible).tests/unit/test_suggestion_dedup.py(~365 lines, 11 tests across 4 classes). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff/mypy clean; 11 new tests pass; 84 suggestion regression tests pass (zero churn); full unit — 2532 passed.
- Review notes:
- Approved. All 3 axes pinned by tests. Concept-axis wins over suggestion-axis when both could fire (
test_concept_axis_wins_over_pending_axis). - Residual risks:
- Fuzzy matching deferred — exact
payload.concept_keyonly. Drafts without that field land unflagged. Tracked in P2-005-FU-A (Backlog). - Within-batch dedup not exhaustive —
create_batchevaluates against DB state at entry; two same-key drafts in one batch won't flag each other (second sees empty DB until post-loop flush). Documented inline. - Transitivity —
is_duplicate_ofis a single soft reference; reviewer UI should follow the chain. - Race condition — concurrent ingests on the same scheme could each see "no duplicate" and both land unflagged. Acceptable for single-writer-per-scheme today; a partial unique index could enforce it later.
P2-006: Confidence bands (high / medium / low / needs-review)¶
- Phase: Phase 2
- Area: Backend
- Owner: Unassigned
- Reviewer: Unassigned
- Status: Done
- Priority: Medium
- Risk: Low
- Source: Proposal §"Phase 2" task P2-006.
- Dependencies: P2-002 (Done).
- Owner: Backend Agent (
general-purpose, agentIda061e4b9537006e98) - Reviewer: Orchestrator (independent re-run)
- Write scope:
semantic_suggestions.py(one computed field onSuggestionResponse) + new test file. Coordinated with parallel P2-005 (no overlap). ✅ - Acceptance criteria:
- Every suggestion has a derived
confidence_band. ✅ (Always populated; computed at serialization.) - Banding thresholds documented (in
Field(..., description=...)+ module docstring). ✅ - At least one test per band. ✅ (20 tests covering all bands + boundaries + NULL.)
- Test evidence:
- Banding scheme:
>=0.9high;[0.7, 0.9)medium;[0.5, 0.7)low;<0.5needs_review;None→ needs_review. - Implementation: Pydantic v2
@computed_fieldover a pure_derive_confidence_bandhelper. Never stored; computed at serialization.Literal["high","medium","low","needs_review"]type. - Files modified (1):
semantic_suggestions.py— module-level threshold constants,ConfidenceBandLiteral alias,_derive_confidence_bandhelper,@computed_fieldproperty. NO other field touched. - Files added (1):
tests/unit/test_suggestion_confidence_bands.py(20 tests: 13 parametric boundary cases + 7 response-level tests including OpenAPI serialization). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff/mypy clean; 20 new tests pass; full unit — 2532 passed.
- Review notes:
- Approved. Thresholds in module constants (not runtime-configurable) — documented as future enhancement.
- Residual risks:
- Thresholds not runtime-configurable (would need settings module / DB-backed config).
confidence_bandis computed at serialization only — never stored. DB-layer filtering by band would require derivation in SQL or persistence.type: ignore[prop-decorator]needed for known mypy/Pydantic v2 interaction; this is the upstream-recommended pattern.
P2-007: UI import wizard in unified editor¶
- Phase: Phase 2
- Area: Frontend
- Owner: Frontend Agent (
general-purpose, agentIdac9b9c7ca51163897) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: High → landed cleanly (zero lint/type/test regressions).
- Source: Proposal §"Phase 2" task P2-007.
- Dependencies: P2-001 (Done), UX0-002 (Done).
- Write scope: New wizard + new additive
Stepper/WizardHostprimitives + 4 new API methods + tests. Composes P2-008'sIngestRunSummaryon success. Disjoint from P2-008. ✅ - Acceptance criteria:
- Wizard renders at
/editor/import-wizardinside the unified editor. ✅ - User completes the flow and sees the P2-008 summary on success. ✅
- Failures surface (PATCH ingest-run status to
failed+ render error). ✅ - Tests + manual browser-check protocol documented. ✅
- Test evidence:
- Files added (15):
components/ui/stepper.tsx(147) +components/ui/wizard-host.tsx(148) — net-new additive primitives;lib/api/assetLineage.ts(122) — 4 typed methods (recordIngestRun,getIngestRun,updateIngestRunStatus,recordAsset);components/semantic/ImportWizard/with 8 files (1126 LOC:ImportWizard.tsxorchestrator + 6 step components +types.ts+ barrel);app/editor/import-wizard/page.tsx(20); 2 test files (5 + 4 tests). - Files modified (1):
lib/api/index.ts— addedassetLineageApito the combinedapiobject. - Mount point:
/editor/import-wizard(Next.js App Router page). - State management: local
useReducer(wizard state) +useState(step index, submitted flag). No Zustand slice — wizard state is transient. Justified per Frontend Standards. - Steps: Source type → Target scheme (workspace dropdown via
getSchemes, free-text fallback) → Policy (3 governance presets flowing throughconfig) → Extraction options (concepts-only, extract-relations, run-inference) → Review (read-only recap) → Submit (4-call API chain + renders P2-008 summary). - Composition with P2-008:
SubmitStepimports<IngestRunSummary runId={result.runId} />from@/components/semantic/IngestRunSummary; compact local count panel shown while the summary fetch settles. - Quality gates (orchestrator independent re-run on 2026-05-19):
npm run lint— 0 errors (26 pre-existing warnings);npm run type-check— clean;npm run test— 468/468 pass across 47 files;npm run build— success; route registered. - A11y: keyboard-only nav;
aria-current="step";role="region"on body with step label;aria-live="polite"on submitting status; native semantic controls throughout. - Layout: Flexbox + Grid; zero
position: absolute. Responsive via Tailwindsm:breakpoints. - Review notes:
- Approved via orchestrator independent re-run. Stepper + WizardHost are clean additive primitives suitable for future wizard reuse.
- Residual risks:
- Backend route consolidation gap — wizard chains 4 endpoints. Future card may add a single
POST /imports/start. - Governance policy presets flow to
configJSON; backend has no policy enum yet. - Upload-arbitrary path uses paste-text; multipart upload variant is a follow-up.
- No nav entry today — wizard reachable only via URL. Follow-up to surface in
EditorDashboardquick actions.
P2-008: Ingest summary screen with links to suggestion inbox¶
P2-008: Ingest summary screen with links to suggestion inbox¶
- Phase: Phase 2
- Area: Frontend
- Owner: Frontend Agent (
general-purpose, agentIdab18fc400b1ea5b31) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium → landed cleanly.
- Source: Proposal §"Phase 2" task P2-008.
- Dependencies: P2-001 (Done).
- Write scope: New
IngestRunSummaryreusable component + standalone route + ONE API method + tests. Disjoint from P2-007. ✅ - Acceptance criteria:
- Renders every field from
ImportRunSummary. ✅ - "View suggestions" link with correct URL pattern. ✅
- Loading / error / empty / populated states. ✅
- Test evidence:
- Files added (5):
components/semantic/IngestRunSummary/IngestRunSummary.tsx(~410),IngestRunSummary/index.ts(10),app/editor/ingest-runs/[runId]/page.tsx(20),app/editor/ingest-runs/[runId]/IngestRunDetailPageClient.tsx(~33),__tests__/components/semantic/IngestRunSummary.test.tsx(~210, 14 tests). - Files modified (2):
lib/api/ingest.ts(addedSourceReference,ImportRunCounts,ImportRunSummarytypes +getImportRunSummarymethod);components/semantic/index.ts(4-line export block). - Standalone route:
/editor/ingest-runs/{runId}(Next.js App Router withgenerateStaticParams). - Component props:
runId?: string(fetch mode) +summary?: ImportRunSummary(pre-fetched mode for wizard) +onRerunExtraction?+className?. Plus exportedbuildSuggestionsLink(ingestRunId): stringhelper. - "View suggestions" link:
/ontology/suggestions?ingest_run_id=<urlencoded(run_id)>. - Composition with P2-007: wizard's
SubmitSteppassesrunId(fetch mode); standalone route uses the same. The component has zero dependency on the wizard. - Quality gates (orchestrator independent re-run on 2026-05-19):
npm run lint— 0 errors;npm run type-check— clean;npm run test— 468/468 pass;npm run build—/editor/ingest-runs/[runId]listed in route table. - A11y:
role="status"+aria-live="polite"on loading;role="alert"on errors; semantic<dl>/<ul>for data;aria-labelon the "View suggestions" link; decorative icons markedaria-hidden. - Layout: Flexbox + Grid only; zero
position: absolute. Responsive viagrid-cols-2 sm:grid-cols-3 lg:grid-cols-5. - Review notes:
- Approved via orchestrator independent re-run. Component supports both fetch-mode and pre-fetched-mode for composition; tests pin both paths.
- Residual risks:
- In-flight polling not implemented — when
status === "running"andfinished_at === null, page is read-once. Follow-up could addsetIntervalor websocket once P2-007 lands streaming. - Suggestions inbox doesn't yet consume
ingest_run_idquery param — backend filter works (P1-005); frontend filter UI needs a follow-up card to wire the query param into the page's filter state. Deep link is correct; inbox just hasn't consumed it. extracted_candidatestile reads identical tosuggestions_emittedpost-P2-001-FU-A. Two adjacent tiles with the same number is intentional but may confuse users. Worth copy review.StatusBadgecolour map doesn't yet coverrunning/partial(vocabulary guide reserves amber/orange). Aligning is flagged in vocabulary doc; out of scope here.
P2-009: Integration tests for ingest → suggestions with source evidence¶
- Phase: Phase 2
- Area: QA / Backend
- Owner: QA Agent (
general-purpose, agentIdae1ebaec07d3805ee) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Low
- Source: Proposal §"Phase 2" task P2-009.
- Dependencies: P2-002 (Done), P2-002-FU-A (Done).
- Write scope:
tests/integration/test_ingest_to_suggestions_flow.py(new file, 392 lines). No source touched. ✅ - Acceptance criteria:
- End-to-end test for the import → suggestions path. ✅
- Asserts on evidence shape, not just counts. ✅
- Runs under the integration suite with
@pytest.mark.integration. ✅ - Test evidence:
- Approach: Adapter-driven. Builds 2 minimal
ExtractionResultpayloads inline (contract.yaml+schema.csv), runs through productionextraction_result_to_drafts, persists via realSuggestionManager.create_batch(..., require_evidence=True). - Fixture: revenue/billing — scheme
revenue-billing; actorbilling-importer@example.com; 2 assets (s3://example/revenue/billing/contracts.yaml,.../invoices.csv); conceptsbilling.Invoice,billing.Contract,billing.LineItem; one relationInvoice -bills_for-> Contract. - Assertions per persisted suggestion: status, scheme_id, ingest_run_id, source_asset_id (matched by source_uri), source_span (non-empty + correct locator format), rationale (contains source_uri), proposed_action ∈ {add_concept, add_relationship}, confidence_score float in [0,1], provenance merged. Plus shape: contract metadata override of
confidence_score=0.7lands on contract rows; invoice falls back to adapter default 0.6. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean;
pytest tests/integration/test_ingest_to_suggestions_flow.py -m integration -v— 1 passed in 1.08s; integration suite (mapping projection + suggestion traceability + ingest flow) — 3 passed. - Review notes:
- Approved. Adapter-driven approach exercises the same code path the materializer uses; doesn't require bootstrapping tree-sitter / docs-markdown extractors.
- Residual risks:
- On-disk SQLite covers schema + ORM round-trip but masks Postgres-specific behaviour (JSONB indexing, native UUID type, case-sensitive ordering). Same test should pass on Postgres CI without modification.
- Test doesn't assert on
confidence_band(P2-006) oris_duplicate_of(P2-005) — landed concurrently; extension is a thin follow-up if desired. - Uses
extraction_result_to_draftsdirectly; future canonical "run an extraction" entrypoint would be a one-line swap.
P4-001: Add semantic health issue list¶
- Phase: Phase 4
- Area: Backend/Frontend
- Owner: Full-stack Agent (
general-purpose, agentIdaa0afa36039d1b229) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: KB/Ontology Workflow Proposal, semantic health and governance gates
- Dependencies: P1-001, UX0-002
- Write scope: Health service, health API, existing editor semantic health panel
- Acceptance criteria:
- Health checks produce actionable issues with severity, affected entity, and suggested fix. ✅
- UI shows issue status, severity, owner when available, and direct navigation to the affected concept or mapping. ✅
- Tests cover duplicate concept, missing mapping, deprecated concept usage, and broken relationship cases. ✅
- Test evidence:
- Panel surface: new "Health" tab added to the existing
EditorCompaniontab strip (alongside Validation / Impact / Runs / Completeness) — reuses the bottom-companion shell rather than introducing a new chrome. UX0-002 row "5. Semantic Health" explicitly authorised P4-001 to add a thin wrapper alongside existing panels. - Health-check coverage (all 4 working, no stubs):
duplicate_concept— case-insensitive grouping ofconcepts.name(catchesInvoicevsinvoiceeven though SQLunique=Truepasses them). Severitywarning. Suggested fix: "Merge or rename".missing_mapping— joinsconcepts.concept_keyagainst activemapping_projections.concept_key; skips deprecated concepts and concepts withoutconcept_key. Severityinfo. Suggested fix: "Bind to a contract field via the mapping workbench".deprecated_concept_usage— joins activemapping_projectionstoconceptswheredeprecated=True. Severityerror. Suggested fix: "Remap to a non-deprecated alternative".broken_relationship— scansconcept_relationships.{source,target}_concept_idfor ids absent fromconcepts. Severityerror. Production schema'sondelete=CASCADEmakes this rare; tests explicitly disable PRAGMA foreign_keys to seed the case.
- Backend service architecture (Protocol + factory + SQLAlchemy + in-memory, mirroring P1-001/P1-007-IMPL):
src/pycharter/semantic/health/_types.py—Severity,HealthIssueKind,AffectedEntityKind,HealthIssue,HealthReport,SemanticHealthServiceProtocol._checks/{duplicate_concept,missing_mapping,deprecated_concept_usage,broken_relationship}.py— pure read-onlycheck_*(session) -> list[HealthIssue]._service.py—SqlAlchemySemanticHealthService(live-session or session-factory mode; per-check exceptions logged and swallowed so one failing check cannot kill the report) +InMemorySemanticHealthServicefor tests.factory.py—make_semantic_health_service(*, backend=None, session=None, session_factory=None, seed_issues=None). Defaultsqlalchemy. Selectable via env varPYCHARTER_SEMANTIC_HEALTH_BACKEND.
- HTTP endpoint:
GET /api/v1/semantic/healthundersrc/pycharter/api/routes/v1/semantic_health.py, registered viapycharter.api._routersunder the existing_semantictuple.- Query params:
?severity=error,warning,info,?kind=...,?affected_entity_kind=concept,...,?limit=N(1–500),?offset=N. - Response:
HealthReportResponse { issues: HealthIssueResponse[], total: int, summary: dict[str,int] }. Each issue:kind, severity, affected_entity_kind, affected_entity_ids, summary, details, suggested_fix, actionable_link, owner, detected_at. - Unknown filter tokens silently dropped. Exceptions surface through
raise_logged_internal_from_exception.
- Query params:
- Frontend panel (
src/pycharter/ui/src/components/editor/panels/SemanticHealthPanel.tsx):- States: loading skeleton (5 ghost rows); empty success (
CheckCircle2+ "No semantic health issues found"); filtered-empty; error banner (role="alert"+ Retry); list (severity icon + summary + kind badge + details + suggested fix + owner + Open link). - Filter controls: severity counter chips with
aria-pressed; kind dropdown witharia-label; manual refresh button; 30 s auto-refresh interval. - Navigation per kind:
duplicate_concept → /editor/concepts/<id>;missing_mapping → /editor/mapping-workbench?selectedConcept=<id>;deprecated_concept_usage → /editor/mapping-workbench?contract=<contract_id>&contractVersion=<version>&fieldPath=<field>;broken_relationship → /editor/relationships/<relationship_id>.
- States: loading skeleton (5 ghost rows); empty success (
- API client:
semanticHealthApi.list(filters?)insrc/pycharter/ui/src/lib/api/semanticHealth.ts. - A11y: severity counters with
aria-pressed; severity icons witharia-label(not colour-only); kind dropdown witharia-label; semantic<ul aria-label>; real<a href>links (keyboard + open-in-tab);role="alert"on error; full keyboard tab order with visible focus rings;RefreshCwwitharia-label. - Tests (49 new total):
- Backend 6 files / 41 tests:
test_semantic_health_checks_duplicate_concept.py(5),..._missing_mapping.py(6),..._deprecated_concept_usage.py(5),..._broken_relationship.py(5, PRAGMA foreign_keys=OFF fixture),test_semantic_health_service.py(13 — aggregation, sort, filter, pagination, factory),test_semantic_health_api.py(7). - Frontend 1 file / 8 tests:
SemanticHealthPanel.test.tsx— issue rendering, action-link routing, severity counter filter, kind dropdown filter, empty state, error + Retry, deprecated link params, severity icon ARIA labels.
- Backend 6 files / 41 tests:
- Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean (364 files);
pytest tests/unit/ -q --timeout=60— 2675 passed, 1 skipped, 8 deselected (delta +41 vs. 2634 baseline);npm run lint— 0 errors;npm run type-check— clean;npm run test— 555 passed (62 files) (delta +8);npm run build— clean prod build. - Review notes:
- Approved on orchestrator independent re-run. Panel mounts inside the existing EditorCompanion shell (reuses authorised by UX0-002). All 4 check kinds are real working SQLAlchemy queries — no stubs. Protocol+factory pattern matches the established repository idiom. Filter set + a11y + navigation mapping is complete.
- Residual risks:
concept_relationshipscascade: production tables useondelete=CASCADEsobroken_relationshipwill rarely fire in steady state. The check still protects against import paths that bypass the cascade (external dump ingest). Tests disable FK enforcement explicitly.server_default="false"onconcepts.deprecatedreads as Python truthy under SQLite; themissing_mappingcheck coerces via_is_truthy_deprecated. Real PostgreSQL stores actual booleans, so SQLite-test-only subtlety.- Auto-refresh choice: 30 s
setInterval+ manual Refresh (matches sibling companion tabs). SWR is not currently a dependency; later swap is localised. - Performance:
duplicate_conceptandmissing_mappingload all concepts into memory. At low-thousands registry size this is fine; a registry with hundreds of thousands of concepts would want indexed-aggregate SQL. Out of scope for MVP. - No persistence of issue history: issues evaluated on demand; trend/owners-over-time tracking is a future card.
P3-006-FU-A: Wire proposals.suggestion_id FK + link endpoint¶
- Phase: Phase 3 follow-up
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIdac4540da8fb6d699d) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: P3-006 residual risk (proposal wiring MVP-only).
- Dependencies: P3-006 (proposal gate landed).
- Write scope:
- Add
suggestion_idcolumn (nullable, indexed) to theproposalstable via Alembic migration. - Update
ProposalModelSQLAlchemy mapping. - New
POST /api/v1/semantic/proposalsendpoint to create a proposal row linked to a suggestion (and unlock the P3-006 gate for that suggestion). - Replace
_default_proposal_lookup_factoryin_acceptance.pyto use the FK join instead ofpayload["proposal_id"]. - Acceptance criteria:
- Domain/core suggestions can be unblocked by creating a proposal that links to them. ✅
- Reverse migration cleanly drops the FK and index. ✅
- Tests cover: lookup-by-FK finds the proposal; lookup misses when no proposal exists; cascade behaviour on suggestion deletion. ✅
- Test evidence:
- Migration:
src/pycharter/db/migrations/versions/20260519000003_add_proposals_suggestion_id.py. Revisionproposals_suggestion_id(down-revisionsemantic_review_events). Addssuggestion_id String(36) NULL+ indexix_proposals_suggestion_id. Soft-ref (no hard FK) — matches the program convention from P1-005'ssource_asset_id. Upgrade-downgrade-upgrade round-trip verified clean. - Model:
src/pycharter/db/models/semantic/proposal.py—suggestion_id = Column(String(36), nullable=True, index=True)added. - New endpoint:
POST /api/v1/semantic/proposals/link— body{suggestion_id, actor, summary, details?}; returnsLinkProposalResponse {id, suggestion_id, summary, status, actor, created_at}with 201 Created. Validatessuggestion_idis a UUID (400 otherwise), 404 if suggestion absent. NewLinkProposalRequest/LinkProposalResponsePydantic models insrc/pycharter/api/models/semantic_proposals.py. - Lookup swap:
_default_proposal_lookup_factoryin_acceptance.pynow queriesProposalModel.suggestion_id == str(suggestion.id). Legacypayload["proposal_id"]resolves via_legacy_payload_proposal_lookupand emitsDeprecationWarningwith one-cycle removal note. - Tests:
tests/unit/test_proposals_link_endpoint.py— 8 tests: link creates proposal + suggestion_id; missing suggestion → 404; malformed UUID → 400; default-title fallback; lookup finds linked proposal; lookup returns None when none exists; legacy fallback emits warning; proposal survives suggestion delete (soft-ref cascade). - Existing tests:
test_proposal_gate.py(injects fakes) unaffected;test_semantic_suggestions_api.pydomain-tier accept test still passes because the new lookup also returnsNonewhen no proposal links the suggestion. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean;
pytest tests/unit/ -q --timeout=60— 2693 passed, 1 skipped, 8 deselected (delta +18 vs. 2675 baseline reflects this card's 8 + parallel P3-009-FU-A's 10). - Review notes:
- Approved on orchestrator independent re-run. Soft-ref pattern matches the program convention; round-trip verified clean; legacy fallback preserves backward compat with a clear deprecation horizon.
- Residual risks:
- Soft-ref vs. FK: the new
suggestion_idis a string with no FK constraint — deleting the suggestion leaves a dangling pointer (covered by thesurvives_suggestion_deletetest). Matches the agreed program convention; an alternative would be to addondelete="SET NULL"if a hard FK is later preferred. - One-cycle deprecation horizon —
payload["proposal_id"]fallback emitsDeprecationWarningand is scheduled for removal next minor release. Future card must delete_legacy_payload_proposal_lookupand the call site. - Contract scoping — the link endpoint uses a sentinel
contract_id="suggestion-link"because the suggestion has no contract today. Follow-up may derive the contract from the scheme or accept it explicitly. - Audit trail on link — not implemented here. Reviewers may want a review-event row when a proposal is linked. P3-009-FU-A's endpoint can later be extended to include
decision="link_proposal"rows.
P3-008-FU-A: Promote technical_only and annotation to first-class FSM states¶
- Phase: Phase 3 follow-up
- Area: Backend + Frontend
- Owner: Backend Agent (
general-purpose, agentIdada950fb9cc9f615e) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: P3-008 residual risk (reason-prefix markers).
- Dependencies: P3-008 (action endpoints landed), P1-008 (lifecycle FSM).
- Write scope:
- Extend
BindingStateto includeTECHNICAL_ONLY. - Add a dedicated annotation-event row shape (separate from state transitions).
- Replace
[technical_only]/[annotation]reason-prefix markers insemantic_bindings.pywith the new FSM and event paths. - Update P3-009's history drawer to render the new event shape (drop reason-prefix parsing).
- Migration if a new column or table is required.
- Acceptance criteria:
mark-technical-onlyissues a real state transition (not a same-state event). ✅annotateissues a typed annotation event (not a reason-prefixed state event). ✅- Existing P3-009 reason-prefix code is removed cleanly; backward compat for already-persisted markers is documented (one-shot migration OK). ✅
- Test evidence:
- FSM extension: new
BindingState.TECHNICAL_ONLY = "technical_only"; newBindingEventAction = TRANSITION | ANNOTATION | REMAPdiscriminator. Transitions:UNMAPPED/SUGGESTED/MAPPED/APPROVED ↔ TECHNICAL_ONLY(fully reversible). Annotation events are event-only (no state change). Remap events carry bothfrom_concept_idandto_concept_idon a single row. - New API methods:
BindingLifecycle.annotate(...)emits anANNOTATIONevent without state change;BindingLifecycle.remap(...)(see P3-008-FU-B) emits a single atomic event withaction=REMAP. - Migration:
20260519000005_extend_binding_events_for_actions.py(revisionbinding_events_actions,down_revision="semantic_health_issue_snapshots"). Adds threeconcept_binding_eventscolumns:action VARCHAR(32) NOT NULL DEFAULT 'transition',from_concept_id VARCHAR(255) NULL,to_concept_id VARCHAR(255) NULL. Reversibility verified. - Backward-compat strategy: Option 1 (data migration). Upgrade rewrites any legacy
concept_binding_eventsrow whosereasonstarts with[technical_only](→to_state="technical_only", marker stripped) or[annotation](→action="annotation", marker stripped). Clean data forward — eliminates the marker-prefix shim entirely. - Route changes in
semantic_bindings_actions.py:mark-technical-onlyissues a realTECHNICAL_ONLYtransition;annotatecallslifecycle.annotate(...); marker constantsTECHNICAL_ONLY_REASON_MARKER/ANNOTATION_REASON_MARKERdeleted (verified zero external consumers). - Frontend:
BindingHistorySubscriber.tsxdrops reason-prefix parsing; delegates row header to newLifecycleRowHeader.tsx. Three event shapes rendered:transition(state pills),annotation(indigo badge + current state pill, no arrow),remap(violet badge +from → to).BindingHistoryDrawer.tsxaddstechnical_onlycolour entry. - Tests: 12 new unit tests (
TestTechnicalOnlyStatex3,TestAnnotationActionx4,TestAtomicRemapx5) intest_semantic_lifecycle.py; route tests intest_semantic_bindings_route.pyrewritten for the new event shapes + failed-remap atomicity test; 5 new integration tests intests/integration/test_binding_lifecycle_actions.pycovering accept persistence + atomic remap row shape + failed-remap atomicity + technical-only state transition + annotation event shape. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean;
pytest tests/unit/ -q --timeout=60— 2718 passed, 1 skipped, 8 deselected (delta +25 vs. 2693 baseline reflects this card + parallel P4-001-FU-A); migration upgrade-downgrade-upgrade round-trip clean. - Review notes:
- Approved on orchestrator independent re-run. Data migration removes the marker-prefix shim entirely (clean forward). FSM extension preserves reversibility of
TECHNICAL_ONLY. P3-008-FU-B atomic remap lands inside the same FSM extension (see that card for details). - Residual risks:
- Atomic remap side-effect on mapping projection. The new REMAP event does not emit an implicit "from" retirement event. The mapping-projection updater routes by
to_state, so its reject/remap dispatch keeps working; the projection for the prior concept will not flip tounresolvedfrom the remap event alone. If that side-effect is needed for the UI's reverse-index, a follow-up should wireon_transitionto treat REMAP events as an implicit retirement of the from-key. - Marker-export removal:
TECHNICAL_ONLY_REASON_MARKER/ANNOTATION_REASON_MARKERdeleted. Verified zero external grep hits today; any future consumer that imports them getsImportError. - Postgres-only CI: the data-migration
LIKErewrite was exercised on SQLite locally. PostgresLIKEsemantics are equivalent for the literal prefix patterns used.
P3-008-FU-B: Atomic remap action (single FSM transition)¶
- Phase: Phase 3 follow-up
- Area: Backend + Frontend
- Owner: Backend Agent (
general-purpose, agentIdada950fb9cc9f615e— landed alongside P3-008-FU-A) - Reviewer: Orchestrator (independent re-run)
- Status: Done
P3-008-FU-B: Atomic remap action (single FSM transition) — landed with P3-008-FU-A¶
- Phase: Phase 3 follow-up
- Area: Backend + Frontend
- Owner: Backend Agent (
general-purpose, agentIdada950fb9cc9f615e— bundled with P3-008-FU-A) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Low
- Risk: Medium
- Source: P3-008 residual risk (remap non-atomicity).
- Dependencies: P3-008 (route landed).
- Write scope:
- Add a first-class
remap(from_concept, to_concept)action toBindingLifecycle.transition(or a sibling method). - One atomic emission rather than today's detach-then-attach (UNMAPPED → MAPPED) two-event pattern.
- Update the HTTP
POST /api/v1/semantic/bindings/remaproute to use the new action. - Acceptance criteria:
- Failure mid-action does not leave the field UNMAPPED. ✅
- History drawer renders a single "remapped" event row instead of two. ✅
- Test evidence:
- New API:
BindingLifecycle.remap(*, contract_id, field_path, from_concept_id, to_concept_id, actor, comment="", contract_version=None) -> BindingEvent. Emits exactly oneBindingEventwithaction=REMAP,to_state=MAPPED,from_concept_idandto_concept_idpopulated. - Route change:
POST /bindings/remapnow returns a singleBindingEventResponseinstead of{detach_event, attach_event}(breaking response-shape change for any external client; UI updated in the same commit). - Frontend:
RemapBindingResponseremoved fromlib/api/semanticBindings.ts;remapclient method returnsBindingEventResponse. History drawer'sremaprow renders a single violet "remapped" badge withfrom_concept_id → to_concept_iddetail. - Tests:
TestAtomicRemap(5 tests intest_semantic_lifecycle.py) — single event row, correct from/to concepts, contract_version threading, atomicity under failure injection. Integration testtest_atomic_remap_emits_single_eventintests/integration/test_binding_lifecycle_actions.py. - Quality gates: rolled up with P3-008-FU-A's gates above.
- Review notes:
- Approved on orchestrator independent re-run as part of the P3-008-FU-A bundle. Failure-mid-action invariant tested explicitly; no half-committed UNMAPPED state achievable.
- Residual risks:
- Response-shape breaking change for
POST /bindings/remap(returnsBindingEventResponseinstead of{detach_event, attach_event}). UI updated; external clients (if any) need to migrate. - Projection retirement of "from" key: see P3-008-FU-A residual risk for the same coordination note.
P3-008-FU-C: Split semantic_bindings.py under 400-line cap¶
- Phase: Phase 3 follow-up
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda62f11679f8235a59) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Low
- Risk: Low
- Source: P3-008 residual risk (file at 480 lines).
- Dependencies: P3-008 (action endpoints).
- Write scope:
- Extract the 4 action endpoints + their request models into
semantic_bindings_actions.py. semantic_bindings.pykeeps the listing/read endpoints.- Acceptance criteria:
- Both files under 400 lines. ✅
- No behaviour change; route registration shape preserved. ✅
- Existing tests pass without modification. ✅
- Test evidence:
- Final line counts:
semantic_bindings.py— 191 lines (was 480);semantic_bindings_actions.py— 332 lines (new). Both under 400. - Move set: constants (
TECHNICAL_ONLY_REASON_MARKER,ANNOTATION_REASON_MARKER); request models (AcceptBindingRequest,RemapBindingRequest,MarkTechnicalOnlyRequest,AnnotateBindingRequest); response model (RemapBindingResponse); private helpers (_empty_actor_error,_bad_transition_from_exception); handlers (accept_binding,remap_binding,mark_technical_only,annotate_binding). - Kept in
semantic_bindings.py:BindingEventResponse,BindingStateResponse,TransitionRequest,list_events,get_state,create_transition. - Router composition: separate router (preferred).
semantic_bindings_actions.pydeclares its ownAPIRouter(tags=["Semantic - Bindings"]). Both routers registered under the same/api/v1/semantic/bindingsprefix via a second entry in the_semanticlist in_routers.py. URL surface and OpenAPI tags unchanged. - Re-exports added: None. Grep confirmed the only external import is
router; the moved request/response models and constants have no consumers outside the route file. - Test impact: Test bodies unchanged. The fixture setup gained two non-behavioural lines (one import + one
app.include_router(bindings_actions_router, prefix=...)) so the action endpoints remain reachable in the isolated TestClient. - Tests:
tests/unit/test_semantic_bindings_route.py— 16 passed, identical to baseline. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean (140 files);
pytest tests/unit/ -q --timeout=60— 2693 passed, 1 skipped, 8 deselected (no test count change attributable to this card — pure refactor). - Review notes:
- Approved on orchestrator independent re-run. Pure refactor; both files comfortably under the 400-line soft cap; OpenAPI surface preserved; zero behavioural change.
- Residual risks:
- None significant. If a future consumer imports a moved model directly from
semantic_bindings, they will getImportError— but the pre-split grep confirmed there are zero such consumers in the repo today.
P3-009-FU-A: GET /suggestions/{id}/review-events backend route¶
- Phase: Phase 3 follow-up
- Area: Backend + Frontend
- Owner: Full-stack Agent (
general-purpose, agentIda2b30e98898afcd16) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Low
- Source: P3-009 residual risk (review-events endpoint gap;
getReviewEventsis a stub returning[]). - Dependencies: P3-004 (review-event repo), P3-009 (stub wrapper in place).
- Write scope:
- New
GET /api/v1/semantic/suggestions/{id}/review-eventsroute returning the suggestion'sSemanticReviewEventModelrows in reverse-chronological order. - Switch the frontend
getReviewEvents(suggestionId)stub to call the real route. - Update the history drawer to merge lifecycle events + review events on a single timeline.
- Acceptance criteria:
- Route returns events for a known suggestion id; 404 for unknown. ✅
- History drawer in the mapping workbench renders merged stream sorted by timestamp. ✅
- Tests cover happy + 404 paths + merged-timeline ordering. ✅
- Test evidence:
- New endpoint:
GET /api/v1/semantic/suggestions/{suggestion_id}/review-events. Query params:limit(default 100,ge=1,le=500),offset(default 0,ge=0). Response:list[ReviewEventResponse]with{id, suggestion_id, actor, decision (Literal: accepted|dismissed|request_changes), comment, timestamp}, newest-first. Errors: 404 "Suggestion not found"; 422 for non-UUID id and forlimit/offsetout of bounds. Reads viamake_semantic_review_event_repository(backend="sqlalchemy", session=session)so behaviour is deterministic regardless of the manager's env-var-driven backend. - Pydantic model:
ReviewEventResponseadded tosrc/pycharter/api/models/semantic_suggestions.py. - Frontend stub swap:
src/pycharter/ui/src/lib/api/bindingEvents.ts— replaced empty-array stub withget<SemanticReviewEvent[]>call.SemanticReviewEventinterface +SemanticReviewDecisionunion aligned with backend Literal. Supports optional{limit, offset}. - History drawer merge:
src/pycharter/ui/src/components/editor/MappingWorkbench/BindingHistorySubscriber.tsx— fetches lifecycle + review streams in parallel; merges via new exportedmergeHistoryStreamshelper. Sort: timestamp descending, lifecycle wins ties (stable sort). New optional URL paramhistoryForSuggestionactivates the review fetch; absent → identical pre-existing lifecycle-only behaviour. - Tests:
- Backend: new
tests/unit/test_review_events_endpoint.py— 10 tests across 4 classes:TestReviewEventsHappyPath(request_changes + dismissed round-trip),TestReviewEventsErrorPaths(404 unknown, 422 non-UUID),TestReviewEventsOrderingAndPagination(newest-first, limit, offset, limit>500 → 422, negative offset → 422),TestReviewEventsEmpty(200 + []). - Frontend: 4 new
bindingEvents.test.tscases (URL shape, query-string composition, encoding) + 1 new merged-timeline component test (3 entries interleaved across timestamps assert newest-first bydata-testid) + 2 newmergeHistoryStreamspure-function unit tests.
- Backend: new
- Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean (140 files);
pytest tests/unit/ -q --timeout=60— 2693 passed, 1 skipped, 8 deselected (delta +18 vs. 2675 baseline; +10 this card + 8 P3-006-FU-A);npm run lint— 0 errors;npm run type-check— clean;npm run test— 560 passed (62 files) (delta +5);npm run build— clean. - Review notes:
- Approved on orchestrator independent re-run. Merge helper is pure + tested separately. Pagination is offset+limit because audit log is append-only and per-suggestion cardinality is small (typically <10 rows); cursor migration is cheap when needed.
- Residual risks:
decisionLiteral drift:ReviewEventResponse.decisionis kept in deliberate lock-step withSuggestionResponse.status's non-pendingvalues. New verb (e.g."escalate") requires updating both Pydantic models AND the frontendSemanticReviewDecisionunion. The_DECISION_*constants inpycharter.persistence.collaboration.suggestionsare the single source-of-truth.- In-memory backend mismatch:
SuggestionManagerdefaults to the in-memory review-event backend; the new route reads via SQLAlchemy. Production deployments that have not opted intoPYCHARTER_SEMANTIC_REVIEW_EVENT_BACKEND=sqlalchemywill see decisions persisted only in-process — GET returns[]. Pre-dates this card (flagged in__init__.pyas "route wiring is a follow-up"). Separate residual-risk follow-up. - Cursor vs. offset pagination: offset+limit chosen for simplicity; rows have stable
id+timestampso cursor migration is straightforward when needed.
P3-010-FU-A: Phase-3 integration tail (P3-006 / P3-008 / P3-009 coverage)¶
- Phase: Phase 3 follow-up
- Area: QA / Backend
- Owner: Test Engineer Agent (
general-purpose, agentIda4f056be61c46597b) — proposal-gate + review-events portion; FSM action coverage delivered by P3-008-FU-A'stest_binding_lifecycle_actions.py. - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Low
- Source: P3-010 residual risk (no P3-006/008/009 coverage; P2-009/P3-010 pattern proven).
- Dependencies: P3-006, P3-008, P3-009 (all landed).
- Write scope:
tests/integration/test_proposal_gate_routing.py— domain-tier suggestion accept attempt without proposal → 409 + audit event; create proposal → linked suggestion now accepts.tests/integration/test_review_events_endpoint_flow.py— GET review-events round-trip + pagination + 404.- FSM action coverage delivered by the P3-008-FU-A agent in
test_binding_lifecycle_actions.py(sibling integration file). - Acceptance criteria:
- At least one test per follow-up surface (P3-006 routing, P3-008 each-action, P3-009 review-events). ✅
- Assertions on event/projection state, not just counts. ✅
@pytest.mark.integrationmarkers. ✅- Test evidence:
- Files added:
tests/integration/test_proposal_gate_routing.py(553 lines, 4 tests).tests/integration/test_review_events_endpoint_flow.py(436 lines, 3 tests).- (FSM coverage in
tests/integration/test_binding_lifecycle_actions.py— owned by P3-008-FU-A.)
- Test list:
test_domain_tier_accept_without_proposal_returns_409— 409 with structured{"message", "code": "ProposalRequiredError"}body; suggestion stays pending; onerequest_changesaudit row withPROPOSAL_GATE_AUDIT_COMMENT.test_link_proposal_unblocks_domain_accept—POST /proposals/linkround-trip;ProposalModel.suggestion_idverified in DB; second accept succeeds (200), two-row audit timeline.test_core_tier_accept_blocked_until_proposal_linked— same flow fortier="core".test_legacy_payload_proposal_id_fallback_succeeds_with_warning— legacy path resolves end-to-end;DeprecationWarningcaptured viapytest.warns.test_review_events_returned_after_accept_and_dismiss— two suggestions; each GET returns one row with correct decision/actor/comment/timestamp.test_review_events_paginated_newest_first— single suggestion through request-changes → accept;?limit=1returns accept (newest),?limit=1&offset=1returns request-changes (older).test_review_events_404_for_unknown_suggestion— random UUID → 404.
- Fixture pattern: reused P3-010's pattern (real
SuggestionManager, real SQLAlchemy review-event repo via patched factory). Switched engine to shared-connectionStaticPoolin-memory SQLite (matches unit suites) to preventdatabase is lockedwhen route session + audit write race. Addedsession.commit()on dependency-override yield to mirror production session lifecycle. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean;
pytest tests/integration/test_proposal_gate_routing.py tests/integration/test_review_events_endpoint_flow.py -m integration -v— 7 passed;pytest tests/integration/ -m integration -v— 26 passed, 2 skipped, 3 failed (3 failures are pre-existingtest_api_pipeline_runs_route.pyPostgres-schema-dependent tests). Unit regression vs. 2693 baseline — unchanged (this card touchestests/integration/only). - Review notes:
- Approved on orchestrator independent re-run. Reuses the validated P3-010 fixture pattern; pairs with P3-008-FU-A's FSM integration coverage. The
StaticPoolswitch is the right call — eliminates a known race in concurrent-session tests. - Residual risks:
- SQLite vs Postgres: integration suite runs against in-memory SQLite (
StaticPool); theproposals.suggestion_idindex is dialect-agnostic and row shapes are identical to Postgres, so the gate lookup SQL is the same path production runs. - No merged-timeline integration coverage: the frontend
mergeHistoryStreamshelper is pure-function unit-tested; backend coverage would require driving the merge from the API surface, which is unnecessary given the helper's purity.
P4-001-FU-A: Persist semantic health issue history (trend + owner)¶
- Phase: Phase 4 follow-up
- Area: Backend + Frontend
- Owner: Full-stack Agent (
general-purpose, agentIda7b2d17e7b06923fc) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Low
- Risk: Medium
- Source: P4-001 residual risk (no persistence of issue history).
- Dependencies: P4-001 (health service + endpoint), P1-001 (KB repository pattern).
- Write scope:
- New
semantic_health_issue_snapshotstable — one row per (issue_fingerprint, evaluation_id, severity, detected_at). Fingerprint stable across evaluations (e.g. SHA of kind + sorted affected_entity_ids). - Persistence backend that records each
evaluate_healthrun. - History endpoint
GET /api/v1/semantic/health/issues/{fingerprint}/historyreturning observations over time. - UI: per-issue history sparkline + owner field (if available).
- Acceptance criteria:
- History endpoint returns observations sorted by
detected_at. ✅ - Resolved issues stop appearing in the current report but remain queryable. ✅
- UI sparkline renders without performance issues for ≤90 days of data. ✅
- Test evidence:
- Migration:
20260519000004_add_semantic_health_issue_snapshots.py(revisionsemantic_health_issue_snapshots, down-revisionproposals_suggestion_id). Createspycharter.semantic_health_issue_snapshotswith(id UUID PK, evaluation_id UUID, issue_fingerprint String(64), kind, severity, affected_entity_kind, affected_entity_ids JSON, summary, suggested_fix, actionable_link, owner, detected_at)and 3 indexes:(fingerprint, detected_at),(evaluation_id),(detected_at). Reversibility verified (upgrade → downgrade → upgrade). - Fingerprint formula:
sha256_hex(f"{kind}:{','.join(sorted(affected_entity_ids))}".encode("utf-8")). Stable across evaluations and acrossaffected_entity_idsorderings (verified bytest_fingerprint_is_order_invariant). - New endpoint:
GET /api/v1/semantic/health/issues/{fingerprint}/historywith path validator^[0-9a-fA-F]{64}$. Query params?since=<ISO>&until=<ISO>&limit=N(default 100, max 500). ResponseIssueHistoryResponse { fingerprint, kind, observations: [{evaluation_id, severity, detected_at, summary, owner}] }. 404 when no snapshots for the fingerprint. Observations sorted oldest-first. - Persistence trigger: every
SqlAlchemySemanticHealthService.evaluate_health()call writes one row per current-issue with a freshevaluation_idUUID. Snapshot insert wrapped inlogger.exceptionswallow so a write failure does not affect the live report. Opt-out flagpersist_history=False. - Resolved-issues semantics: the live
GET /api/v1/semantic/healthkeeps returning the in-memory current report; absence from the latestevaluation_id= resolved. Historical observations remain queryable indefinitely.test_resolved_issue_still_queryable_by_fingerprintcovers this. - Frontend:
Sparkline.tsx— inline SVG sparkline (no chart-lib dep). Single<path>driven by severity→numeric rank. ≤90 points = ≤89 line commands. Empty-state renders a dashed baseline.SemanticHealthPanel.tsx— addsuseIssueHistoryhook (lazy per-row fetch, 30-day window, limit 90); renders sparkline inline with kind badge; owner display pinned by test.semanticHealthApi.history(fingerprint, params)API method.
- Tests:
- Backend (12 new):
TestFingerprint(4),TestPersistIssueSnapshots(2),TestServiceWritesHistory(2 — distinct evaluation_ids, resolved-issue still queryable),TestHistoryEndpoint(4 — happy path sorted, 404, limit, 422 invalid fingerprint). - Frontend (6 new):
Sparkline.test.tsx(4 — empty-state, path command count, severity colour, ariaLabel) + 2 newSemanticHealthPanelcases (lazy history load, owner rendering).
- Backend (12 new):
- Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean;
pytest tests/unit/ -q --timeout=60— 2718 passed, 1 skipped, 8 deselected (delta +25 vs. 2693 baseline reflects this card + parallel P3-008-FU-A/B);npm run lint— 0 errors;npm run type-check— clean;npm run test— 567 passed (63 files);npm run build— clean prod build. - Review notes:
- Approved on orchestrator independent re-run. Fingerprint is order-invariant (load-bearing for grouping); snapshot insert is defensive (try+log+swallow); the sparkline is dependency-free SVG. Migration chains cleanly off
proposals_suggestion_id(P3-006-FU-A) and is itself the down-revision for P3-008-FU-A's FSM migration. - Residual risks:
- Storage growth: one row per current-issue per
evaluate_health()call. If the panel auto-refresh stays at 30s and 100 issues persist, that is 288k rows/day. Consider a TTL/compaction follow-up before this lands on a long-running prod environment. - Per-row fetch fan-out: the panel issues one
history(fingerprint, ...)call per visible row plus one SHA-256 digest. With ≤50 visible rows it is fine; for very large reports a debounced batched-history endpoint would be a future follow-up. - Fingerprint stability under id-set change: the fingerprint is a function of
(kind, sorted(affected_entity_ids)). If a check's id set evolves (e.g. duplicate-concept check adds a third concept to an existing pair), the fingerprint changes and the history splits — by design, but worth flagging.
P4-001-FU-B: Guardrail health snapshot retention before TTL strategy lands¶
- Phase: Phase 4 follow-up
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIdab31a117b10e67a0c) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: P4-001-FU-A residual risk (storage growth — 288k rows/day on a 100-issue + 30s-refresh deployment).
- Dependencies: P4-001-FU-A (snapshots table landed).
- Write scope:
- Switch
SqlAlchemySemanticHealthService.__init__default forpersist_historyfromTruetoFalse. - Update
make_semantic_health_servicefactory to plumb the flag. - Update tests + docs so opt-in is explicit.
- Acceptance criteria:
- Default deployment writes zero snapshot rows. ✅
- Tests that exercise snapshot persistence opt in via
persist_history=True. ✅ - Docstring + module docstring explain the trade-off and reference P4-001-FU-D for the full TTL strategy. ✅
- Test evidence:
SqlAlchemySemanticHealthService.__init__(persist_history=False)(wasTrue). Module docstring documents the trade-off and points at P4-001-FU-D.make_semantic_health_service(*, persist_history: bool = False, ...)plumbs the flag.- Two existing snapshot tests (
test_each_call_produces_distinct_evaluation_id,test_resolved_issue_still_queryable_by_fingerprint) opt in withpersist_history=True. - New
test_default_persist_history_is_offconfirms the default service writes zero snapshot rows across multiple evaluations. - Before/after for a 100-issue + 30s-refresh deployment: 288k rows/day → 0 rows/day until the operator opts in.
- Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean;
pytest tests/unit/ -q --timeout=60— 2737 passed, 1 skipped, 8 deselected (delta +19 from this batch). - Review notes:
- Approved on orchestrator independent re-run. Default-off is the correct guardrail until P4-001-FU-D ships TTL/compaction.
- Residual risks:
- Tests opt in explicitly. Any future test that adds snapshot persistence must remember to set
persist_history=True.
P4-001-FU-C: Batched semantic health history endpoint¶
- Phase: Phase 4 follow-up
- Area: Backend + Frontend
- Owner: Full-stack Agent (
general-purpose, agentIdac3e6cb1879efa29f) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Low
- Risk: Low
- Source: P4-001-FU-A residual risk (per-row fetch fan-out — one history call per visible row).
- Dependencies: P4-001-FU-A (history endpoint).
- Write scope:
- New
POST /api/v1/semantic/health/issues/historyacceptingfingerprints: list[str]body returning a map{fingerprint: observations}. - Frontend swap from per-row fetch to a single batched call when the panel mounts.
- Acceptance criteria:
- Single round-trip per panel mount instead of N round-trips. ✅
- Response shape preserves per-fingerprint observation lists sorted oldest-first. ✅
- Backward compatible — per-fingerprint endpoint stays. ✅
- Test evidence:
- New endpoint:
POST /api/v1/semantic/health/issues/history. BodyIssueHistoryBatchRequest { fingerprints: list[str] (1..200), since?, until?, limit? (1..500, default 100) }. Per-fingerprint validation against^[0-9a-fA-F]{64}$via Pydanticfield_validator→ 422 on malformed input. ResponseIssueHistoryBatchResponse { histories: {<fingerprint>: IssueHistoryResponse}, unknown_fingerprints: list[str] }. - SQL strategy: single
SELECT ... WHERE issue_fingerprint IN (:fingerprints) ORDER BY detected_at ASCper request (new helperfetch_issue_histories_batch()in_history.py). Replaces N round-trips with 1. - Frontend hook: new
useIssuesHistoryBatch.tsextracted to keepSemanticHealthPanel.tsxunder the soft cap. Hook computesvisibleIssues, derives a primitiveissuesSignaturefromkind + sorted(ids), and runs ONEPromise.all(computeIssueFingerprint(...))+ ONEsemanticHealthApi.historyBatch(...)per signature/revalidate change. ExposeshistoryByIssueKey(Map keyed byissueKey()) andloadState: 'idle' | 'loading' | 'ready'. Refresh bumpshistoryRevalidateKeyto re-fire even when visible-issue set is unchanged. - Per-row UX preserved:
IssueRowshows a sparkline-sizedanimate-pulseplaceholder whileloadState !== 'ready' && observations.length === 0; once batch resolves, the real<Sparkline>(or its dashed-line empty state for unknown fingerprints) renders. - Tests (9 new backend + 3 new frontend):
TestHistoryBatchEndpointcovers mixed-known-and-unknown, all-unknown, all-known, observations_oldest_first, limit_per_fingerprint, since_filter, malformed_fingerprint (422), empty_fingerprints (422), over_max_fingerprints (422). Frontend assertshistoryBatchMock.mock.calls.length === 1(not N), empty sparkline fallback for unknown fingerprints, and re-fetch on Refresh. - Backward compat: per-fingerprint
GET /api/v1/semantic/health/issues/{fingerprint}/historyendpoint untouched. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean;
pytest tests/unit/ -q --timeout=60— 2746 passed, 1 skipped, 8 deselected (delta +9 vs. 2737 baseline);npm run lint— 0 errors;npm run type-check— clean;npm run test— 569 passed;npm run build— clean. - Review notes:
- Approved on orchestrator independent re-run. Single
IN (...)query + Python-side partitioning is the correct shape; backward-compat preserved; hook signature memoisation avoids re-fires on unrelated parent renders. - Residual risks:
- Payload size at 200 fingerprints × 100 observations ≈ ~5 MB worst case. Today the panel caps observations to 90 and the issue list is small (~50 rows), so real-world payloads stay well under 1 MB. Future follow-up if scale grows: gzip response or add a row-cap.
- Ordering on duplicate fingerprints: handler dedupes with a
seenset in request order; duplicates collapse silently. Caller can rely onlen(histories) + len(unknown_fingerprints) ≤ len(fingerprints). - File size:
semantic_health.pyis now 442 lines (over the 400 soft cap, cohesive). Splitting is a separate refactor.
P4-001-FU-D: TTL/compaction strategy for semantic health snapshots¶
- Phase: Phase 4 follow-up
- Area: Backend
- Owner: Unassigned
- Reviewer: Unassigned
- Status: Backlog
- Priority: Low
- Risk: Medium
- Source: P4-001-FU-A residual risk (storage growth; P4-001-FU-B guardrails by defaulting off — TTL is the long-term plan).
- Dependencies: P4-001-FU-A, P4-001-FU-B.
- Write scope:
- Define retention policy in product/PM discussion (suggested: 90 days for full snapshots, then daily roll-ups).
- Implement a
cleanup_history(older_than: timedelta)maintenance fn + CLI entry underscripts/. - Implement an optional aggregation table (daily severity counts per fingerprint).
- Acceptance criteria:
- Snapshot rows older than the configured retention window are purged.
- Aggregation table queryable as a fallback for older windows.
- Idempotent (script can be re-run safely).
- Test evidence:
- Review notes:
- Residual risks:
P3-008-FU-D: Remap event should retire the prior concept binding in the mapping projection¶
- Phase: Phase 3 follow-up
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIdab31a117b10e67a0c) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: P3-008-FU-A/B residual risk (REMAP event does not signal retirement of the "from" key to the projection updater).
- Dependencies: P3-008-FU-A/B (FSM REMAP + atomic remap landed), P1-007-IMPL (mapping projection lifecycle hooks).
- Write scope:
- On REMAP events, fire BOTH the existing
on_concept_attached(to_concept)AND a newon_concept_detached(from_concept)hook on the projection updater. - Update P1-008's
on_transitiondispatcher to route REMAP events to both hooks. - Acceptance criteria:
- After a remap, the projection's row keyed by (contract, field, from_concept) is no longer "active". ✅
- Frontend mapping-workbench reverse-index queries reflect the remap. ✅
- Tests confirm both hooks fire on REMAP events. ✅
- Test evidence:
src/pycharter/semantic/mapping_projection/lifecycle_hooks.py—on_transitionnow detectsBindingEventAction.REMAPevents and fireson_concept_detached(from_concept_id, ...)before routing through the existingon_approve/on_reject/on_remapdispatch. Two new methods:on_concept_detached(flips matching rows tostate="unresolved") andon_concept_attached(delegates toon_remap).- Dispatch site:
lifecycle_hooks.py:114-122. - Unit tests:
tests/unit/test_mapping_projection_lifecycle.py— new classesTestRemapDispatchesDetachAndAttach(5 tests: both hooks fire on REMAP, attach-only when no from_concept, per-method behaviour) +TestRemapEndToEndProjection(1 end-to-end viaBindingLifecycle.remap). - Integration test: extended
tests/integration/test_binding_lifecycle_actions.pywithTestRemapDetachesFromConceptProjection— projection updater wired intoBindingLifecycle.on_transition; asserts from-concept reverse-index row reachesstate="unresolved"after a/remapHTTP call. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean; pytest 2737 passed (delta +19 from this batch).
- Review notes:
- Approved on orchestrator independent re-run. Detach precedes attach in dispatch order — guarantees the reverse-index is consistent after each REMAP without a transient "double-bound" window.
- Residual risks:
- No backfill helper for pre-existing REMAP rows. REMAP events that pre-date this card never fired
on_concept_detached, so projection rows from those events may still claim the from-concept is bound. If projection drift is observed in prod, a one-shot reconciliation script can scan REMAP events and re-fire the hook.
P3-009-FU-B: Default review-event persistence to SQLAlchemy backend¶
- Phase: Phase 3 follow-up
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIdab31a117b10e67a0c) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium
- Source: P3-009-FU-A residual risk (in-memory backend mismatch — deployments not opted into
PYCHARTER_SEMANTIC_REVIEW_EVENT_BACKEND=sqlalchemylose review events on restart AND get empty GET responses). - Dependencies: P3-004 (review-event repo), P3-009-FU-A (endpoint).
- Write scope:
- In
make_semantic_review_event_repository, default to"sqlalchemy"when a session is provided; fall back to"in_memory"only when explicitly requested or when no session is available. - Update
SuggestionManagerdefaults to align. - Tests pin the new default behaviour.
- Acceptance criteria:
- Production deployments emit and read review events through SQLAlchemy by default — no env-var opt-in needed. ✅
- In-memory backend remains available for tests (explicit opt-in). ✅
- Tests cover both backends. ✅
- Test evidence:
src/pycharter/semantic/review_events/factory.py—resolve_backendgained ahas_sessionparameter. New resolution order: explicit arg → env var → SQLAlchemy when session/factory provided → in-memory fallback.src/pycharter/semantic/review_events/__init__.py— module docstring documents the four-step resolution.src/pycharter/persistence/collaboration/suggestions.py—SuggestionManager.__init__now forwards itssessioninto the factory call, soSuggestionManager(session=db_session)automatically gets the SQLAlchemy review-event repo.- Before/after for production:
SuggestionManager(session=db_session)previously got an in-memory review-event repo unlessPYCHARTER_SEMANTIC_REVIEW_EVENT_BACKEND=sqlalchemywas set; review events were lost on restart andGET /api/v1/semantic/suggestions/{id}/review-eventsreturned[]. After this change, the same construction yieldsSqlAlchemySemanticReviewEventRepository(session)automatically. - New tests:
tests/unit/test_review_event_factory_defaults.py— 13 tests acrossTestSessionAwareDefault,TestExplicitBackendOverrides,TestEnvVarOverrides,TestSuggestionManagerDefault.SuggestionManager(session=None)still returns the in-memory repo (verified bytest_manager_without_session_uses_in_memory_review_repo). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean; pytest 2737 passed (delta +19 from this batch); 19/19 affected integration tests pass.
- Review notes:
- Approved on orchestrator independent re-run. The session-aware default is the cleanest non-breaking way to flip production behaviour to durable persistence while keeping the in-memory backend trivially available for tests.
- Residual risks:
- Proposal-gate audit still uses in-memory backend.
_emit_proposal_gate_audit(P3-006) discovered the route callssession.rollback()onProposalRequiredError, which would silently drop the audit row if the session were passed. The helper was deliberately reverted to a session-free factory call. Production deployments that want durable proposal-gate audit rows must either setPYCHARTER_SEMANTIC_REVIEW_EVENT_BACKEND=sqlalchemy(then the route needs to inject a session-factory mode repo) or accept the unchanged in-memory behaviour. Follow-up card if/when the proposal-gate audit needs durable storage.
P3-006-FU-B: Remove legacy payload["proposal_id"] fallback¶
- Phase: Phase 3 follow-up
- Area: Backend
- Owner: Unassigned
- Reviewer: Unassigned
- Status: Backlog
- Priority: Low
- Risk: Low
- Source: P3-006-FU-A residual risk (one-cycle deprecation horizon for
_legacy_payload_proposal_lookup). - Dependencies: P3-006-FU-A (FK lookup landed + DeprecationWarning live).
- Write scope:
- Delete
_legacy_payload_proposal_lookupfrom_acceptance.py. - Delete the legacy-path test in
tests/unit/test_proposals_link_endpoint.py(and the integration testtest_legacy_payload_proposal_id_fallback_succeeds_with_warning). - Confirm no production extractor sets
payload["proposal_id"]. - Acceptance criteria:
- All callers go through the FK lookup.
- Tests still pass.
- Documentation removed from the proposal-gate module.
- Test evidence:
- Review notes:
- Residual risks:
P5-001: Stable context_pack_id and content hash on context pack responses¶
- Phase: Phase 5
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda527e6ecc405add96) — bundled with P5-002. - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 5" task P5-001.
- Dependencies: P1-001 (KB management), P1-007-IMPL (mapping projection).
- Write scope:
- Add a stable
context_pack_id(UUID, deterministic by inputs) and a content hash to context pack responses. - Acceptance criteria:
- Repeated calls with the same inputs return the same
context_pack_id. ✅ - Content changes flip the hash. ✅
- Tests cover stability + change-detection + edge cases (empty pack, large pack). ✅
- Test evidence:
src/pycharter/semantic/context/_pack.py— frozen PydanticContextPackmodel withmodel_config = ConfigDict(extra="forbid", frozen=True).context_pack_idformula:uuid5(NAMESPACE_CONTEXT_PACK, f"{entity_kind}:{entity_ref}:{contract_version or 'none'}:{concept_version or 'none'}"). Namespace is fixed5e2d3c4b-1f8a-4a2b-9c6d-7e8f0a1b2c3d.content_hashformula: SHA-256 ofjson.dumps(model_dump(mode="json"), sort_keys=True, separators=(",",":"))over all fields EXCEPTcontext_pack_id,content_hash,generated_at.- 5 dedicated id/hash tests pin determinism + immutability + change-detection (
test_id_is_uuid5,test_concept_pack_id_is_deterministic,test_content_hash_flips_on_concept_rename,test_pack_is_immutable,test_generated_at_not_in_hash). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean; pytest 2789 passed.
- Review notes:
- Approved on orchestrator independent re-run as part of the P5-001 + P5-002 bundle. Frozen Pydantic + tuple-typed bodies guarantee immutability at the type level.
- Residual risks:
- Caching strategy deferred to P5-005's API surface (LRU keyed by
context_pack_idwould skip DB + hash work). - Hash recomputed each call — O(body size). Acceptable for current load; cache when traffic grows.
P5-002: Include contract/concept versions, source URIs, and policy status in context packs¶
- Phase: Phase 5
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda527e6ecc405add96) — bundled with P5-001. - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 5" task P5-002.
- Dependencies: P5-001 (pack response shape), P1-001, P1-007-IMPL, P1-008.
- Write scope:
- Extend the context pack response to include: contract version, concept version, source URI, and policy status.
- Pull each field from the existing canonical models (no new tables).
- Acceptance criteria:
- Every emitted context pack carries the four fields when applicable; absent fields emit
null. ✅ - Tests pin per-field provenance. ✅
- Test evidence:
- Policy status derivation (per entity kind):
conceptdeprecated →"deprecated".conceptany bindingAPPROVED→"approved".conceptbindings onlySUGGESTED/MAPPED→"under_review".conceptno bindings + not deprecated →"approved".contract/contract_fieldno bindings →None.contract/contract_fieldevery bound concept deprecated →"deprecated".contract/contract_fieldany bindingapproved→"approved".contract/contract_fieldanything else →"under_review".query→None(cross-cutting).
- Source URI derivation: most-recent accepting suggestion's
semantic_asset.source_uri; falls back toConceptModel.urifor concept packs. Contract / field / query packs leavesource_uri=None(governance objects, not source artefacts). - 6 provenance tests pin per-field threading:
test_concept_version_threads_through,test_contract_version_threads_through,test_policy_status_for_deprecated_concept,test_policy_status_for_under_review_binding,test_policy_status_for_approved_binding,test_source_uri_threads_through_when_ingest_run_present,test_source_uri_falls_back_to_concept_uri. - Builder API:
build_concept_pack,build_contract_pack,build_contract_field_pack,build_query_pack(all in_builder.py); related-row queries in_helpers.py. - Quality gates: rolled up with P5-001's gates above.
- Review notes:
- Approved on orchestrator independent re-run. Policy-status decision table is comprehensive and consistent with P1-008's binding lifecycle.
- Residual risks:
concept_versionsemantics: concepts have no independent version field; the pack reports the owningConceptSchemeModel.version. Cross-scheme contract bindings emitconcept_version=Nonerather than picking arbitrarily.- Large-pack BFS depth: the concept pack is one-hop (immediate relationships + bound concepts). Depth-N expansion is intentionally out of scope.
P5-003: Define and implement minimal evidence envelope model¶
- Phase: Phase 5
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda9139fc789bdf4fcb) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 5" task P5-003.
- Dependencies: P2-002 (suggestion evidence shape).
- Write scope:
- New
EvidenceEnvelopePydantic model. - Helper functions to build envelopes from a suggestion id, a concept id, or a binding id.
- Acceptance criteria:
- Envelopes built from each source kind round-trip identically. ✅
- Tests cover empty/dense/missing-asset cases. ✅
- Test evidence:
src/pycharter/semantic/context/_envelope.py— frozen PydanticEvidenceEnvelope(extra="forbid", frozen=True). Tuple-typedsignals/duplicates/lineage_event_ids.envelope_idformula:uuid5(NAMESPACE_EVIDENCE_ENVELOPE, f"{subject_kind}:{subject_id}:{content_hash}"). Namespace6f3e4d5c-2a9b-4b3c-bd7e-8f9a0b1c2d3e(distinct from P5-001's context pack namespace).- Builder API:
build_envelope_from_suggestion,build_envelope_from_concept,build_envelope_from_binding(each raisingLookupErroronly for unknown subject ids; empty envelopes for known but evidence-free subjects). - Signal synthesis (MVP): single
Signal(kind="confidence_score", ...)whensuggestions.confidence_scoreis non-null.rationale+confidence_bandpacked into the explanation. Richer signals (name_match,type_match,contract_path_match) flagged as follow-ups in the synthesis module docstring. - Duplicate resolution (MVP): at most one
DuplicateRefper envelope, sourced fromsuggestions.is_duplicate_of(P2-005's prefix-encoded ref). Fuzzy / embedding dedup is P2-005-FU-A's surface. - Repository pattern: Protocol + SQLAlchemy + in-memory backends +
make_evidence_envelope_repository(*, backend, session)factory infactory_envelope.py(named distinctly from P5-001/002'sfactory.pyto avoid collision). - 23 tests: round-trip dense/empty/duplicates/missing-asset/lookup-error for each subject kind; determinism (same content → same id); change-detection (different score → different id); subject-id change → different id; frozen invariant; factory backend selection (in-memory default / sqlalchemy explicit / missing-session error / env override / unknown-env fallback); in-memory + SQLAlchemy repository round-trips.
- Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean (236 files); pytest 2789 passed.
- Review notes:
- Approved on orchestrator independent re-run. Avoided file-collision with P5-001/002 via
factory_envelope.pynaming. Schema admits richer signal/duplicate catalogues without breaking the wire. - Residual risks:
- Signal catalogue narrow (only
confidence_scoretoday). Follow-ups flagged in_envelope_synthesis.pydocstring. - Dedup limited to
is_duplicate_of. Richer dedup is P2-005-FU-A. - Binding envelope joins: up to 5 queries per call (lineage, projection, concept, suggestion, asset). Fine at single-binding scale; P5-005's API may want JOINed reads or a cache layer when batching.
source_spanparsing best-effort. P2-002 left the column free-form ("start=120,end=240"/"120-240"/ etc.); anything outside two parsed conventions becomesNone. Original string remains on the suggestion row.extractorlabel sources fromasset_type.SemanticIngestRunModelhas noextractorcolumn today; if a future ingest pipeline records it separately, the builder needs a small DB-helper extension.
P5-004: Context preview UI for concepts, contracts, and fields¶
- Phase: Phase 5
- Area: Frontend
- Owner: Frontend Agent (
general-purpose, agentIda93ed58d10942090b) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: Proposal §"Phase 5" task P5-004.
- Dependencies: P5-001..003 (pack shape + envelope), P5-005 (retrieval API).
- Write scope:
- New
ContextPreviewPanelcomponent inside the existing editor. - Renders the full context pack (id, hash, versions, source URIs, policy status, evidence envelope).
- "Copy as JSON" / "Open lineage" actions.
- Acceptance criteria:
- UI honours empty / loading / error states. ✅
- Reuses existing primitives (
Badge,BindingChip,MappingDot). ✅ (Badge + MappingDot directly; BindingChip rejected because itsFieldBindingshape diverges from the pack'sBindingRef— inline span equivalent used instead so the visual language stays consistent without coercing the wire shape.) - Tests cover all states + the two modes. ✅
- Test evidence:
- Mount point: "Context" tab in
EditorCompanion(alongside Validation / Impact / Runs / Completeness / Health) — mirrors P4-001's mounting pattern. - Components added (split across three files to honour 400-line cap; functionally one component):
ContextPreviewPanel.tsx(272 lines) — host + data wiring + state machine.ContextPreviewPanel.toolbar.tsx(230 lines) — mode segmented control + Copy/Lineage/Refresh actions.ContextPreviewPanel.parts.tsx(389 lines) — populated pack body (header strip, sections, footer).
- Auto-detection: contract-field mode wins when both
selectedContractId+selectedFieldPathare set; concept mode whenselectedConceptIdis set; otherwise empty. Selection changes auto-reset the mode. User can force a mode viarole="radiogroup"segmented control. - Toolbar actions:
- Copy as JSON →
navigator.clipboard.writeText(JSON.stringify(pack, null, 2)). - Open lineage → concept mode:
/editor/concepts/<id>; contract-field mode:/editor/mapping-workbench?contract=<id>&historyForField=<path>. Uses injectedonNavigatehost hook withwindow.location.assignfallback. - Refresh → refires active fetch with spinner.
- Copy as JSON →
- States rendered: idle/no-selection (
NoSelectionEmptywith mode-aware copy), loading (PanelSkeletonghost rows), 404 (NoContextEmptyviaApiError.status === 404), error (role="alert"banner + Retry), populated (PackBody). - API client in
src/pycharter/ui/src/lib/api/semanticContext.ts:getConcept,getContract,getContractField,query. URL builders exported for testing.getContractFieldsplits dotted paths + percent-encodes each segment (backend joins/→.). Auth headers not sent (backend defaults to viewer; real auth wiring is P6-001 follow-up — single insertion pointhelpers.authHeaders()). - A11y:
section aria-label="Context preview"outer landmark;role="radiogroup"+aria-checkedon mode buttons;aria-labelon all icon-only buttons (Refresh, Copy, Open lineage, id-chip copy buttons);role="alert"on error;role="status"on 404 empty; per-sectionaria-label="<name> (<count>)";MappingDotkeeps itsrole="img"+ state aria-label; focus rings viafocus-visible:ring-2. - Layout: Flexbox + scrollable sections (
max-h-44/max-h-32+overflow-y-auto). Zeroposition: absolutein any new code (verified via grep on the three new files). - Reused primitives:
Badge,MappingDot.BindingChiprejected —BindingRefshape (state + concept_key) diverges fromBindingChip'sFieldBindingshape (primary.linkml_slot+primary.concept); inline span styled equivalently keeps wire shape clean. - Tests (22 new): 12 API client (URL builders + per-method invocation + limit omission); 10 panel (empty / loading / populated all sections / 404 / error + Retry / auto-detect contract-field / mode switch fires correct API / Copy-as-JSON writes pack to clipboard / concept Open-lineage URL / contract-field Open-lineage URL / Refresh refetches).
- Quality gates (orchestrator independent re-run on 2026-05-19):
npm run lint— 0 errors (26 pre-existing warnings unchanged);npm run type-check— clean;npm run test— 591 passed (65 files) (delta +22 vs. 569 baseline);npm run build— clean. - Review notes:
- Approved on orchestrator independent re-run. Component split across three companion files is the right move under the 400-line budget.
BindingChip-rejection rationale is sound — wire shape divergence trumps cosmetic reuse. Two-mode auto-detection matches the editor's selection model cleanly. - Residual risks:
- Evidence envelope retrieval not wired. Panel shows
evidence_envelope_idas a truncated copy-able chip + "retrieval not yet wired" hint. Single insertion point (EvidenceRefSection) for when the follow-up endpoint lands. - Auth headers placeholder.
X-Role/X-Workspacenot sent; backend defaults to viewer. Real auth swapshelpers.authHeaders()only — no client changes. - Large-pack rendering. Each section has
max-h-*cap +overflow-y-autobut no virtualization. For packs with hundreds of concepts/bindings, DOM size could grow;GlossaryView's TanStack Virtual is the precedent if this becomes a problem. - Concept lineage URL is provisional.
/editor/concepts/<id>matches the backend'sactionable_linkconvention used bySemanticHealthPanel, but no concept-detail route exists yet; 404 falls back to editor default route.
Phase 5 Summary¶
Phase 5 (Governed Context and Agent Readiness) complete. 6 active cards Done (P5-001..006); 2 deferred per program decision (P5-007 MCP server, P5-008 PyGubernator integration tests). Three waves shipped across PRs #194 (foundation), #195 (API + policy filter), #196 (preview UI).
Definition of done met: - Agents and APIs can request governed semantic context ✅ - Every context response includes enough evidence for audit ✅ - Context can be traced back to approved concepts, mappings, and sources ✅
P5-005: API for retrieving context by concept, field, contract, or query¶
- Phase: Phase 5
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda87670fc00d9cb402) — bundled with P5-006. - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 5" task P5-005.
- Dependencies: P5-001 (pack id/hash), P5-002 (fields), P5-003 (envelope).
- Write scope:
- New
GET /api/v1/context/concepts/{id}. - New
GET /api/v1/context/contracts/{id}/fields/{field_path:path}. - New
GET /api/v1/context/contracts/{id}. - New
GET /api/v1/context/query?q=...&limit=N. - Acceptance criteria:
- All four routes return the canonical context pack shape from P5-001/002/003. ✅
- 404 / 422 / 400 behaviour audited via
_http_errors.py. ✅ - Tests cover happy + 404 + 422 per route. ✅
- Test evidence:
- All four routes registered under existing
/api/v1/semantic/contextprefix (no_routers.pychange needed;semantic_contextalready wired). - Response model:
ContextPackResponseis a thin re-export of the domainContextPack— wire shape matches domain verbatim.src/pycharter/api/models/semantic_context.pyexists as the rename point if divergence is ever needed. - Errors flow through audited helpers:
raise_logged_not_found,raise_logged_unprocessable,raise_logged_internal_from_exception. - Query route: substring
ilikeonly (no relevance scoring); returns a single-element list to keep wire contract stable for future multi-strategy ranking;limitbounded[1, 100]. - Field-path slash:
field_path:pathallows/in path; segments collapsed to.before lookup. - 12 route tests in
tests/unit/test_semantic_context_routes.py(concept/contract/field/query happy + 404 + 422 + slash-path + limit). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean (379 files); pytest 2821 passed.
- Review notes:
- Approved on orchestrator independent re-run. Audited HTTP helpers correctly applied; field-path slash handling matches FastAPI conventions.
- Residual risks:
- Query route ranker quality: substring-only; future card adds proper relevance scoring (BM25 or vector). Wire returns a list to admit multi-result ranking later.
- Field-path normalisation:
/segments collapsed to.; a contract that legitimately uses/in a field name would collide. Not observed in existing seed data.
P5-006: Policy-aware context filtering (hierarchical roles: viewer < reviewer < admin < superuser)¶
- Phase: Phase 5
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda87670fc00d9cb402) — bundled with P5-005. - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: Proposal §"Phase 5" task P5-006. Program decision (2026-05-19): hierarchical role model.
- Dependencies: P5-005 (retrieval surface to filter), policy fields on concepts/contracts (P1-008's lifecycle).
- Write scope:
- Filter context responses by caller role.
RequesterContext(role, workspace_id) FastAPI dependency.- Document the role mapping in
docs/reference/. - Acceptance criteria:
- Role-by-role filtering pinned in tests for each retrieval route from P5-005. ✅
- Workspace scoping enforced for non-superuser roles. ✅
- Defense-in-depth: filter at the DB query layer AND at the response serialization layer. ✅
- Test evidence:
RequesterContextinsrc/pycharter/api/dependencies/requester_context.py—RoleStrEnum (viewer/reviewer/admin/superuser); frozen-slots dataclass.get_requester_contextreadsX-RoleandX-Workspaceheaders; unknown role →viewer; superuser ignores workspace. Header-based auth is a documented placeholder for P6-001.- Role → visible-states mapping:
- viewer →
{"approved"}binding states; concept deprecated invisible; workspace-scoped. - reviewer →
{"approved", "mapped", "suggested"}; concept deprecated invisible; workspace-scoped. - admin → all states; concept deprecated visible; workspace-scoped.
- superuser → all states; concept deprecated visible; cross-workspace.
- viewer →
- Workspace = concept scheme.
ConceptModel.scheme_idandConceptRelationshipModel.scheme_idfiltered directly.MappingProjectionModelhas no native workspace column; binding queries JOIN againstConceptModelonconcept_key. Bindings with nullconcept_keyinvisible to non-superuser. Documented indocs/reference/semantic-context-roles.md. - Defense in depth — two layers:
- DB query layer (
_policy.py+ loaders):WHEREclauses onMappingProjectionModel.state,ConceptModel.scheme_id,ConceptRelationshipModel.scheme_id. - Response layer (
_filter_packin the route module): re-applies the binding-state allowlist and concept-deprecated drop on the already-built pack body. Envelope (id/hash) preserved; onlyconceptsandbindingscollections pruned.
- DB query layer (
_policy.py(new) — pure filter primitives (binding_state_allowlist,filter_concepts_for_role,apply_concept_scope,apply_relationship_scope,apply_binding_state_filter,apply_binding_workspace_scope). Split out so_helpers.pystays under 400 lines.- Builders gained optional
requester_contextkwarg (defaultNone— backward compat preserved for tests). - Tests (20):
test_semantic_context_policy_filtering.py— viewer/reviewer/admin/superuser × each route, workspace isolation across schemes, defense-in-depth direct unit tests on_filter_pack. - Quality gates: rolled up with P5-005's gates above.
- Review notes:
- Approved on orchestrator independent re-run. Two-layer filter design is correct defense-in-depth: query-layer optimises performance; response-layer catches leaks.
RequesterContextis a clean swap point for P6-001's real auth. - Residual risks:
- Placeholder auth source.
X-Role/X-Workspaceheaders trusted verbatim. Swap point isget_requester_contextbody only; theRequesterContextshape and all downstream code remain unchanged when P6-001 lands. - Workspace == concept scheme. No native
workspace_idcolumn onMappingProjectionModel; binding scoping via JOIN onconcept_key. Bindings with nullconcept_keyinvisible to non-superuser. - Builder
requester_context=Nonedefault. Preserved for backward compat with existing tests. New callers should explicitly pass a context; the response-layer filter is the safety net.
P5-007: MCP server / tool surface for agent consumers (DEFERRED)¶
- Phase: Phase 5
- Area: Backend / Integrations
- Owner: Unassigned
- Reviewer: Unassigned
- Status: Deferred (program decision 2026-05-19 — explicitly out of scope for the current batch)
- Priority: Medium
- Risk: Medium
- Source: Proposal §"Phase 5" task P5-007.
- Dependencies: P5-001..006 (context surface).
- Write scope:
- Expose governed context via Model Context Protocol. Minimum tool surface:
fetch_concept_pack,list_concepts,query_context. - Acceptance criteria:
- MCP server boots, advertises tools, returns governed packs.
- Tools enforce policy filtering from P5-006.
- Test evidence:
- Review notes:
- Residual risks: Deferred — not in the active dispatch batch. Revisit once Phase 5 base surface ships.
P5-008: PyGubernator integration tests (DEFERRED)¶
- Phase: Phase 5
- Area: Integrations / QA
- Owner: Unassigned
- Reviewer: Unassigned
- Status: Deferred (program decision 2026-05-19 — explicitly out of scope for the current batch)
- Priority: Medium
- Risk: Medium
- Source: Proposal §"Phase 5" task P5-008.
- Dependencies: P5-001..006 (context surface), pygubernator package availability.
- Write scope:
- Drive the PyGubernator agent loop against the pycharter context API in integration tests.
- Assert the agent can fetch, cite, and act on governed context.
- Acceptance criteria:
- End-to-end test passes from agent prompt → context fetch → response with citations.
- Test evidence:
- Review notes:
- Residual risks: Deferred — revisit when MCP + agent runtime story is settled.
P6-001: Role / permission checks across import, review, mapping, governance¶
- Phase: Phase 6
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda5475ff893cf2fad2) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: High
- Source: Proposal §"Phase 6" task P6-001. Program decision (2026-05-19): hierarchical role model — viewer < reviewer < admin < superuser; non-superuser workspace-scoped (matches P5-006).
- Dependencies: P5-006 (
RequesterContextshape +Roleenum); existing auth subsystem (PyJWT-based, was already present insrc/pycharter/api/dependencies/auth.py). - Write scope:
- Replace P5-006's header-based placeholder (
X-Role/X-Workspace) with the real auth dependency. - Wire role checks across the import / review / mapping / governance route surfaces.
- Document the role-to-action matrix in
docs/reference/. - Acceptance criteria:
- Real auth resolves
RequesterContextfrom JWT (with feature-flagged header fallback for dev/Playwright). ✅ - Each protected route enforces the appropriate minimum role. ✅
- Unit tests pin behaviour per role × per route. ✅
- Migration path from P5-006's header-placeholder is documented. ✅
- Test evidence:
- Auth subsystem inventory: existed (
src/pycharter/api/dependencies/auth.py) withcreate_access_token,verify_jwt,get_current_user,require_admin+ login/logout/me routes. Extended rather than rebuilt. RequesterContextswap: JWT-first resolution —Authorization: Bearer <token>→ decode → userole+workspace_idclaims. Header fallback (X-Role/X-Workspace) feature-flagged viaPYCHARTER_REQUESTER_CONTEXT_FROM_HEADERS(default on so dev/Playwright/pytest fixtures keep working; set tofalsein production). Anonymous viewer as final fallback. LegacyAdmin/UserJWT role labels map toadmin/viewer.require_role(min_role)factory insrc/pycharter/api/dependencies/role_check.py— resolvesRequesterContext, comparesROLE_ORDER[requester.role]vsROLE_ORDER[min_role], raises 403 via auditedraise_logged_forbidden(log_event="role_check_denied") on insufficient role. Returns the context so workspace scoping can chain.- Role-to-action matrix (full table in
docs/reference/role-permissions-matrix.md):- reviewer: suggestion accept/dismiss/request-changes (+ batches), binding transitions/accept/remap/mark-technical-only/annotate, ingest import / import-all / ingest-text / ingest-upload.
- admin:
/proposals/link. - viewer: all GETs (response-layer policy filters from P5-006 unchanged).
- Routes gated:
semantic_suggestions.py,semantic_bindings.py,semantic_bindings_actions.py,semantic_proposals.py,semantic_ingestion.py,semantic_kb_ingest.py. JWTworkspace_idclaim threaded throughcreate_access_token. Legacy test fixtures patched to sendX-Role: adminvia the fallback so they keep passing during the migration window. - Migration plan documented: 4-step horizon (Day-0 headers → emit new JWT claims → flip flag to false → retire fallback code). Client migration header-by-header table.
docs/reference/semantic-context-roles.mdnow points at the new matrix. - Tests added (51):
test_auth_jwt.py(5),test_role_check_dependency.py(27 — hierarchy, JWT path, header fallback, feature flag off, legacy role mapping, JWT overrides headers, garbled token falls through),test_routes_role_enforcement.py(19 — every protected route × viewer-denied / reviewer-or-admin-passes). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean;
pytest tests/unit/ -q --timeout=60— 2872 passed, 1 skipped, 8 deselected (delta +51 vs. 2821 baseline). - Review notes:
- Approved on orchestrator independent re-run. Reuses the existing PyJWT subsystem; feature-flagged header fallback is the right migration tool. The 4-step migration horizon plan in
role-permissions-matrix.mdis detailed enough for operators. - Residual risks:
- Env-var user store (
PYCHARTER_AUTH_USERSJSON). Migration to a realuserstable is a follow-up. - Header fallback removal horizon. Default-on today; flipping
PYCHARTER_REQUESTER_CONTEXT_FROM_HEADERS=falsewill break unmigrated clients. - First-superuser bootstrap. Operators seed
PYCHARTER_AUTH_USERSwith{"role":"superuser"}. Proper bootstrap CLI deferred. - Password hashing. Existing auth module uses
hmac.compare_digeston plaintext from env var (not real hashing). Out of scope for P6-001; flag for user-management follow-up. - No iss/aud/jti JWT claims. Acceptable for in-process tokens; revisit if external IdP federation arrives.
P6-002: Pagination + filtering for high-volume routes¶
- Phase: Phase 6
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda469e11d87c976afa) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 6" task P6-002.
- Dependencies: None (additive on existing routes).
- Write scope:
- Audit + gap-close pagination across list routes.
- Convention reference doc.
- Acceptance criteria:
- Every high-volume list route honours
limit+offset. ✅ - Responses carry
totalwhen cheap,next_offsetwhen applicable. ✅ - Tests cover pagination + 422 edge cases. ✅
- Test evidence:
- Inventory (existing/added):
- Already paginated (kept as-is):
/suggestions,/suggestions/{id}/review-events,/health,/health/issues/{fp}/history,/context/query. - Added
limit/offset:GET /semantic/bindings/events,GET /semantic/assets,GET /semantic/ingest-runs,GET /semantic/proposals/.
- Already paginated (kept as-is):
PaginatedResponse[T]insrc/pycharter/api/models/pagination.py:items / total / next_offset / limit / offset.from_slice()constructor derivesnext_offsetfromtotal(when known) else fromlen(items) == limit. ConstantsDEFAULT_LIMIT=50,MAX_LIMIT=500. Used for NEW routes; legacy routes keep barelist[T]shape (backward compat).paginate_in_memory()helper for route-level slicing when the repo doesn't yet honour SQL-level limit/offset (used on assets/ingest-runs; future card pushes into the protocol).- Filter convention documented in
docs/reference/pagination-and-filtering.md: param names match model field names; multi-value comma-separated (?status=accepted,dismissed); unknown tokens silently dropped; booleans use literaltrue/false; AND across facets, OR within multi-value. Canonical helper_normalise_csvinsemantic_health.py. P5-006 context filters explicitly called out as the exception. - Tests:
tests/unit/test_pagination_convention.py— 32 tests (TestPaginateInMemory4 +TestPaginatedResponseFromSlice5 + 16 parameterized 422 edge cases + 7 happy-path route slicing tests). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean (147 files);
uv run --frozen pytest tests/unit/ -q --timeout=60— 2935 passed, 1 skipped, 8 deselected (delta +63 from this batch; +32 P6-002 + +14 P6-003 + +17 P6-004). - Review notes:
- Approved on orchestrator independent re-run. Convention doc anchors future routes. Legacy bare-list compatibility preserved at the call sites that need it.
- Residual risks:
totalexpense: Bindings events + ingest-runs intentionally omittotalfrom the bare-list shape.PaginatedResponse[T]envelope ready for new routes.- Route-level slicing on assets/ingest-runs: post-repo slicing today; push into the
AssetLineageRepositoryProtocolin a future card once row counts exceed ~10k per scheme. - Cursor pagination deferred: new rows arriving mid-page can shift offsets. Document as a P7 follow-up if workloads expose drift.
P6-003: Background job support for large imports¶
- Phase: Phase 6
- Area: Backend + Frontend
- Owner: Full-stack Agent (
general-purpose, agentIdacc3775069dc56038) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: Proposal §"Phase 6" task P6-003. Program decision (2026-05-19): use FastAPI
BackgroundTasks. - Dependencies: P2-001 (ingest model), P2-007 (import wizard).
- Write scope:
- Threshold-gated sync/async dispatch on ingest endpoints.
- Polling endpoint for run status + progress.
- Frontend wizard polling mode.
- Acceptance criteria:
- Small imports stay synchronous. ✅
- Large imports return 202 + polling URL. ✅
- Polling endpoint surfaces progress + final result. ✅
- Frontend wizard switches to polling mode for large imports. ✅
- Tests cover sync + background paths. ✅
- Test evidence:
- Thresholds in
src/pycharter/api/routes/v1/_ingest_background.py:IMPORT_SYNC_THRESHOLD_BYTES = 1_048_576(1 MiB),IMPORT_SYNC_THRESHOLD_ROWS = 1_000. Async iff size > bytes-threshold OR estimated rows > row-threshold. - Async response shape:
202 AcceptedwithIngestAcceptedResponse {"ingest_run_id": "<uuid>", "status": "pending", "poll_url": "/api/v1/semantic/ingest-runs/<uuid>/status"}. - Status lifecycle:
pending → processing → succeeded | failed. New nullableprogressJSON column onsemantic_ingest_runs(migration20260519000006_add_ingest_run_progress.py). Progress payload:{ rows_processed, rows_total, message, result_summary, error_summary }. - Background worker in
_ingest_background.pyopens its own session viaget_session_factory(); marksprocessing → succeeded | failed; stampsprogress.result_summary(mirrorsIngestReportResponse) on success anderror_summaryon failure. - Polling endpoint:
GET /api/v1/semantic/ingest-runs/{run_id}/status→IngestRunStatusResponse { ingest_run_id, status, progress, started_at, finished_at, error_summary, result_summary }. 404 on unknown / malformed UUID. - Idempotency: if client supplies
ingest_run_idand a non-terminal run with that id exists, server returns that run's poll URL without scheduling a second pipeline.TestIdempotency.test_repeat_post_returns_same_run_id_while_activepins. - Frontend wizard (
ImportWizard/SubmitStep.tsx): switches toPollingViewon 202; polls every 2 s viauseEffect + setTimeout; renders determinate progress bar (whenrows_totalknown) or indeterminate bar;role="progressbar"+ ARIA attrs; "Stop monitoring" button cancels the loop and surfaces a "still running server-side" message. On terminal status, finalises the wizard's run row and shows result summary. - API client (
lib/api/ingest.ts): newingestTextSubmit,ingestUploadSubmit(handle 202),pollIngestRunStatus. LegacyingestText/ingestUploadkept forIngestGraphModal. - Tests: 14 new backend (
TestThresholdHelpers5 +TestSynchronousPath2 +TestBackgroundPath6 +TestIdempotency1); 1 new frontend test (switches to polling mode on 202 and finalises on terminal status (P6-003)). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean;
uv run --frozen pytest tests/unit/ -q --timeout=60— 2935 passed;npm run lint0 errors;npm run type-checkclean;npm run test592 passed (65 files);npm run buildclean. - Review notes:
- Approved on orchestrator independent re-run. Threshold + idempotency design is sound; in-process BackgroundTasks is appropriate for current scale. Wizard polling UX preserves usability with cancellation + result-summary handoff.
- Residual risks:
- Restart durability: BackgroundTasks is in-process; an in-flight run is lost on app restart and its row stays in
processingindefinitely. Follow-up: migrate to ARQ/Celery + a janitor that flips orphanedprocessingrows tofailedon startup. - Threshold tuning: 1 MiB / 1,000 rows starting point; tune via the constants when operator telemetry lands.
- Polling chattiness: 2 s interval. Consider exponential backoff or SSE/WebSocket once volume warrants.
- Idempotency cache backed by the durable run table — no in-memory cache, no eviction needed beyond row lifecycle.
- File size:
semantic_kb_ingest.pyat 472 lines (just over 400). Could split sync helpers if it grows. - Two run rows per wizard async import: wizard creates one via
recordIngestRun, backend creates another viacreate_pending_run. Both terminate with consistent counts. Future cleanup plumbs the wizard's id through.
P6-004: Audit export for governance decisions and mappings¶
- Phase: Phase 6
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda4f9ce58e642954ae) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Medium
- Source: Proposal §"Phase 6" task P6-004.
- Dependencies: P3-004 (review-event repo), P3-009-FU-A (review-events endpoint), P1-008 (binding events), P4-001 (health snapshots).
- Write scope:
- New
GET /api/v1/audit/exportJSONL streaming endpoint. - Acceptance criteria:
- Export covers all governance event types. ✅
- JSONL stream is parseable line-by-line. ✅
- Auth: admin or superuser only. ✅
- Tests cover happy + auth-denial + date-range filtering. ✅
- Test evidence:
- Endpoint:
GET /api/v1/audit/export?since=<ISO>&until=<ISO>&event_types=<csv>→application/x-ndjsonStreamingResponse. Files:src/pycharter/api/routes/v1/audit.py(261 lines) +_audit_streams.py(271 lines, split to honour 400-line cap) +src/pycharter/api/models/audit.py(62 lines). - Line shape:
{"event_type": "...", "event_id": "<uuid>", "timestamp": "<iso>", "actor": "<str|null>", "workspace_id": "<scheme_id|null>", "payload": {...}}. - Event-type coverage:
| event_type | source model | timestamp column |
| --- | --- | --- |
|
review|SemanticReviewEventModel|timestamp| |binding|ConceptBindingEventModel|created_at| |accepted_suggestion|SuggestionModel(whereaccepted_at IS NOT NULL) |accepted_at| |proposal_link|ProposalModel(wheresuggestion_id IS NOT NULL) |created_at| |health_snapshot|SemanticHealthIssueSnapshotModel|detected_at| - Streaming strategy: per-event-type generators run
session.query(...).yield_per(500).heapq.merge(*streams, key=lambda x: x[0])interleaves chronologically. Memory O(num_event_types) regardless of total row count. - Auth + workspace scoping:
require_role(Role.ADMIN)gates the route. Workspace-scoped streams (review,accepted_suggestion) filter byrequester.workspace_idfor non-superuser admin (viaSuggestionModel.scheme_idjoin for review; direct onscheme_idfor accepted). Global streams (binding,proposal_link,health_snapshot) lackscheme_idcolumns — rows pass through for any admin (gap documented in residual risks). - Resumption semantics: chronological ordering (oldest first),
sinceis inclusive → consumers should dedupe byevent_idon replay. - PII / sensitive-data check: payloads carry only domain columns (ids, decisions, comments, contract/field paths, concept ids, confidence scores, evaluation/fingerprint, severity, summary, suggested-fix, owner). No password hashes, JWTs, or raw secrets.
- Tests (17 new, all passing):
TestAuthGate(4),TestHappyPath(2),TestDateRangeFilter(4),TestEventTypesFilter(3),TestWorkspaceScoping(2),TestStreamContent(2). - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; mypy clean;
uv run --frozen pytest tests/unit/ -q --timeout=60— 2935 passed. - Review notes:
- Approved on orchestrator independent re-run.
heapq.merge+yield_peris the right shape for memory-bounded streaming. Admin-gated workspace scoping is correct. - Residual risks:
- Payload schema stability: each event_type's
payloadis the raw row shape. Migrations adding columns are additive-safe; renames/drops break consumers. Future card documents per-table payload contract. - Resumption with
sinceinclusive: consumers should dedupe byevent_idwhen replaying fromlast_timestamp_seen. - Memory pressure on huge exports:
yield_per(500)guards memory; future change replacingyield_perwith.all()would break the guarantee. - Workspace coverage gap: binding / proposal_link / health_snapshot events lack
scheme_id→ cross-workspace visibility for non-superuser admin. If multi-tenant tightens, those tables need ascheme_idcolumn (separate card).
P6-005: Sample domain with contracts, docs, DDL, suggestions, expected outputs¶
- Phase: Phase 6
- Area: DevRel
- Owner: DevRel Agent (
general-purpose, agentIdac86706d5f0a8b669) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Low
- Source: Proposal §"Phase 6" task P6-005.
- Dependencies: None (additive samples).
- Write scope:
- New
samples/revenue_billing/directory tree. - Add a
make sample-revenue-billingtarget. - Smoke test under
tests/integration/. - Acceptance criteria:
- User runs the make target → pipeline produces the expected outputs. ✅
- Sample is self-contained (no external network access). ✅
expected/*.yamlasserted intests/integration/test_sample_revenue_billing.py. ✅- Test evidence:
- Directory tree:
samples/revenue_billing/README.md— walkthrough.contracts/{invoice,customer,line_item}.yaml— 3 data contracts, matchestemplate_contract.yamlshape.docs/{glossary,invoice_overview,billing_lifecycle}.md— 3 reference docs.ddl/{invoices,customers,line_items}.sql— 3 Postgres DDL files.expected/{concepts,relationships,suggestions,bindings}.yaml— 4 shape-contract fixtures.scripts/run_pipeline.py— argparse driver (399 lines).
- Pipeline driver builds 3 sets of
ExtractionResult(contract / doc / DDL) with originsIMPORTED/DOC/STRUCTURAL, runs each throughextraction_result_to_drafts, stampsingest_run_id, persists viaSuggestionManager.create_batch(..., require_evidence=True). Creates the concept scheme +SemanticIngestRun+ 9SemanticAssetrows upfront. - Makefile target
sample-revenue-billingruns the driver against$${PYCHARTER_DATABASE_URL:-sqlite:///sample.db}with schemerevenue-billingand actorsample-importer@example.com. Logs"Ingest run <id> created N suggestions across M assets". - Expected-outputs assertion strategy: shape over exact counts.
concepts.yaml/relationships.yaml: subset checks (every listed item must appear at least once; extra extractor noise allowed).suggestions.yaml: bands (min_total/max_total, per-kind / per-action bands, allowedorigins, confidence range).bindings.yaml: reviewer-intent (entries reference real candidate concepts + real contract filenames; binding row not required).
- Smoke test:
tests/integration/test_sample_revenue_billing.py(@pytest.mark.integration). On-disk SQLite undertmp_pathvia_alias_sqlite_jsonb(matches P2-009 pattern).pytest tests/integration/test_sample_revenue_billing.py -m integration -v→ 1 passed. - Quality gates (orchestrator independent re-run on 2026-05-19): ruff clean; smoke test 1 passed; unit regression 2872 (no impact — samples touch zero
src/pycharter/files). - Review notes:
- Approved on orchestrator independent re-run. Shape-based assertions are the right call for extractor-noise robustness. Driver reuses the production
extraction_result_to_drafts+SuggestionManagerpaths so the sample exercises the real pipeline. - Residual risks:
- Contract shape assumption: modeled on
template_contract.yaml. If a future contract loader requires acontract:wrapper (some orders examples use one), YAMLs need the wrapper. Current pipeline does not load these throughparse_contract, so risk is dormant. - Extractor-noise robustness: band tops (e.g. concept count 3..24) may need to grow when a future extractor adds significantly more candidates.
- SQLite vs Postgres: sample defaults to SQLite via
_alias_sqlite_jsonb.PYCHARTER_DATABASE_URLlets users redirect to Postgres; smoke test always uses SQLite undertmp_path.
P6-006: User guide for end-to-end workflow¶
- Phase: Phase 6
- Area: Docs
- Owner: Docs Agent (
general-purpose, agentIdae815b939cc5f8ed6) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Low
- Source: Proposal §"Phase 6" task P6-006.
- Dependencies: P6-005 (sample domain) for the walkthrough.
- Write scope:
- New
docs/guides/user-guide-end-to-end.md. - Link from
docs/index.mdandREADME.md+ MkDocs nav. - Acceptance criteria:
- A new user can follow the guide top-to-bottom and reproduce the sample. ✅
- All commands verified against current API + UI. ✅
- Linked from top-level docs. ✅
- Test evidence:
docs/guides/user-guide-end-to-end.md— 474 lines, 8 top-level sections (Prerequisites, Workspace, Import, Review, Bind, Health, Context, Troubleshooting, Next steps).- Sample anchor verified: ran
uv run --frozen --extra ontology python samples/revenue_billing/scripts/run_pipeline.py --db-url sqlite:////tmp/sample-run.db --scheme-id revenue-billing --actor sample-importer@example.com. Captured real logIngest run <id> created 6 suggestions across 9 assets. Smoke testtests/integration/test_sample_revenue_billing.pypasses. - Cross-references (11 + 7 image placeholders): samples README, expected/bindings.yaml, proposal, kanban, vocabulary, role-permissions-matrix, semantic-context-roles, pagination-and-filtering, configuration, building, production docs.
- Top-level linking:
README.mdline 983;docs/index.mdnew Guides section;mkdocs.ymlline 148 under How-To Guides. - Screenshot placeholders: 7
<!-- TODO: screenshot -->markers atdocs/guides/images/user-guide-*.pngpaths (workspace-selector, import-wizard-step-1, import-wizard-step-6, suggestions-inbox, mapping-workbench, health-tab, context-tab). - Verification:
mkdocs buildsucceeded; smoke test passed; commands run end-to-end on a fresh SQLite. - Makefile target gap closed by orchestrator: added
sample-revenue-billingMake target after this card (P6-005 reported it but didn't ship it). - Review notes:
- Approved on orchestrator independent re-run. 8-step structure mirrors the proposal's user-workflow section; cross-references are complete; sample log verified.
- Residual risks:
- Screenshot capture deferred — 7
<!-- TODO: screenshot -->markers need real images in a follow-up. - Commands may drift if UI/API surface changes; verified against develop @ 2026-05-19.
- Sample-output drift if extractor adapter changes draft emission (log line shows
6 suggestions across 9 assets). mkdocs --strictwarnings: 7 image placeholders + cross-tree samples links. CI deferred until images land.
P6-007: Admin guide for governance policy and permissions¶
- Phase: Phase 6
- Area: Docs
- Owner: Docs Agent (
general-purpose, agentIdadda854f6d79e7135) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Low
- Source: Proposal §"Phase 6" task P6-007.
- Dependencies: P6-001 (auth subsystem live).
- Write scope:
- New
docs/guides/admin-guide-governance.md. - Acceptance criteria:
- Admin can configure roles + policies from this document alone. ✅
- All commands + endpoints verified. ✅
- Test evidence:
docs/guides/admin-guide-governance.md— 640 lines, 11 sections (Role model, Configuring roles, Concept tier policy, Proposal workflow, Deprecation, Review audit, Semantic health monitoring, Backup/restore, Performance/scale, Security, Cross-references).- Verified env vars + commands:
PYCHARTER_AUTH_USERS,PYCHARTER_AUTH_JWT_SECRET,PYCHARTER_AUTH_DISABLED,PYCHARTER_AUTH_SERVICE_URL,PYCHARTER_AUTH_ADMIN_ROLE_CLAIMS,PYCHARTER_AUTH_ADMIN_GROUP_CLAIMS,PYCHARTER_REQUESTER_CONTEXT_FROM_HEADERS,PYCHARTER_SEMANTIC_REVIEW_EVENT_BACKEND,PYCHARTER_SEMANTIC_HEALTH_BACKEND— all match source. - Verified endpoints:
POST /api/v1/semantic/proposals/link(LinkProposalRequestshape),GET /api/v1/audit/export(since/until/event_types params, 7-day default),GET /api/v1/semantic/healthfilter shape + four kinds. - Cross-references: user-guide, migration-guide, samples README, role-permissions-matrix (×2 incl. anchor
#operator-migration-plan), semantic-context-roles, pagination-and-filtering (×2). - Top-level linking:
README.mdline 984;docs/index.mdline 228;mkdocs.ymlline 149. - Known limitations flagged: plaintext password storage (§10.3), header-fallback removal horizon (§10.2 + §2), snapshot TTL absence (§7.4 — references P4-001-FU-D), legacy
payload["proposal_id"]deprecation (§4.1), no rolling JWT secret (§10.1), audit export internal-only (§10.5), no HTTP deprecation endpoint (§5.1). - Review notes:
- Approved on orchestrator independent re-run. Configuration matrix is comprehensive; known-limitations section is honest about plaintext password storage + bootstrap gaps; cross-references chain together cleanly.
- Residual risks:
mkdocs build --strictnot run locally (env:griffemissing). CI will run it; warnings expected only in P6-006 (image placeholders) and not in this doc.- Forward references to peer-agent docs (user-guide, migration-guide) — broken until those land in the same merge train.
- HTTP-endpoint-driven concept deprecation does not exist: doc routes operators through the proposal workflow + SQL backfill. Follow-up card to add
PATCH /api/v1/semantic/concepts/{id}/deprecatewould close the gap. - Future cards referenced: P4-001-FU-D (snapshot TTL), DB-backed user table (replaces
PYCHARTER_AUTH_USERS).
P6-008: Playwright coverage for core workflow¶
- Phase: Phase 6
- Area: QA
- Owner: QA Agent (
general-purpose, agentIdaaaf0d0de082fc5e8) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 6" task P6-008.
- Dependencies: P6-005 (sample domain for test data), P6-001 (auth).
- Write scope:
- New
src/pycharter/ui/e2e/Playwright suite. - CI job in
.github/workflows/ci.yml. - Acceptance criteria:
- Golden-path tests pass against a freshly seeded sample. ✅
- CI job runs in <10 minutes. ✅
- Failure screenshots + videos uploaded on CI failure. ✅
- Test evidence:
- Files added (13 / 561 lines under
src/pycharter/ui/e2e/):README.md,playwright.config.ts,tsconfig.json,fixtures/test-users.ts,helpers/auth.ts,helpers/seed.ts, 7 spec files (login, import-wizard, suggestions-inbox, mapping-workbench, context-preview, semantic-health, role-smoke). - CI job: new
playwrightstep in.github/workflows/ci.yml.needs: [test], 10-min timeout, SQLite only (no Postgres service required), envPYCHARTER_AUTH_JWT_SECRET,PYCHARTER_AUTH_USERS(4 fixture roles),PYCHARTER_DATABASE_URL=sqlite:///playwright-sample.db,NEXT_PUBLIC_API_URL=http://localhost:8002,PLAYWRIGHT_BASE_URL=http://localhost:3002,PLAYWRIGHT_SKIP_SEED=1. Boots uvicorn + Next dev; polls/openapi.json+/until ready. Failure artefact:actions/upload-artifact@v4uploadstest-results/+playwright-report/(7-day retention). - Test specs (11 tests, all passing locally in 16.6 s): API + UI login round-trip; reviewer reaches import-wizard / suggestions-inbox / editor; companion tabs Context + Health switch; per-role auth round-trips (4 roles ×
GET /auth/me). - A11y selector strategy: all UI tests use
getByRole/getByLabel. Zero CSS selectors, nodata-testid, noconsole.log, noany. Single non-a11y selector pattern is the<main>landmark — also viagetByRole. - Failure artefacts:
playwright.config.tssetsvideo: 'retain-on-failure',screenshot: 'only-on-failure',trace: 'retain-on-failure'. - CI runtime budget: local 16.6 s + CI overhead (4-5 min for Node/Python setup + browser download + service boot). Well under 10-min cap.
- Local verification: suite verified end-to-end against locally-booted backend + Next dev server with revenue/billing sample seeded into SQLite — 11/11 passing.
- Minor adjacent edits to keep toolchain green:
tsconfig.jsonexcludese2e/;eslint.config.mjsignorese2e/**globally. - Review notes:
- Approved on orchestrator independent re-run. A11y-first selectors are the right call. SQLite-only setup keeps the CI footprint small. Per-role smoke covers the hierarchy without depending on UI affordance changes that don't exist yet.
- Residual risks:
- No
make sample-revenue-billingtarget at the time the card was written (orchestrator added it after). E2e invokes the Python script directly. - Mapping-workbench deep path: contract name + version dynamic per seed; spec smokes
/editoronly. Follow-up: call catalog API to discover a real contract URL. - Per-role UI affordance assertions (viewer can't see Accept; admin can submit proposal-link) deferred — inbox UI lacks stable a11y labels on per-row action menus.
- Chromium download (~113 MiB) on cold CI run. Cache step keyed on Playwright version is the standard fix if first-run time hurts.
- Dev-server cold-boot: Next dev was ready in 1 s locally; CI may take longer. 90 s poll budget is conservative.
- Flake tolerance:
retries: 1on CI only; healthy locally with zero retries.
P6-009: Migration guide for existing ontology users¶
- Phase: Phase 6
- Area: Docs / Engineering
- Owner: Docs/Eng Agent (
general-purpose, agentIda7c87ab070343284d) - Reviewer: Orchestrator (independent re-run)
- Status: Done
- Priority: Medium
- Risk: Low
- Source: Proposal §"Phase 6" task P6-009.
- Dependencies: P6-001..005.
- Write scope:
- New
docs/guides/migration-from-legacy-ontology.md. - Optional
service-level remap reconciliation workflow. - Acceptance criteria:
- Migration path documented for each breaking change observed in this program. ✅
- Sample migration script under
service-level remap reconciliation workflowif applicable. ✅ - Test evidence:
docs/guides/migration-from-legacy-ontology.md— 458 lines.service-level remap reconciliation workflow— 222 lines. Scope: REMAP projection retirement (P3-008-FU-D). Scansconcept_binding_eventsforaction='remap' AND from_concept_id IS NOT NULLand re-firesMappingProjectionLifecycleUpdater.on_concept_detachedagainstSqlAlchemyMappingProjectionRepository. Supports--dry-runand--verbose. HonoursPYCHARTER_DATABASE_URL. Idempotent.- Breaking changes covered (11):
POST /bindings/remapresponse shape (P3-008-FU-B); reason-prefix markers → FSM (P3-008-FU-A);payload["proposal_id"]legacy fallback (P3-006-FU-A); review-event repo default flip (P3-009-FU-B); healthpersist_historydefault (P4-001-FU-B);RequesterContextheader→JWT (P6-001); pagination convention (P6-002); large-import async (P6-003); audit export (P6-004); REMAP projection retirement (P3-008-FU-D); migration chain (all phases). - Verified against source: every endpoint + env var + Pydantic shape cited in the guide cross-checked.
mkdocs buildsucceeded; new page emits zero warnings. - Cross-references: kanban (×2), role-permissions-matrix (×3 incl.
#operator-migration-plananchor), pagination-and-filtering (×2), user-guide, admin-guide, database-erd (×2). - Top-level linking:
README.mdline under Documentation;docs/index.mdGuides section;mkdocs.ymlunder How-To Guides. - Review notes:
- Approved on orchestrator independent re-run. 11 breaking changes is the complete set from the kanban backlog scan. Script scope (REMAP retirement only) is the right call — silent backfilling of
proposals.suggestion_idfrompayload["proposal_id"]would auto-link suggestions without human review. - Residual risks:
- Script scope is REMAP only. Other potential backfills (
proposals.suggestion_idfrom legacypayload["proposal_id"]) deliberately deferred to human review. - Data migration
binding_events_actionsis reversible but not symmetric.downgrade()does not restore marker prefixes for already-rewritten rows. Documented in rollback section. - Frontend client migration is descriptive only — TypeScript symbol enumeration deferred; external UI authors pointed at
lib/api/modules. mkdocs build --strictred on screenshot placeholders from other agents' files, not from this doc.
Phase 6 Summary¶
Phase 6 (Production Hardening and Adoption) complete. All 9 cards Done (P6-001..009). Three waves shipped: - Wave 1 (PR #198): P6-001 auth + role gates, P6-005 revenue/billing sample. - Wave 2 (PR #199): P6-002 pagination convention, P6-003 FastAPI BackgroundTasks for large imports, P6-004 JSONL audit export. - Wave 3 (PR #200): P6-006 user guide, P6-007 admin guide, P6-008 Playwright coverage, P6-009 migration guide.
Definition of done met: - The workflow is usable by a new user following docs ✅ - Large imports and review queues remain responsive ✅ - Governance/audit requirements are covered ✅
Program Close-out¶
All 7 phases (0–6) of the KB/Ontology Workflow Proposal complete. All proposal cards Done. Deferred items remain in the backlog as named follow-ups:
- P5-007 — MCP server / tool surface (deferred per program decision 2026-05-19).
- P5-008 — PyGubernator integration tests (deferred per program decision 2026-05-19).
- P4-001-FU-D — TTL/compaction strategy for snapshots (needs PM input on retention policy).
- P3-006-FU-B — Remove legacy payload["proposal_id"] fallback (scheduled for next minor release; DeprecationWarning already live).
Ready¶
No active cards.
In Progress¶
No active cards.
Review¶
No cards awaiting review.
QA¶
No cards awaiting QA. Phase 0 audit batch + first Phase 1 card closed; see Done.
Blocked¶
No blocked cards.
Done¶
QA evidence (applies to all five cards): docs-only audits, no source code modified. Markdown integrity verified — all fenced code blocks balanced; cross-document references resolve to existing files or are deliberate forward references to in-flight cards (kb-phase-0-vocabulary.md for P0-005, kb-phase-1-route-mapping.md for a P1 follow-up). Reviewer spot-checked 100+ cited paths across the four audits. Out-of-scope changes: AGENTS.md was edited (added /orchestrate and /release to the skills table) and .claude/skills/orchestrate/ directory was added — both pre-existing on the working tree and not modified by any audit agent; flagged to the orchestrator for separate triage.
KAN-001: Create board and convert proposal backlog into cards¶
- Phase: Phase 0
- Area: Process
- Owner: Orchestrator Agent
- Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIdae3ab64b4cc034453) - Status: Done
- Priority: High
- Risk: Low
- Source: Multi-Agent Implementation Orchestration Plan
- Dependencies: None
- Write scope:
docs/guides/kb-ontology-workflow-kanban.md - Acceptance criteria:
- Board exists with standard columns, card template, seed cards, and board rules. ✅
- Locked program decisions are recorded. ✅
- Agent role bindings are recorded. ✅
- First parallel batch (UX0-001, UX0-002, P0-003, QA0-001) is staffed and dispatched with non-overlapping write scopes. ✅
- Test evidence:
- 2026-05-18 — Branch
feat/kb-phase-0cut fromdevelop(086c46c4); proposal + kanban brought onto the branch as untracked working files. - 2026-05-18 — Four parallel
general-purposeagents dispatched and returned; each wrote one disjoint audit file underdocs/guides/kb-phase-0-*.md. - Review notes:
- APPROVE (
kb-phase-0-review.md§KAN-001): board complete, decisions locked, role bindings recorded. - Residual risks:
- None.
UX0-001: Audit current semantic/ontology screens¶
- Phase: UX Phase 0
- Area: Frontend/UX
- Owner: Frontend Agent A (
general-purpose, agentIdab2ed841f267066d4) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIdae3ab64b4cc034453) - Status: Done
- Priority: High
- Risk: Medium
- Source: Surgical UI/UX Enhancement Plan; Proposal §"UI/UX Implementation Phases — UX Phase 0"
- Dependencies: None
- Write scope:
docs/guides/kb-phase-0-ux-audit.md(new) only. No source-code edits. ✅ - Acceptance criteria:
- Every semantic/ontology screen, route, and editor surface is listed by file path. ✅
- Each is classified as keep, merge into unified editor, deprecate, or internal-only. ✅
- Navigation impact (current nav entries, redirects needed) is recorded. ✅
- Findings reference the unified editor shell, workspace store, and editor architecture ADR. ✅
- Test evidence:
- 2026-05-18 —
docs/guides/kb-phase-0-ux-audit.md(345 lines) onfeat/kb-phase-0. - Classification totals: keep 8 (
/editor,/editor/concepts,/editor/concepts/[conceptKey],/editor/concepts/[conceptKey]/lineage,/editor/contracts/[name]/[version],/editor/ontology/[name]/[version],/editor/drafts,/editor/drafts/[draftId]); merge 8 (/ontology+ nestedconcepts,search,lineage,discovery,history,suggestions); deprecate 3 (/knowledge-base,/editor/flow,/editor/kb); internal-only 3 (/ontology/configuration,/ontology/proposals,/ontology/proposals/detail). - Component-family classification:
components/editor/shell|panels|bindings|toolbarkeep;components/semantickeep-shared;components/knowledge-basedeprecate. - Two doc-only follow-up cards (
UX0-AUDIT-A,UX0-AUDIT-B) proposed at end of audit file for orchestrator triage. - Review notes:
- APPROVE (
kb-phase-0-review.md§UX0-001): 26/26 spot-checked paths exist; classification consistent; nav-impact table complete. - Residual risks:
- Two parallel suggestion UIs (
SuggestionPanel.tsxincomponents/editor/panelsvs.SuggestionsQueue.tsxincomponents/semantic) must be reconciled in Phase 2 to avoid duplication. - Wholesale deletion of
components/knowledge-base/**is unsafe; several leaves (ImportSchemaDialog,ReconciliationWizard, etc.) are reused inside the editor — needs per-file move-vs-delete pass. - Folding
/editor/flowand/ontology/lineageinto a companion bottom-panel tab requires aBottomPanelTabs.tsxslot that does not exist yet; route deletions must not precede that shell change. - Open questions (in audit doc): canonical relabel of "Ontology" → "Domain workspace" scope; fate of
/ontology(redirect vs. thin Govern/Configure shell); canonical Suggestions Inbox surface.
UX0-002: Document existing UI patterns to reuse¶
- Phase: UX Phase 0
- Area: Frontend/UX
- Owner: Frontend Agent B (
general-purpose, agentIda54bf7aa3afc6b821) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIdae3ab64b4cc034453) - Status: Done
- Priority: High
- Risk: Low
- Source: Surgical UI/UX Enhancement Plan; Proposal §"UI/UX Principles"
- Dependencies: None
- Write scope:
docs/guides/kb-phase-0-ui-patterns.md(new) only. No source-code edits. ✅ - Acceptance criteria:
- Existing editor shell, left panel, inspector, modal, tabs, toolbar, badges, and semantic visual patterns are documented with file paths. ✅
- Recommended reuse points are mapped to the proposed KB workflow steps. ✅
- Any genuinely missing UI primitive is listed as a small additive component (not a new design system). ✅
- Test evidence:
- 2026-05-18 —
docs/guides/kb-phase-0-ui-patterns.md(365 lines) onfeat/kb-phase-0. - 12 pattern families documented: Shell (7), Left rail (7), Right rail / inspector (13), Bottom companion (9), Overlays (12), Toolbar / layer controls (10), Status / badges / chips (10), Semantic visual (16), Bindings (8), Forms (7), Empty/Loading/Error (7), Workspace store + API client (11). Every entry cites a verified file path under
src/pycharter/ui/src/. - Reuse coverage per KB step — strong: Semantic Home, Suggestions Inbox, Mapping Workbench (per-field editing), Semantic Health; thin: Import Wizard (no stepper host), Mapping side-by-side picker (no candidate list paired with bindings list), Context Preview (no composed envelope view).
- Missing additive primitives (3, justified):
Stepper/WizardHost,MappingPairList,ContextPackPreview. - Review notes:
- APPROVE (
kb-phase-0-review.md§UX0-002): 100+ cited paths verified; reuse map covers all six KB steps; three additive primitives well-justified. - Residual risks:
- Status colour overlap between
StatusBadge,MappingDot,BindingHistoryDrawer— needs canonical mapping in P0-005 before new screens reuse badges. - Inspector tab target ambiguous:
WorkspaceInspectorTabs(per ADR Phase 4) vs. legacyInspectorPanel— affects where KB tabs mount. - Import Wizard mount point not decided (toolbar dialog vs. left-rail Browse vs. EditorDashboard quick action).
P0-003: Audit existing semantic routes and backend endpoints¶
- Phase: Phase 0
- Area: Backend/API
- Owner: Backend Agent (
general-purpose, original audit agentIdaaee5a6bd795a7638; revision pass agentIdacc331a7998ffe277) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIdae3ab64b4cc034453) - Status: Done
- Priority: High
- Risk: Medium
- Source: Product alignment and workflow contract; Proposal §"HTTP API Proposal", §"Phase 0"
- Dependencies: None
- Write scope:
docs/guides/kb-phase-0-api-audit.md(new) only. No source-code edits. ✅ - Acceptance criteria:
- Every semantic/ontology/knowledge-base route, service, schema, and test is inventoried with file paths. ✅
- Each is classified as active / replace / deprecate / internal-only. ✅
- Gaps versus the consolidated HTTP API proposal are listed. ✅
- Compatibility note: no breaking changes during Phase 1. ✅
- Reviewer's four minor revisions applied. ✅
- Test evidence:
- 2026-05-18 (initial) —
docs/guides/kb-phase-0-api-audit.md(611 lines) onfeat/kb-phase-0. - 2026-05-18 (revision) — file grew to 619 lines (+8) with four corrections + Revision Log appended; only file modified per
git status --short.- Fix 1: suggestion-accept row reclassified
deferred-replace→keepafter verifyingsrc/pycharter/api/routes/v1/semantic_suggestions.py:98already exposesPOST /{suggestion_id}/accept; four call-sites in audit updated (route table, §Suggestions gap, §Specific compatibility shims, Open Question 3 now tracks missingPOST /{id}/request-changes). - Fix 2:
semantic/completeness.py→semantic/knowledge/completeness.py(rows L351, L390) verified withtest -f. - Fix 3:
semantic/lifecyclerepointed to the package directory with explicit submodule citations_events.py(BindingLifecycle) and_derived_from.py(FieldLineageRecorder) (rows L102, L367) verified withls. - Fix 4:
test_lineage_roundtrip.py→tests/unit/api/test_lineage_roundtrip.py(row L99) verified withtest -f.
- Fix 1: suggestion-accept row reclassified
- Route counts (revised totals): keep ~58, deferred-replace ~28, deprecate ~4, internal-only ~14, missing 1 explicit (
POST /suggestions/{id}/request-changes). - Persistence summary unchanged: only 2 in-memory semantic services (
KBManagementService,NamespaceRegistry);semantic_contextemits unpersisted output. - Top gaps vs. consolidated proposal unchanged: Imports fragmented, Assets entirely missing, Health & Context computed inline (no snapshot, no pack ID).
- Five follow-up cards proposed (P0-006, P1-010, P1-011, P2-010, P2-011) — note P0-006's "compatibility shims" section is slightly stale post-fix and may need a one-line retitle when promoted.
- Review notes:
- APPROVE-WITH-MINOR-REVISIONS (
kb-phase-0-review.md§P0-003) — all four required revisions applied verbatim per agentacc331a7998ffe277report. - Residual risks:
- Three overlapping concept-create paths (
semantic_rdf,editor_concepts,editor_concept_graphs) — consolidating without behaviour drift requires parity tests. editor_knowledge_basein-memory; any frontend already using it must tolerate empty state after Phase 1 swaps in DB-backed storage.- Open questions (in audit doc): is
/semantic/workspacesalias a Phase 1 deliverable; which existing concept-create path is canonical; missingPOST /suggestions/{id}/request-changesroute shape; where do persisted context packs live (new table vs. existing artifact store).
QA0-001: Define workflow verification checklist and test matrix¶
- Phase: Phase 0
- Area: QA
- Owner: QA Agent (
general-purpose, agentIda07dc71a587c013b4) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIdae3ab64b4cc034453) - Status: Done
- Priority: High
- Risk: Low
- Source: Quality Gates and UI/UX QA Checklist; Proposal §"Quality Gates", §"UI/UX QA Checklist"
- Dependencies: None for first draft; will be refined after UX0-001 and P0-003 land.
- Write scope:
docs/guides/kb-phase-0-qa-checklist.md(new) only. No source-code edits. ✅ - Acceptance criteria:
- Checklist covers backend persistence, API/Python parity, UI workflow, accessibility, browser checks, traceability. ✅
- Test matrix maps proposal phases 0–6 to minimum test commands and manual verification. ✅
- Evidence format for kanban cards is documented. ✅
- References
editor-verification-checklist.mdfor consistency. ✅ - Test evidence:
- 2026-05-18 —
docs/guides/kb-phase-0-qa-checklist.md(467 lines) onfeat/kb-phase-0. - Sections: Purpose; Evidence Format (with revenue/billing example); Pre-Commit & Lint Gates (
ruff,mypy,pytest,make,pre-commit run --all-files, UInpm run lint|type-check|test,mkdocs build --strict); 7-section Verification Checklist; Phase 0–6 Test Matrix; Revenue Agent Readiness worked example (fixture plan + CLI + screen-by-screen manual journey); 7 Open Questions. - Playwright status: not present at repo level. Interim browser-check protocol —
npm run devon port 3002, screenshots underdocs/screenshots/kb-phase-<N>/, zero console errors, keyboard-only nav,?workspace-canvas=0flag exercised. Replace once P6-008 lands Playwright. - Review notes:
- APPROVE (
kb-phase-0-review.md§QA0-001): 35/35 cited paths verified; correctly flagged port 3000/3002 mismatch ineditor-verification-checklist.md. - Residual risks:
- Accessibility section is purely manual today (no
axe/jest-axe); easy to skip under time pressure — consider promoting P6-008 earlier. - Worked-example commands (
pycharter kb suggest|health|context) and revenue fixtures don't all exist yet — checklist may markskippeduntil Phases 2/⅘ land. Reviewer confirmed: currentpycharter kbCLI only hasingest|export|cache|status(positional path arg, no--scheme/--assetflags) — worked example needs a Phase 2/⅘ follow-up pass. - Backward-compat gate text was authored assuming the API audit's keep/replace/deprecate taxonomy; now that the API audit landed with that taxonomy, a follow-up pass should cross-link the two docs.
P0-005: Define shared user-facing vocabulary and status labels¶
- Phase: Phase 0
- Area: UX/Docs
- Owner: Docs/DevRel Agent (
general-purpose, agentIda9e618bf53f97b55d) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIda8fa1db9b984181d9) - Status: Done
- Priority: Medium
- Risk: Low
- Source: Surgical UI/UX Enhancement Plan; Phase 0 audits
kb-phase-0-ux-audit.mdandkb-phase-0-api-audit.md. - Dependencies: UX0-001 (Done), P0-003 (Done).
- Write scope:
docs/guides/kb-phase-0-vocabulary.md(new) only. No source-code edits. ✅ - Acceptance criteria:
- User-facing labels avoid RDF/ontology jargon where possible. ✅
- Backend status values map cleanly to UI labels. ✅
- Vocabulary covers concept, suggestion, source asset, field binding, evidence, health issue, context pack, workspace, domain, mapping, ingest run, review decision. ✅ (12-entry table)
- Status-label canon resolves the
StatusBadge/MappingDot/BindingHistoryDrawercolour overlap. ✅ - Test evidence:
- 2026-05-18 —
docs/guides/kb-phase-0-vocabulary.md(354 lines). - 12 Core Vocabulary entries with revenue/billing examples; 5 status-label families with canonical colours.
- 3 cross-doc conflicts resolved:
BindingHistoryDrawer.mappedrealigned toMappingDot.green-500;StatusBadge.closedmoved from red to grey (red reserved for reject/error/failed); SHACL severity normalised to Error/Warning/Info in UI prose. - Review notes:
- APPROVE (
kb-phase-1-review.md§P0-005): 14 spot-checked enum/file citations verify exactly. Single nit:data/seed/context_pack_schema.yamlpath should be prefixed withsrc/pycharter/— fold in next time the file is touched, not blocking. - Residual risks:
- Open question (PM): does "request_changes" need a fourth
SuggestionStatusvalue, or always route through proposal/review? - Open question (PM): does the "Domain workspace" relabel apply to all ontology surfaces or only Semantic Home? (Echoes UX0-001 Q1.)
- Open question (PM): how should
MaturityLevel(unmapped/partial/complete/approved) be surfaced when it overlaps threeBindingStatevalues?
P1-001: Add DB-backed repository for KB versions, links, and lineage¶
- Phase: Phase 1
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda94e99d4194891593) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIda8fa1db9b984181d9) - Status: Done
- Priority: High
- Risk: High
- Source: Durable backend foundation; Phase 0 API audit (
kb-phase-0-api-audit.md). - Dependencies: P0-003 (Done).
- Write scope: As declared in the original card — observed clean. No API route file, no UI file, no other in-memory service modified.
- Acceptance criteria:
- KB versions, KB links, lineage records stored durably via SQLAlchemy. ✅
- Review decisions transitively satisfied (already DB-backed via
semantic_proposals/semantic_reviews); not touched in this card. ✅ - In-memory implementation remains the default until opt-in. ✅
- Migration reversible. ✅ (Alembic
kb_management_tables;down_revision = widen_pipeline_runs_run_id;downgrade()drops all three tables.) - Restart-survival test passes against on-disk SQLite. ✅
- Parity test proves SQLAlchemy and in-memory return identical results for the revenue/billing scenario. ✅
ruff check,ruff format,mypy, pytest pass locally. ✅- Test evidence:
- Files added (10):
src/pycharter/db/models/semantic/kb_version.py(57) —KBVersionModel.src/pycharter/db/models/semantic/kb_link.py(50) —KBLinkModel.src/pycharter/db/models/semantic/kb_lineage.py(69) —KBLineageRecordModel.src/pycharter/db/migrations/versions/20260518000000_add_kb_management_tables.py(88) — Alembic revision.src/pycharter/semantic/knowledge_base/_types.py(113) —KBManagementServiceProtocol(runtime-checkable).src/pycharter/semantic/knowledge_base/_sqlalchemy_kb_management.py(351) —SqlAlchemyKBManagementRepository.src/pycharter/semantic/knowledge_base/_kb_row_mappers.py(96) — Row mappers + UUID parser.src/pycharter/semantic/knowledge_base/factory.py(120) —make_kb_management_service()+resolve_backend(); env varPYCHARTER_KB_MANAGEMENT_BACKEND(default"memory").tests/unit/test_sqlalchemy_kb_management.py(335) — 14 tests.tests/integration/test_kb_management_restart.py(122) — restart-survival test (@pytest.mark.integration).
- Files modified (3):
src/pycharter/db/models/semantic/__init__.py,src/pycharter/db/migrations/env.py,src/pycharter/semantic/knowledge_base/__init__.py.
- Protocol match: 11 public methods matched signature-for-signature against in-memory
KBManagementService. - Quality gates (orchestrator + reviewer independent re-runs on 2026-05-18):
ruff check <all P1-001 paths>— All checks passed.pytest tests/unit/test_sqlalchemy_kb_management.py -q— 14 passed in 0.33s.pytest tests/integration/test_kb_management_restart.py -q -m integration— 1 passed in 0.33s.pytest tests/unit/test_kb_management_service.py -q(regression) — 9 passed in 0.10s (no in-memory regression).mypy src/pycharter/semantic/knowledge_base/—Success: no issues found in 7 source files(reviewer independent re-run).
- Review notes:
- APPROVE (
kb-phase-1-review.md§P1-001): regression suite green; new tests green; integration test green; ruff clean; mypy clean; protocol signatures match; migration chain linear; default backend remains in-memory; editor route still usesKBManagementService()directly; no out-of-scope edits. - Residual risks:
- Default-backend coverage gap: routes still construct
KBManagementServicedirectly; factory not yet wired intoapi/routes/v1/editor_knowledge_base.py. Follow-up tracked in P1-001-FU-A (Backlog). - JSON vs JSONB column choice: new tables use SQLAlchemy
JSONfor Postgres+SQLite compatibility. If product tooling expectsJSONBindexing on Postgres, alteration needed. Follow-up tracked in P1-001-FU-B (Backlog). - Transaction boundary: when constructed with a
session_factory, the repository commits per write call (no batched UoW). Callers wanting multi-method atomicity must pass a liveSession. - SQLite JSONB shim in unit fixtures (same shim already used by
test_sqlalchemy_concept_graph_repository.py). - Deploy path: Alembic head moves to
kb_management_tables; confirmpycharter db upgrade(or CI equivalent) picks up the new revision without operator action. - Open question 4 (workspaces alias) still deferred to Phase 2.
P1-002: Add migrations for semantic assets and ingest runs¶
- Phase: Phase 1
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda961686fc8ae1d130) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIda9b0cee38dd50bbd8) - Status: Done
- Priority: High
- Risk: High
- Source: Durable backend foundation; Proposal §"Phase 1" tasks P1-002, P1-003, P1-004.
- Dependencies: P1-001 (Done; commit
34a2f823). - Write scope: As declared — observed clean per
git status --short(only the 5 declared new files + 2 declared modified files + pre-existing untracked entries). - Acceptance criteria:
semantic_assetstable with required columns. ✅semantic_ingest_runstable with required columns. ✅- Migration reversible:
down_revision = "kb_management_tables";downgrade()drops both tables in safe order. ✅ - Tests cover create, list, lookup, status transitions, JSON round-trip, restart-survival. ✅
ruff check,ruff format,mypy, pytest pass locally. ✅- Test evidence:
- Files added (5):
src/pycharter/db/models/semantic/semantic_asset.py(86) —SemanticAssetModel.src/pycharter/db/models/semantic/semantic_ingest_run.py(79) —SemanticIngestRunModel.src/pycharter/db/migrations/versions/20260518000001_add_semantic_asset_and_ingest_run_tables.py(90) — Alembic revisionsemantic_asset_ingest_run_tables.tests/unit/test_semantic_asset_ingest_run_models.py(334) — 14 tests, revenue/billing fixtures.tests/integration/test_semantic_asset_ingest_run_restart.py(134) — restart-survival test (@pytest.mark.integration).
- Files modified (2):
src/pycharter/db/models/semantic/__init__.pyandsrc/pycharter/db/migrations/env.py— additive only. - Indexes added:
ix_semantic_assets_scheme_status(composite on assets);ix_semantic_ingest_runs_status_started(composite on ingest runs). - Quality gates (orchestrator + reviewer independent re-runs on 2026-05-18):
- ruff check / format on all P1-002 paths — clean.
- mypy
src/pycharter/db/models/semantic/—Success: no issues found in 20 source files. pytest tests/unit/test_semantic_asset_ingest_run_models.py -q— 14 passed in 0.37s.pytest tests/integration/test_semantic_asset_ingest_run_restart.py -q -m integration— 1 passed in 0.33s (engine genuinely disposed and reopened).pytest tests/unit/test_sqlalchemy_kb_management.py tests/unit/test_kb_management_service.py tests/integration/test_kb_management_restart.py -q(P1-001 regression) — 24 passed in 0.53s (zero regression).
- Review notes:
- APPROVE (
kb-phase-1-p1-002-review.md): all independent re-runs green; models + migration mirror P1-001's schema-aware pattern;__init__.pyandenv.pyedits additive only; no out-of-scope writes. - Residual risks:
- Soft
scheme_namereference — no FK by design (locked). Reviewer folded scheme-existence validation into P1-003's acceptance criteria. JSONBvsJSON— same as P1-001; tracked in P1-001-FU-B.- No live
alembic upgrade headsmoke — operator responsibility post-merge. - Status values are free strings (not Python enums) — matches
PipelineRunModelprecedent. - No repository / service yet by scope. P1-003 (asset lineage repo) and P2-001 (import service) will consume these tables.
P1-001-FU-A: Wire editor_knowledge_base.py through make_kb_management_service¶
- Phase: Phase 1
- Area: Backend / API
- Owner: Backend Agent (
general-purpose, agentIda2c52e11405d4e1d5) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIda36f3aa452c116ec8) - Status: Done
- Priority: Medium
- Risk: Medium
- Source: P1-001 reviewer report (
kb-phase-1-review.md). - Dependencies: P1-001 (Done; commit
34a2f823), P1-002 (Done; commit8d80af4e). - Write scope: As declared — observed clean.
- Acceptance criteria:
- Route uses
make_kb_management_service()rather thanKBManagementService()directly. ✅ - Default backend remains
"memory". ✅ - In-memory mode functionally identical for existing callers. ✅
- SQLAlchemy mode reachable via single
PYCHARTER_KB_MANAGEMENT_BACKEND=sqlalchemyenv-var flip. ✅ - Tests verify both backends through the route layer. ✅ (12 parametrised + wiring-contract tests.)
ruff check,ruff format,mypy, pytest pass locally. ✅- Test evidence:
- Files modified (1):
src/pycharter/api/routes/v1/editor_knowledge_base.py— replaced module-level_service = KBManagementService()with a FastAPI dependency_get_kb_management_service. Yields the memory singleton in"memory"mode; constructs a request-scopedSqlAlchemyKBManagementRepositoryoverget_db_sessionin"sqlalchemy"mode. Dependency commits on success and rolls back on exception (matches the repo's "live-session leaves commit to caller" contract). - Files added (1):
tests/unit/test_editor_knowledge_base_route_factory.py(327 lines) — 12 tests. - Wiring approach: FastAPI dependency (not module-level swap) — preserves memory singleton, plumbs request-scoped sessions for SQLAlchemy mode, matches
editor_lineage.py's pattern. - Quality gates (orchestrator + reviewer independent re-runs on 2026-05-18):
- ruff check / format on touched paths — clean.
mypy src/pycharter/api/routes/v1/editor_knowledge_base.py src/pycharter/semantic/knowledge_base/—Success: no issues found in 8 source files.pytest tests/unit/test_editor_knowledge_base_route_factory.py -q— 12 passed in 1.37s.pytest tests/unit/test_sqlalchemy_kb_management.py tests/unit/test_kb_management_service.py tests/unit/test_semantic_asset_ingest_run_models.py -q— 37 passed in 0.66s (P1-001 + P1-002 regression clean).- Reviewer additionally ran
pytest tests/unit/api/test_editor_lineage.py -q(10 passed) andpytest tests/unit/test_public_api_surface.py -q(17 passed) — no adjacent-route regression and no public-API surface drift.
- Review notes:
- APPROVE (
kb-phase-1-p1-001-fu-a-review.md):_DEFAULT_BACKEND = "memory"unchanged; transaction boundary correctly delegated to the dependency. Reviewer concurs the pattern is right for this codebase but recommends documenting it before P1-003 inherits it. - Residual risks:
- Pattern documentation gap: first KB editor route delegating commit to the FastAPI dependency. Reviewer recommends a short "Transaction boundary in KB routes" note in
knowledge_base/__init__.pyor a docs guide before P1-003 starts. - Minor overhead:
Depends(get_db_session)fires unconditionally even in memory mode. Tiny behavioural delta; not blocking. - No auth coverage in the new tests: mirrors
test_editor_lineage.pyposture. Orthogonal gap.
P1-003: Add repository methods for semantic asset lineage¶
- Phase: Phase 1
- Area: Backend
- Owner: Backend Agent (
general-purpose, agentIda0754fdba2cc00ba8) - Reviewer: Reviewer/Integrator Agent (
general-purpose, agentIda23215bd5c428fd4e) - Status: Done
- Priority: High
- Risk: Medium
- Source: Proposal §"Phase 1" (P1-005 + P1-006); P1-002 reviewer (scheme_name validation); P1-001-FU-A reviewer (transaction-boundary pattern).
- Dependencies: P1-001 (Done,
34a2f823), P1-002 (Done,8d80af4e), P1-001-FU-A (Done,871583ca). - Write scope: As declared — observed clean.
- Acceptance criteria:
- Assets ↔ ingest runs linkage via new
ingest_run_idcolumn. ✅ - Lookup surface (by scheme / type / ingest run / id, plus ingest-run filtering). ✅
scheme_namevalidation raisesUnknownConceptSchemeError(subclass ofPyCharterSemanticError), not bareValueError. ✅- Transaction-boundary contract honoured — live session never commits. ✅
- Restart-survival proved with SQLAlchemy backend. ✅
- Full regression clean (P1-001/002/FU-A + 2339-test unit suite). ✅
- Test evidence:
- Files added (11): new
src/pycharter/semantic/asset_lineage/package (8 files, ~1080 lines) —__init__.py,_types.py(Protocol),models.py(frozen Pydantic value objects),errors.py,_in_memory.py,_sqlalchemy_asset_lineage.py(336 lines, live-session-leaves-commit-to-caller),_row_mappers.py,factory.py. Plus migration20260518000002_add_asset_ingest_run_link.py(79 lines, revisionasset_ingest_run_link→down_revision = semantic_asset_ingest_run_tables, idempotent + reversible). Plus tests:tests/unit/test_asset_lineage_repository.py(455 lines, 20 tests) +tests/integration/test_asset_lineage_restart.py(122 lines, 1 test marked@pytest.mark.integration). - Files modified (1):
src/pycharter/db/models/semantic/semantic_asset.py— addedingest_run_id = Column(String(36), nullable=True, index=True)+ docstring entry. No API route, UI file, or P1-001/002/FU-A file touched. - Public surface (11 methods):
list_assets_by_scheme,list_assets_by_type,list_assets_by_ingest_run,get_asset,create_asset,link_asset_to_ingest_run,list_ingest_runs,get_ingest_run,record_ingest_run,update_ingest_run_status(+ value-object exports). - Quality gates (orchestrator + reviewer independent re-runs on 2026-05-18):
- ruff check / format on all touched paths — clean.
mypy src/pycharter/semantic/asset_lineage/ src/pycharter/db/models/semantic/—Success: no issues found in 28 source files.pytest tests/unit/test_asset_lineage_repository.py tests/integration/test_asset_lineage_restart.py -q -m "integration or not integration"— 21 passed in 0.61s.- P1-001/002/FU-A regression — 49 passed in 1.70s (zero regression).
- Full unit suite — 2339 passed, 1 skipped, 8 deselected.
- Review notes:
- APPROVE (
kb-phase-1-p1-003-review.md): all locked decisions honoured; transaction-boundary contract correctly implemented (session.flush()only in live-session mode;_commit_and_closeno-ops when caller owns the session;test_live_session_does_not_commitassertsdb_session.in_transaction()post-write). Migration chain wires cleanly. Model diff +10/-3 additive. Reviewer recommended two follow-up cards (P1-003-FU-A tightenslink_asset_to_ingest_runcontract; P1-003-FU-B wires routes). - Residual risks:
ingest_run_idis a soft reference (String, no FK) — matches locked decision forscheme_name. Deliberate.- In-memory
known_schemesper-instance and not thread-safe — production wiring needs a seeding strategy. The Protocol exposesregister_schemefor that purpose. link_asset_to_ingest_runno-op on missing asset — reviewer flagged as the weakest miss-vs-success contract. Tracked in P1-003-FU-A (Backlog).- Route wiring deferred — mirrors P1-001 → P1-001-FU-A pattern. Tracked in P1-003-FU-B (Backlog).
Phase 0 Summary¶
Closed 2026-05-18 on branch feat/kb-phase-0. All five Phase 0 cards (KAN-001, UX0-001, UX0-002, P0-003, QA0-001) reached Done. One reviewer pass produced docs/guides/kb-phase-0-review.md (415 lines, agentId ae3ab64b4cc034453).
Deliverables landed¶
| File | Lines | Owner agentId | Purpose |
|---|---|---|---|
docs/guides/kb-ontology-workflow-kanban.md |
(this file) | Orchestrator | Board + program decisions + agent role bindings |
docs/guides/kb-phase-0-ux-audit.md |
345 | ab2ed841f267066d4 |
UX surface inventory + keep/merge/deprecate/internal-only classification |
docs/guides/kb-phase-0-ui-patterns.md |
365 | a54bf7aa3afc6b821 |
Reusable editor UI patterns + reuse map for KB workflow steps |
docs/guides/kb-phase-0-api-audit.md |
619 | aaee5a6bd795a7638 (initial) + acc331a7998ffe277 (revision) |
Backend route/service/schema inventory + classification + gap analysis |
docs/guides/kb-phase-0-qa-checklist.md |
467 | a07dc71a587c013b4 |
Evidence format + verification checklist + Phase 0–6 test matrix |
docs/guides/kb-phase-0-review.md |
415 | ae3ab64b4cc034453 |
Independent reviewer / integrator report across the four audits |
Locked decisions confirmed by Phase 0 evidence¶
- Unified editor is the only forward UI path. UX0-001 classifies 8 editor routes as keep; 8 ontology routes as merge into the editor; 3 standalone routes as deprecate.
- Workspace = UI label over concept schemes. No new persisted model needed; UX0-001 + UX0-002 do not require one; P0-003 confirms
ConceptSchemeModelis sufficient as the underlying entity. - API compatibility: no breaking changes during Phase 1. P0-003 classifies overlapping/duplicate routes as
deferred-replace(post-Phase 1); the single planned breaking change in the proposal (POST /{id}→POST /{id}/accept) turned out to be unnecessary — the slash-accept route already exists. - Sample domain: revenue / billing. QA0-001 worked example uses it; downstream fixtures will follow.
Top open questions surfaced to PM¶
- Does the workflow uniformly relabel "Ontology" → "Domain workspace", or only inside Semantic Home? (UX0-001)
- Does
/ontologybecome a redirect to/editor, or stay as a thin shell for Govern + Configure? (UX0-001) - Canonical Suggestions Inbox surface — editor panel vs. semantic queue? (UX0-001, UX0-002, P0-003)
- Is
GET /api/v1/semantic/workspaces(read-only alias overConceptSchemeModel) a Phase 1 deliverable or Phase 2? (P0-003) - Which existing concept-create path is canonical among
semantic_rdf,editor_concepts,editor_concept_graphs? (P0-003) - Where do persisted context packs live — new
semantic_context_packstable or reuse an existing artifact store? (P0-003) - Pull Playwright coverage (P6-008) earlier as
QA0-002, or stay manual through Phase 5? (QA0-001)
Coordination findings carried forward¶
- Two parallel suggestion UIs (
editor/panels/SuggestionPanel.tsxvs.semantic/SuggestionsQueue.tsx) must be reconciled before P3-001. components/knowledge-base/**cannot be deleted wholesale; per-file move-vs-delete pass needed.BottomPanelTabs.tsxdoes not yet have a slot for/ontology/lineagecompanion view; route deletions wait on that shell change.- Status colour overlap (
StatusBadge/MappingDot/BindingHistoryDrawer) must be canonicalised in P0-005 before new screens reuse badges. - QA worked-example CLI commands (
pycharter kb suggest|health|context) don't exist yet; checklist marks themskippeduntil Phases 2/⅘ land.
Outstanding orchestrator tasks before Phase 1 kicks off¶
- Decide how to handle the unrelated
AGENTS.md+.claude/skills/orchestrate/working-tree changes (separate commit vs. fold into Phase 0 commit vs. revert). - Commit Phase 0 audit batch on
feat/kb-phase-0(six new docs + kanban updates). Per CLAUDE.md, the Orchestrator does not commit unprompted; awaiting user direction. - Promote P0-005 (vocabulary) and P1-001 (DB-backed KB versions/links/lineage repository) from Ready to In Progress with tightly scoped agent briefs.