Integration seam — cross-package handshake

Status: v0.1 (Workstream D, 2026-05-28) · Audience: integrators wiring a decoupled sibling package (e.g. pyintake) to pycharter

This document describes the opt-in integration surface pycharter exposes to decoupled sibling consumers, governed by docs/proposals/cross-package-decoupling.md. Both packages stay fully decoupled — no shared imports, no shared storage. The wire shapes are defined as duplicated-copy DTOs derived from docs/contracts/integration-dtos.linkml.yaml.

Wire shapes (source of truth)

All DTO shapes live in docs/contracts/integration-dtos.linkml.yaml. The pycharter-side Pydantic mirrors are exported from src/pycharter/api/models/integration.py. The verification reference model is defined in its sibling integration_verification.py module.

Top-level wire DTOs carry schema_version (currently "0.1"). The new VerificationReference requires it explicitly. Receivers reject versions outside their supported range.

Flow A — Inbound suggestions

POST /api/v1/vocabulary/suggestions/inbound?scheme_id=<scheme-uuid>
Authorization: Bearer <token>
Content-Type: application/json

{ ...SuggestionBatch... }
→ 201 Created { ...SuggestionBatchReceipt... }

Persists every ConceptCandidate in the batch as a row in the governed suggestion queue. The route does not map the producer's vocabulary onto pycharter's concept registry runtime-side; it only stages each candidate.

Classification in the receipt:

• accepted_count — candidates whose concept_key already exists in the target scheme. A suggestion row is still created (to extend / reinforce the existing concept's evidence) but the count signals "no new concept proposed."
• auto_staged_count — candidates whose concept_key did NOT match. A FREE-tier suggestion is created for human review.
• rejected — candidates that could not be staged at all (e.g. empty concept_key). Includes a code and detail.

Auth: Role.REVIEWER or higher (see api/dependencies/role_check.py).

Idempotency (caveat closed, GV-107): the server dedupes on batch_id. The first successfully processed batch stores its receipt in the inbound_batch_idempotency table; every later POST with the same batch_id short-circuits and replays the original receipt verbatim (including suggestion_ids and rejected) without creating new suggestion rows. Retries are safe. Semantics producers can rely on:

• batch_id is the idempotency key; request bodies are not compared. A different candidate list under a reused batch_id still replays the original receipt, so mint a fresh batch_id (a UUID is recommended) per logical batch.
• Receipts are cached only after the batch commits. A failed request (4xx/5xx) caches nothing; retrying it re-processes from scratch.
• The cache degrades safely, never fatally: a receipt lookup or store failure is logged and treated as a cache miss (the request is re-processed, or the receipt is simply not cached) rather than failing the request.
• Retention is operator-side: cached rows are pruned by a periodic cleanup (90 days is a sensible start; see db/models/inbound_batch_idempotency.py). A retry arriving after its record was pruned re-processes as a fresh batch — the queue review UI still surfaces duplicates via is_duplicate_of.

No separate Idempotency-Key header is needed: batch_id is carried in-band in the DTO, so a retransmitted request dedupes by construction, and the replayed response is the original receipt (the semantics an Idempotency-Key header would otherwise provide).

Producer responsibilities (NOT pycharter's):

• LLM extraction, embedding, vector retrieval, chunk storage — all out of scope for pycharter per docs/proposals/cross-package-decoupling.md. The producer pre-computes everything and only sends ConceptCandidate / RelationshipCandidate shapes.
• Confidence calibration. The receiver records the value but does not interpret it; review UI may rank by it.

Flow B — Contract snapshot (implemented)

GET /api/v1/contracts/{contract_id}/snapshot serves the ContractSnapshot wire DTO (with nested FieldBindingDoc entries) for pull-based consumers. contract_id is the contract row UUID; unknown ids return 404, malformed ids return 400. The route requires REVIEWER or higher (require_role), mirroring Flow A's auth posture for integration flows.

The response carries:

• schema_version — wire-DTO version, "0.1".
• contract_id / version — contract name and semver.
• format_version — the DG-2 artifact format version the server currently writes; refuse formats you don't recognise.
• schema_payload — the inline JSON Schema.
• domain_schema — optional name:version LinkML reference.
• field_bindings — one FieldBindingDoc per bound field (legacy concept-centric and LinkML class/slot bindings are both projected to the same shape).
• ai_usage_policy (GV-106, additive within 0.1) — the contract's declared AIUsagePolicy (CF-1 shape) as a JSON object, or absent/null when the contract declares none. Consumers that embed, index, or otherwise feed contract data to AI systems MUST read and obey this policy. Server-side, the policy is read from the contract's metadata record payload (metadata.ai_usage_policy) — the same storage convention VC-11 established for extraction_policy.

Who enforces each ai_usage_policy field

pycharter carries the whole policy but enforces only the KB/AI-facing part of it. Read this list before treating a field as a guarantee; the field descriptions in src/pycharter/db/schemas/_policy_models.py say the same thing, and a unit test keeps them saying it.

• allowed_for_embedding — enforced by pycharter's KB emitter (record_builder derives embeddable = allowed_for_embedding and not redacted per field) and honoured by pyintake at embed time.
• pii_redaction_required — enforced by pycharter's extraction capture gate (GV-102: value capture goes shape-only for every field of the contract), the KB emitter, and the LLM-assist gate (GV-109). KB/AI-only: it does not govern pipeline sinks — those key off the per-field pii / restricted flags.
• approved_models — enforced by pyintake at embed time (model allowlist); pycharter only carries it.
• retention_period — applies to embedded/indexed copies of data under this contract; enforced by pyintake worker retention. It does not expire source records or facts.
• citation_required — informational: declared for retrieval consumers; not enforced by pycharter or pyintake today.
• retrieval_eval_threshold — informational: declared for retrieval consumers; not enforced by pycharter or pyintake today.

Status: the pycharter-side enforcement above is live. On the pyintake side, the embed-time hook for allowed_for_embedding / approved_models and the retention_period sweep are workstream B of docs/guides/privacy-flags-enforcement-proposal.md (B1, B2) — until those land, retention_period is carried but not swept.

Serializer: src/pycharter/api/routes/v1/_contracts_snapshot.py (pure projection, no side effects). Tests: tests/unit/test_contract_snapshot.py.

Flow C — ResourceBinding payload (already shipped, DG-1)

Pycharter's existing POST /api/v1/semantic/resource-bindings endpoint accepts the ResourceBindingTransitionRequest shape, which matches Flow C's proposed ResourceBindingPayload field-for-field. There is no new endpoint needed for v0.1.

Flow D — Verification reference for an action host (shape only)

VerificationReference is a metadata-only report of a persisted verification. It carries exactly schema_version, receipt_id, contract_namespace, contract_name, contract_version, verdict (accepted or rejected), context_fingerprint, and timezone-aware verified_at. It contains no candidate record, accepted record, prompt, or action permission. A stale-context or authorization error has no successful verdict and must not be represented as an accepted reference.

contract_namespace is a required JSON key whose value is either a nonempty namespace string or null for the global namespace. LinkML does not distinguish an absent slot from a null value, so the source slot uses wire_required_nullable: true and the Pydantic mirror enforces presence with str | None. Consumers must preserve the explicit null on the wire. context_fingerprint uses sha256: followed by 64 lowercase hexadecimal digits.

The reference may travel as metadata over an authenticated, trusted channel or be resolved through a trusted, authenticated read scoped to the host's principal and workspace. Agent-supplied metadata alone is not evidence. For receipt-ID handoff, the read must return only persisted, currently usable receipts; missing or revoked receipts fail closed. The receipt ID alone is not a bearer token. The host compares the returned ID, exact contract namespace/name/version, governed context fingerprint, verdict, and an appropriate freshness window against values from its own trusted state. It then applies its own operation policy. An agent-provided contract or fingerprint cannot establish those expected values. The DTO does not bind an action payload to the verified bytes; a real host must also obtain the accepted normalized record or an authorized reference to it under R2.4.

The wire shape has a mock action host under examples/agent_verification_bridge/. PyCharter's durable receipt store and authenticated receipt read interface are R3 work and are not exposed by this Flow D yet. The mock reader makes that dependency visible without claiming a deployed endpoint. PyActuator integration is a separate cross-repository change and must use its public operation contract.

Optophi composition pattern

Package	Responsibility at this seam
PyIntake	Proposes candidate meanings through SuggestionBatch. A proposed meaning is not approved context.
PyCharter	Stages and reviews those suggestions, serves approved contract and semantic context, and verifies structured records under an exact version. Its receipt reports that verification; it does not authorize execution.
PyActuator	Decides whether and how an operation may execute, using its own policy and audit trail. It may consume a trusted PyCharter verification report as one input.

The packages exchange the published DTOs and public operations. They do not import each other's internal types or share stores. Each package keeps its own policy decision and audit boundary.

Record sidecar — _concept_annotations as the join key

Beyond these flows, annotated records themselves are an integration surface. Pipelines running with annotate: true stamp every record with a _concept_annotations object (records with no bound fields still carry the key as an empty dict). The shape is a stable record-level contract — see "Annotation as a record-level contract" in docs/proposals/cross-package-decoupling.md, which governs it on equal footing with the wire DTOs; the emitter is annotate_record in src/pycharter/semantic/contracts/runtime.py.

Resolution pattern. Per bound field path the sidecar carries a primary block:

{
  "_concept_annotations": {
    "trade_date": {
      "primary": {
        "kind": "concept",
        "concept": "trade.execution_date",
        "concept_scheme": "trading-domain:1.0.0"
      }
    }
  }
}

concept + concept_scheme (a name:version reference) form a resolvable pointer, not a payload. Consumers (e.g. pyintake) resolve full concept context out-of-band via:

• Flow B — GET /api/v1/contracts/{contract_id}/snapshot (schema_payload, field_bindings, ai_usage_policy);
• concept packs / MCP context surfaces — fetch_concept_pack / query_context in src/pycharter/mcp/server.py, backed by the governed context packs in src/pycharter/semantic/context/.

Do not expect concept metadata (definitions, altLabels, relationships) inline in records. A kind: "linkml_slot" primary carries class + slot instead of concept; optional annotations (secondary bindings), tags, owner, and confidence may also appear.

Why the sidecar stays thin (GV design decision). Duplicating concept metadata per record bloats the substrate and goes stale the moment a concept is edited — resolution happens at read time against the governed registry instead. Sidecar evolution is additive only; a breaking shape change requires a format_version bump on the contract artifact (DG-2 machinery, docs/reference/artifact-format-versioning.md).

Extraction-side provenance (GV-101/102). Consumers reading pycharter's extraction output may additionally observe:

• Evidence for bound fields is keyed by the bound concept_key and its provenance carries binding: {field, scheme, kind} (GV-101).
• Restricted fields are value-free in evidence (restricted: true, shape statistics only — no top_values / enum_candidate) and appear masked as "__redacted__" in any reservoir-derived record samples (GV-102). The _concept_annotations sidecar itself is binding metadata, not data, and is exempt from masking.

Configuration

The inbound endpoint is gated by the PYCHARTER_INBOUND_SUGGESTIONS_ENABLED environment variable (BACKLOG Theme C item C4 — shipped as an env var rather than the originally sketched integrations.inbound_sources.<name>.enabled config key). Default is enabled, preserving v0.1 behaviour; set false / 0 / no / off (case-insensitive) to disable. The route stays registered — the gate is evaluated per request — and disabled deployments answer 404 so the surface is not discoverable. When enabled, the require_role (REVIEWER) dependency still applies. See src/pycharter/api/routes/v1/inbound_suggestions.py and tests/unit/test_inbound_suggestions.py (TestInboundDisabledByConfig).

Schema drift discipline (developer-facing)

The Pydantic models exported from src/pycharter/api/models/integration.py are hand-authored to match docs/contracts/integration-dtos.linkml.yaml. Treat the LinkML file as the contract; the Pydantic is a derived artifact. When you edit either side, update both in the same PR, including the sibling integration_verification.py model. A structural drift check for the DTOs has shipped and runs in pre-commit/CI: the make generate-integration-dtos target is a drift-check alias (it verifies that the hand-authored Pydantic mirrors the LinkML source), not a code generator. Fix any reported drift by editing the Pydantic mirror to match the LinkML shapes.

Pycharter's src/pycharter/semantic/generators/_pydantic.py already produces deterministic Pydantic from a LinkMLSchema; tying it into a make target and a pre-commit drift check is straightforward follow-up work tracked under Cross-Package Decoupling in BACKLOG.

References

• Proposal: docs/proposals/cross-package-decoupling.md
• LinkML / shape vs. meaning: docs/proposals/linkml-as-schema-substrate.md
• DTO source-of-truth: docs/contracts/integration-dtos.linkml.yaml
• Pydantic mirror: src/pycharter/api/models/integration.py
• Route: src/pycharter/api/routes/v1/inbound_suggestions.py
• Annotation runtime: src/pycharter/semantic/contracts/runtime.py
• Tests: tests/unit/test_inbound_suggestions.py