Fixture-only recurring FMP reference¶

This Linux reference adds an operator-owned Prefect 3 schedule around the bounded EOD application. Each scheduled flow launches that application in a fresh process. A separate monitor reads durable destination receipts and continues when ingestion stops. No agent, MCP connection, PyCharter API or queue worker stays open. This profile always uses synthetic loopback HTTP; it cannot schedule live FMP requests.

It requires an authenticated reviewed source checkout. The public 0.0.67 package does not include the underlying PipelineIO API. This guide does not establish a publicly distributable ingestion bundle, production deployment, financial-market calendar, shared API-key budget, or all-symbol capacity. Those gates remain open.

Review and register¶

Use a clean pinned checkout and a dedicated environment:

python -m venv .venv-reference
. .venv-reference/bin/activate
pip install -e '.[reference,prefect]'
pip install -r examples/fmp_reference/requirements-prefect.txt
cp -R src/pycharter/data/seed/examples/fmp_reference ./my-fmp-project

The tested scheduler toolchain pins Prefect 3.8.8 and SQLAlchemy 2.0.51. Actual scheduler acceptance found that Prefect 3.8.8's bulk insertion fails with SQLAlchemy 2.1.4; the optional prefect extra constrains SQLAlchemy below 2.1. The standalone EOD requirements file has a different tested SQLAlchemy version; use requirements-prefect.txt for this combined profile.

Review and version runner.yaml, scheduled-project.yaml, pipeline.yaml, contract.yaml, and fixture.json in the user's project. Runner policy declares the cadence, maximum start lateness, runner polling, child deadlines, independent monitor cadence, retry/overlap ownership and team names. Team names identify responsibility; they do not configure notifications or grant permissions. Only mode: fixture, zero automatic flow retries and one active flow are allowed. The seed schedule checks two synthetic dates every ten seconds with a forty- second observation-age limit. These short intervals demonstrate behavior; they are not financial-data refresh recommendations.

Validate the supported native profile, then inspect the exact bundle without destination or source access:

python -m examples.fmp_reference validate \
  --project my-fmp-project/scheduled-project.yaml
python -m examples.fmp_reference.runner inspect --policy my-fmp-project/runner.yaml

Validation checks the native profile semantics; inspection fingerprints the reviewed inputs and recorded toolchain rather than certifying their semantics.

Review the reported policy and bundle_sha256, then explicitly register that digest to an operator-owned path:

python -m examples.fmp_reference.runner register \
  --policy my-fmp-project/runner.yaml \
  --registration /path/to/approved-registration.json \
  --expected-sha256 REVIEWED_BUNDLE_SHA256

Registration atomically publishes complete evidence without replacing a different registration. It binds exact project/pipeline/contract/runtime and fixture hashes, authored runner bytes, Git revision, and selected direct dependency/ driver/parser/Python versions. The imported PyCharter engine must come from this checkout; executed source and package configuration must be committed and clean. The reported dependency list is not a complete transitive environment lock. Keep the reviewed environment immutable; the public-bundle gate includes a resolved dependency lock and clean installation from the distributed artifacts. No secret values are stored. A changed bundle requires review and a new registration path. Registration is a local change-control check; an agent with write access can replace the policy and registration. It is not an authorization boundary against that agent.

Set the named destination environment variable through the operator's local secret mechanism. Provision the dedicated reference tables once using a DDL role, then use a routine SELECT/INSERT/UPDATE role as described in the EOD guide:

python -m examples.fmp_reference provision \
  --project my-fmp-project/scheduled-project.yaml

No FMP API key is needed. Fresh CLI children receive the named database secret and a small environment allowlist; they do not inherit FMP keys, proxy credentials or Prefect settings. Provider data and database URLs are absent from flow parameters, flow results and correlation logs.

Operate three independent components¶

Component	Owner and lifetime	Failure behavior
Prefect API/server and its own state database	Scheduler operator; continuously available	New schedules stop if unavailable; destination observations remain inspectable
`runner serve`	Ingestion operator/supervisor; continuously available	No new jobs start while down; independent monitor becomes overdue
`runner monitor`	Monitor operator/supervisor; separate process	Must be supervised independently; its absence cannot declare data healthy
Reference CLI job	Fresh child per flow/poll	Terminates after work or its hard bound; committed state remains in the user destination
Destination PostgreSQL	Database owner; persistent	Failed/unavailable observation is unhealthy; inspect possible receipts before replay

For a local demonstration, set a private PREFECT_HOME, an explicit loopback API URL, and disable server analytics in these processes, without changing global Prefect settings. Example environment values:

export PREFECT_HOME=/path/to/private-prefect-home
export PREFECT_API_URL=http://127.0.0.1:4200/api
export PREFECT_SERVER_ANALYTICS_ENABLED=false
prefect server start --host 127.0.0.1 --port 4200 --no-ui

Use a separate terminal or supervisor for ingestion:

python -m examples.fmp_reference.runner serve \
  --policy my-fmp-project/runner.yaml \
  --registration /path/to/approved-registration.json

Use another independently supervised process for monitoring:

python -m examples.fmp_reference.runner monitor \
  --policy my-fmp-project/runner.yaml \
  --registration /path/to/approved-registration.json

The monitor does not use the Prefect API and makes no source requests. It emits timestamped JSON with current-artifact coverage/observation health. Unknown child output, destination errors, timeout and registration drift are unhealthy events. An operator must supervise this process and route its output to an alert system; this reference does not implement notification delivery or monitor-heartbeat alerts. Freshness means the reviewed static window was successfully observed recently, not that current trading dates or source publication times are known.

Failure, restart and changes¶

Prefect owns schedule/overlap/cancellation; the project owns bounded HTTP retries. There are no automatic flow retries. Overlapping flow starts use a single-slot deployment limit with new collisions cancelled; destination advisory locking remains the last reference-writer protection. Late attempts fail rather than silently replaying an arbitrarily old outage queue. pause_on_shutdown: false keeps the declared schedule through supervisor restarts; late queued work may appear as failed/cancelled scheduler attempts before a new timely run succeeds.

The reviewed bundle is checked at serve startup and each flow/health poll. The CLI checks source/configuration identity before I/O and captures retained input bytes. Deploy the checkout read-only and keep the same reviewed interpreter and environment; this is not protection against concurrent hostile source edits. Jobs have hard subprocess bounds, terminate owned process groups on cancellation or timeout, and reap their direct children. Linux guards stop a CLI whose parent or serve owner dies, and stop a flow whose serve owner disappears. Guards are process cleanup, not a security sandbox. A flow gets five seconds to report its terminal state and release the Prefect concurrency slot before a hard kill. If the Prefect API is also unavailable, scheduler state/slot release can remain unknown until operator reconciliation or Prefect lease recovery; restarting serve alone is not proof that a fresh job was accepted. The verified killed-runner scenario keeps the independent Prefect API available.

Each attempt has a reference UUID before subprocess launch, derived from the Prefect flow-run UUID and engine attempt number. Safe reference_attempt and reference_child INFO logs correlate that UUID with child PID and registration. Keep INFO correlation logs in the operator's logging sink if absence of a receipt must be investigated. A committed/failed reference receipt atomically records the scheduler UUID, attempt, child PID, policy/registration hashes and source revision. Commit acknowledgement or stdout loss does not prove rollback; query the destination by the known reference UUID before deciding to replay.

Stop ingestion while leaving server, destination and monitor available. After the last compatible commit exceeds max_observation_age_seconds, the independent monitor reports observation_overdue, even with no new ingestion attempt. Detection also includes monitor polling, process startup and its configured timeout; the configuration check is a bounds sanity check, not a scheduler SLA. Restart the same serve command with the same registration. New timely runs re-observe the fixture and report two unchanged rows without duplicate keys.

Changing source, fixture, native artifacts, operating policy, recorded dependency or interpreter identity refuses an old registration before new extraction. Review, inspect and register the new bundle, then restart under the new pinned policy. This does not reconcile manual edits to the Prefect deployment, implement a pause/retire command, or roll back database/schema state. Those broader PIPE-9 gates remain open. Preserve the server state and registration on routine restart; delete only the operator-owned disposable demonstration resources after stopping their processes. Never remove an existing application database or container.

Acceptance¶

tests/integration/test_fmp_prefect_schedule.py uses an isolated real Prefect server and real SQLite/PostgreSQL destinations. It verifies two scheduled fresh job processes, matching durable correlation receipts, a killed serve owner while a CLI is active, termination of owned descendants, continued independent monitor and server operation, overdue observation, and safe resumed publication. test_fmp_runner_recovery.py proves receipt inspection after commit/output loss and rejects changed core/package configuration on restart. Unit checks cover registration drift, policy limits, unknown monitoring, output bounds, and real process-group cleanup under timeout/cancellation. All source responses are synthetic; no live FMP requests are made by these recurring tests.

Acceptance uses the documented seed policy's process deadlines and observation window. Probe waits derive from that policy; the complete multi-process outage scenario has a separate five-minute test cap. PostgreSQL cases own disposable databases so parallel provisioning cannot collide. Failure diagnostics contain only fixed reason codes and health flags. These harness bounds do not change the runner's configured execution deadlines or promise a scheduler SLA.

Official Prefect documentation reviewed 2026-10-13 describes local process serving, deployment creation and cancellation ownership. The reference composes the existing atomic CLI rather than the generic whole-pipeline task adapter, whose publication guarantees differ.