Validate a pandas DataFrame¶
Apply a reviewed contract to every row, then inspect the accepted data and
failures separately. This self-contained example works with public
pycharter==0.0.67 and the base installation:
import pandas as pd
from pycharter import validate_dataframe
df = pd.DataFrame([
{"order_id": "A-100", "amount": 19.95},
{"order_id": "A-101", "amount": -5},
{"order_id": "A-102", "amount": 0},
])
contract = {
"json_schema": {
"version": "1.0.0",
"type": "object",
"properties": {
"order_id": {"type": "string"},
"amount": {"type": "number", "minimum": 0},
},
"required": ["order_id", "amount"],
}
}
report = validate_dataframe(df, contract, on_error="quarantine")
print(len(report.valid)) # 2
print(len(report.quarantined)) # 1
print(report.summary())
report.valid contains passing rows; report.quarantined contains failures and
a _violations column. These are in-memory results. Persist or publish them
explicitly when your workflow requires it. A saved contract path can replace
the dictionary; see the CSV recipe.
Choose what a failure means¶
on_error |
Behavior |
|---|---|
"quarantine" (default) |
Return passing rows and failing rows separately |
"fail" |
Raise DataFrameValidationError; inspect its .report |
"warn" |
Log a warning, collect diagnostics and pass all rows through unchanged; this is not an acceptance gate |
For a batch that must be entirely acceptable before proceeding:
from pycharter import DataFrameValidationError
try:
validate_dataframe(df, contract, on_error="fail")
except DataFrameValidationError as exc:
print(len(exc.report.quarantined)) # 1
# Stop or route the failed batch according to your application's policy.
With coercion enabled, accepted output uses the contract-coerced values.
Pass coerce=False to keep original values in the returned valid frame;
this option is not a substitute for strict-type validation.
Inspect individual records¶
from pycharter import Validator
validator = Validator.from_dict(contract)
results = validator.validate_batch(df.to_dict(orient="records"))
for row_index, result in enumerate(results):
if not result.is_valid:
print(row_index, result.errors)
For large inputs, process bounded chunks appropriate to your memory budget. Keep raw failures out of shared logs unless your access and retention policy allows them. For aggregate quality thresholds and monitoring, continue with quality checks.
See also: CSV validation, local pipeline enforcement, and FastAPI request validation.