Skip to content

Validate a CSV

Start with a draft inferred from your columns, review the requirements, then check the rows. Inference describes the sample; it cannot decide which values are acceptable for your business. This example adds a nonnegative-amount rule and shows the record it rejects.

Install pycharter, then save this synthetic input as orders.csv in a new example directory:

order_id,amount
A-100,19.95
A-101,-5
A-102,0

Run:

import pandas as pd
from pycharter import sketch_from_dataframe, validate_dataframe

df = pd.read_csv("orders.csv")
contract = sketch_from_dataframe(df, name="orders")

# Review the inferred draft and add a real acceptance requirement.
contract["json_schema"]["properties"]["amount"]["minimum"] = 0

report = validate_dataframe(df, contract, on_error="quarantine")
print(len(report.valid))        # 2
print(len(report.quarantined))  # 1
print(report.quarantined[["order_id", "amount"]])  # A-101, -5

Verified with public pycharter==0.0.67. Here, quarantine separates failing rows into an in-memory DataFrame; it does not persist them to a queue or database. Inspect report.summary() or the quarantined frame's _violations column for diagnostics. Treat real rejected values as potentially sensitive.

Reuse the reviewed requirements

Save the reviewed schema in a native contract document so future batches are checked against the same rules. The inferred helper dictionary and the file format differ in 0.0.67; the explicit envelope below is reloadable. This writes orders-contract.yaml in your example directory:

from pathlib import Path
import yaml

document = {
    "api_version": "pycharter.io/v2",
    "kind": "DataContract",
    "metadata": {"name": "orders", "version": "0.1.0"},
    "spec": {"json_schema": contract["json_schema"]},
}
Path("orders-contract.yaml").write_text(yaml.safe_dump(document, sort_keys=False))
report = validate_dataframe(df, "orders-contract.yaml", on_error="quarantine")

Do not re-infer the contract from every new batch and treat passing validation as evidence that nothing changed. Review deliberate changes to types, required fields, ranges and business rules before adopting them.

Next