Data Profiling

Profile datasets to understand their shape, completeness, and distributions. Use profiling to explore new data sources, detect drift between pipeline runs, or produce a quality summary alongside contract validation.

Quick start

The simplest way to profile data is the profile_data() function:

from pycharter import profile_data

data = [
    {"user_id": 1, "age": 28, "email": "alice@example.com", "score": 0.85},
    {"user_id": 2, "age": None, "email": "bob@example.com", "score": 0.62},
    {"user_id": 3, "age": 35, "email": None, "score": 0.91},
]

report = profile_data(data)

print(report["record_count"])  # 3
print(report["field_profiles"]["age"]["null_percentage"])  # 33.33
print(report["field_profiles"]["score"]["mean"])  # 0.793
print(report["overall_stats"])

You can also restrict profiling to specific fields:

report = profile_data(data, fields=["age", "score"])

Using the DataProfiler class

For more control (e.g. reusing the profiler across calls), use the class directly:

from pycharter.quality import DataProfiler

profiler = DataProfiler()
report = profiler.profile(data)