Data Profiling

Profile datasets to understand their shape, completeness, and distributions. Use profiling to explore new data sources, detect drift between pipeline runs, or produce a quality summary alongside contract validation.

Quick start

The simplest way to profile data is the profile_data() function:

from pycharter import profile_data

data = [
    {"user_id": 1, "age": 28,  "email": "alice@example.com", "score": 0.85},
    {"user_id": 2, "age": None, "email": "bob@example.com",  "score": 0.62},
    {"user_id": 3, "age": 35,  "email": None,                "score": 0.91},
]

report = profile_data(data)

print(report["record_count"])   # 3
print(report["field_profiles"]["age"]["null_percentage"])   # 33.33
print(report["field_profiles"]["score"]["mean"])            # 0.793
print(report["overall_stats"])

You can also restrict profiling to specific fields:

report = profile_data(data, fields=["age", "score"])

Using the DataProfiler class

For more control (e.g. reusing the profiler across calls), use the class directly:

from pycharter.quality import DataProfiler

profiler = DataProfiler()
report = profiler.profile(data)