Contributing to PyCharter¶

Thank you for your interest in contributing to PyCharter! This guide will help you get started.

Code of Conduct¶

By participating in this project, you agree to abide by our Code of Conduct. Please be respectful and constructive in all interactions.

Getting Started¶

Development Setup¶

Fork and clone the repository

git clone https://github.com/YOUR_USERNAME/pycharter.git
cd pycharter

Create a virtual environment

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

Install development dependencies

pip install -e ".[dev,api,ui,etl]"

Optional: From the repository root, ./scripts/setup.sh creates venv/, installs editable [dev] plus API extras when present, and registers pre-commit and pre-push hooks using python -m pre_commit (avoids broken venv/bin/pre-commit shebangs if the repo is moved).

Install pre-commit hooks (recommended)

python -m pre_commit install
python -m pre_commit install --hook-type pre-push

Without this step, Ruff and other hooks will not run when you commit or push, and CI may fail on formatting or lint.

Run tests to verify setup

pytest

Project Structure¶

pycharter/
├── src/pycharter/       # Main package
│   ├── api/            # REST API
│   ├── pipeline_generator/  # Pipelines (extract/transform/load)
│   ├── quality/        # Quality monitoring
│   ├── runtime_validator/  # Validation
│   └── ...
├── tests/              # Test suite
├── docs/               # Documentation
└── docs/notebooks/     # Example Jupyter notebooks

Branch model and releases¶

We use a simple Git flow so anyone can contribute without learning a heavy process:

Branches: Work on feature branches, merge into develop, then develop → main.
Releases: Done from main. Pushing a version tag (e.g. v0.0.33) triggers the release pipeline: tests run, the package is built and published to PyPI, and a GitHub Release is created from that tag.

To release a new version (from main):

Merge develop into main and ensure main is in a good state.
Create and push a tag (the tag name is the version used for PyPI and the release):
```
git tag v0.0.33
git push origin v0.0.33
```
The CI workflow will run tests, build, publish to PyPI, and create the GitHub Release. No need to edit pyproject.toml for the version when using tags—the tag is the source of truth for that run.

Optionally, keep pyproject.toml’s version in sync with the latest release so local installs and docs show the right number. The release itself is still tag-driven; you can use ./scripts/release.sh release to run the repo’s release steps, or bump pyproject.toml manually if you want the repo version to match the latest tag.

Making Changes¶

Branch Naming¶

Use descriptive branch names:

feature/add-kafka-extractor
fix/validation-error-handling
docs/improve-etl-tutorial

Coding Standards¶

Python style¶

We use Ruff for linting and formatting (see .pre-commit-config.yaml and Makefile):

# Format and auto-fix (same as make format)
ruff format src/pycharter tests
ruff check --fix src/pycharter tests

# Check only (same as make lint)
ruff format --check src/pycharter tests
ruff check src/pycharter tests

Pre-commit (recommended) — Ruff runs on each commit via ruff and ruff-format hooks; pre-push runs ./scripts/ci.sh --no-ui for local CI parity. Install hooks as in step 4 above. To run hooks manually: python -m pre_commit run --all-files.

Type Hints¶

All public functions should have type hints:

def validate_data(
    schema: Dict[str, Any],
    data: List[Dict[str, Any]],
    strict: bool = False
) -> List[ValidationResult]:
    """Validate data against schema."""
    ...

Docstrings¶

Use Google-style docstrings:

def process_record(record: Dict[str, Any], options: Options) -> Dict[str, Any]:
    """Process a single record.

    Args:
        record: The input record to process.
        options: Processing options.

    Returns:
        The processed record.

    Raises:
        ValueError: If record is invalid.
    """
    ...

Testing¶

Running Tests¶

# All tests
pytest

# Specific file
pytest tests/test_validator.py

# With coverage
pytest --cov=pycharter --cov-report=html

# Only fast tests
pytest -m "not slow"

Writing Tests¶

Place tests in tests/ directory
Name test files test_*.py
Name test functions test_*
Use fixtures from conftest.py

import pytest
from pycharter import Validator

class TestValidator:
    def test_validate_valid_data(self, sample_schema):
        validator = Validator.from_dict(schema=sample_schema)
        result = validator.validate({"name": "Alice"})
        assert result.is_valid

    def test_validate_invalid_data(self, sample_schema):
        validator = Validator.from_dict(schema=sample_schema)
        result = validator.validate({"name": ""})
        assert not result.is_valid

Documentation¶

Building Docs¶

# Install docs dependencies
pip install pycharter[docs]
# or: pip install mkdocs-material mkdocstrings[python]

# Serve locally (from project root, or set DOCS_ROOT)
pycharter docs serve
# or: mkdocs serve

# Build
pycharter docs build
# or: mkdocs build

Documentation Style¶

Use clear, concise language
Include code examples
Link to related documentation
Keep tutorials beginner-friendly

Documentation policy (single source of truth)¶

Python API: Document in docstrings; MkDocs pulls from the library. New public APIs must have docstrings and a place in docs/api/. Do not duplicate Python API prose on the in-app Documentation page.
REST API: Document via FastAPI route summary/description and models; Swagger is the reference. New endpoints are added in code; the in-app API Playground builds its list from GET /api/v1/docs/playground-routes (OpenAPI). Do not maintain a separate hand-written list of REST endpoints in the UI.
UI: The Documentation page (nav: "API Playground") is a playground and links only: try REST endpoints, link to the full Python API (MkDocs site), and link to Swagger. No long prose about endpoints or Python API on that page.

Submitting Changes¶

Pull Request Process¶

Create a feature branch

git checkout -b feature/my-feature

Make your changes
Write code
Add tests
Update documentation
Run checks

# Format and lint (paths match CI)
make format    # or: ruff format + ruff check --fix
make lint      # or: ruff format --check + ruff check

# Types
make type-check    # or: mypy src/pycharter

# Tests
pytest

Commit with descriptive message

git add .
git commit -m "Add Kafka extractor for streaming data sources

- Implement KafkaExtractor class
- Add consumer group support
- Include comprehensive tests
- Update documentation"

Push and create PR

git push origin feature/my-feature

Then create a Pull Request on GitHub.

PR Guidelines¶

Title: Clear, descriptive summary
Description: Explain what and why
Tests: Include tests for new functionality
Docs: Update relevant documentation
Breaking changes: Clearly mark if applicable

PR Template¶

## Summary

Brief description of changes.

## Changes

- Added X
- Fixed Y
- Updated Z

## Testing

How to test these changes.

## Checklist

- [ ] Tests pass
- [ ] Documentation updated
- [ ] Code formatted
- [ ] No breaking changes (or documented)

Types of Contributions¶

Bug Reports¶

Open an issue with:

PyCharter version
Python version
Steps to reproduce
Expected vs actual behavior
Error messages/stack traces

Feature Requests¶

Open an issue with:

Use case description
Proposed solution
Alternatives considered

Code Contributions¶

Good first issues are labeled good-first-issue.

Areas we'd love help with:

New extractors (Kafka, Snowflake, etc.)
New loaders (BigQuery, Redshift, etc.)
Documentation improvements
Test coverage
Performance optimizations

Documentation¶

Fix typos
Improve explanations
Add examples
Translate documentation

Maintainer Guides¶

Publishing to PyPI – PyPI publishing via GitHub Releases and Trusted Publishing
Release Workflow – Step-by-step release commands and the scripts/release.sh script

Getting Help¶

GitHub Issues: Bug reports, feature requests
GitHub Discussions: Questions, ideas
Email: maintainers@example.com

Recognition¶

Contributors are recognized in:

GitHub Contributors page
Release notes
Documentation credits

Thank you for contributing to PyCharter! 🎉