Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
372 lines (259 loc) · 10.3 KB

CONTRIBUTING.md

File metadata and controls

372 lines (259 loc) · 10.3 KB
Raw

Contributing to pydgraph

Thank you for your interest in contributing to pydgraph! We welcome contributions from the community.

Table of Contents

Getting Started

  1. Fork the repository on GitHub

  2. Clone your fork locally:

    git clone https://github.com/YOUR-USERNAME/pydgraph.git
cd pydgraph

  3. Add the upstream repository:

    git remote add upstream https://github.com/dgraph-io/pydgraph.git

Development Setup

Prerequisites

  • Python 3.13+ (for development; Python 3.9+ for using the library)
  • Docker and Docker Compose (for running tests)
  • Git

Makefile Commands

This project uses a Makefile to simplify common development tasks. To see all available commands:

make help

Output:

Environment Variables:
  INSTALL_MISSING_TOOLS=true    Enable automatic installation of missing tools (default: disabled)

Available targets:
help            Show this help message
setup           Setup project (install tools and sync dependencies)
sync            Sets up and syncs project virtual environment.
check           Run pre-commit hooks on all files
protogen        Regenerate protobuf files (requires Python 3.13+)
clean           Cleans build artifacts
build           Builds release package
test            Run tests
publish         Publish a new release to PyPi (requires UV_PUBLISH_USERNAME and UV_PUBLISH_PASSWORD to be set)
deps            Check/install tool dependencies (set INSTALL_MISSING_TOOLS=true to auto-install)
deps-docker     Check and install Docker if needed (requires Docker 20.10.0+)

Setting Up Your Environment

  1. Set up the project:

    make setup

    This will:

    • Check for required tools (uv, trunk, docker)
    • Install pre-commit hooks
    • Set up the correct Python version
    • Create and configure a virtual environment
    • Install all project dependencies

    Note: To automatically install any missing tool dependencies (uv, trunk, docker), you can set INSTALL_MISSING_TOOLS to true:

    INSTALL_MISSING_TOOLS=true make setup

  2. Verify your setup:

    make check

Syncing Dependencies

After making changes to dependencies or pulling updates:

make sync

This syncs the project virtual environment with the latest dependencies.

Regenerating Protocol Buffers

If you make changes to pydgraph/proto/api.proto, regenerate the source files:

make protogen

Or directly with uv:

uv run python scripts/protogen.py

Important: This project uses Python 3.13+ with grpcio-tools 1.66.2+ as the canonical development environment. The generated proto files include mypy type stubs for better type checking. The script will verify you have the correct Python and grpcio-tools versions before generating files.

About grpcio Version Requirements

This project requires grpcio 1.65.0 or higher. Older versions have practical limitations:

  • Compilation failures: grpcio versions older than ~1.60.0 fail to compile from source on modern systems (macOS with recent Xcode, newer Linux distributions) due to C++ compiler compatibility issues and outdated build configurations.
  • No pre-built wheels: PyPI doesn't provide pre-built wheels for very old grpcio versions on modern Python versions (3.11+), forcing compilation from source.
  • Build tool incompatibility: The build process for older grpcio versions uses deprecated compiler flags and build patterns that modern toolchains reject.

Making Changes

  1. Create a new branch for your changes:

    git checkout -b your-feature-name

  2. Make your changes following our Code Style and Standards

  3. Add tests for your changes (see Testing)

  4. Ensure all checks pass:

    make check test

    Important: Before opening a pull request, make sure make check test succeeds locally.

  5. Commit your changes using Conventional Commits format:

    git commit -m "feat: add new feature"
git commit -m "fix: resolve issue with..."
git commit -m "docs: update README"

Code Style and Standards

Python Code Standards

  • Follow PEP 8 style guidelines (enforced by ruff)
  • Use type hints for all function signatures
  • Add docstrings for public APIs
  • Maximum line length: handled by the formatter

File Headers

Every new Python file must start with SPDX headers followed by proper attribution:

# SPDX-FileCopyrightText: © 2017-2026 Istari Digital, Inc.
# SPDX-License-Identifier: Apache-2.0
"""
Module description here.
"""

__author__ = "Your Name"
__maintainer__ = "Istari Digital, Inc."

Requirements:

  • SPDX Headers: All new Python files must start with the two SPDX comment lines shown above
  • __author__: Set to your name (the contributor's name)
  • __maintainer__: Always set to "Istari Digital, Inc."

Code Quality Tools

This project uses several tools to maintain code quality:

  • ruff: Linting and code formatting
  • mypy: Static type checking
  • trunk: Additional code quality checks
  • pre-commit: Automated checks on commit

Run all checks:

make check

Testing

Running Tests

Run the full test suite:

make test

Run specific tests:

make test PYTEST_ARGS="-v tests/test_connect.py::TestOpen"

Run a single test:

make test PYTEST_ARGS="-v tests/test_connect.py::TestOpen::test_connection_with_auth"

Stress Tests

The project includes comprehensive stress tests that verify concurrent operations, transaction conflicts, deadlock prevention, and retry mechanisms for both sync and async clients.

Quick mode (default, ~12 seconds) - 20 workers, 50 ops, 10 iterations:

make test PYTEST_ARGS="tests/test_stress_sync.py tests/test_stress_async.py -v"

Moderate mode (10x quick, includes movie dataset, ~60+ seconds) - 200 workers, 500 ops, 100 iterations:

make test STRESS_TEST_MODE=moderate PYTEST_ARGS="tests/test_stress_sync.py tests/test_stress_async.py -v"

Full mode (10x moderate, maximum stress, ~10+ minutes) - 2000 workers, 5000 ops, 1000 iterations:

make test STRESS_TEST_MODE=full PYTEST_ARGS="tests/test_stress_sync.py tests/test_stress_async.py -v"

The stress tests cover:

  • Sync tests: Run with ThreadPoolExecutor to test concurrent operations
  • Async tests: Use pure asyncio.gather() concurrency (no concurrent.futures mixing)
  • Retry utilities: Tests for retry_async(), with_retry_async(), and run_transaction_async()
  • Deadlock regression: Validates the asyncio.Lock deadlock fix from PR #293

Test Infrastructure

The test script requires Docker and Docker Compose to be installed on your machine.

The script will:

  • Automatically bring up a Dgraph cluster
  • Connect to randomly selected ports for HTTP and gRPC to prevent interference with clusters running on default ports
  • Run the tests
  • Bring down the cluster after tests complete

For Docker installation instructions, refer to the official Docker documentation.

Writing Tests

  • Add tests for all new features
  • Add regression tests for bug fixes
  • Tests should be clear, concise, and well-documented
  • Use descriptive test names that explain what is being tested

Test Requirements

Tests are automatically run in CI/CD against:

  • Python versions: 3.9, 3.10, 3.11, 3.12, 3.13, 3.14
  • Dgraph latest release
  • Dgraph HEAD (main branch)
  • Protobuf versions: default (6.x) for all Python versions, plus 4.x and 5.x on Python 3.9 and 3.14

The CI matrix tests all Python versions with the default protobuf, and additionally tests older protobuf versions (4.x, 5.x) on the edge Python versions (3.9 and 3.14) to ensure backwards compatibility without excessive CI runtime.

Submitting a Pull Request

  1. Push your changes to your fork:

    git push origin your-feature-name

  2. Open a pull request on GitHub

  3. Fill out the pull request template (see PULL_REQUEST_TEMPLATE.md)

Pull Request Requirements

  • Title: Must follow Conventional Commits format
    • Examples: feat: add connection pooling, fix: resolve memory leak, docs: update API reference
  • Description: Clearly explain what the PR does and why
  • Checklist: Complete all applicable items in the PR template
  • Tests: All tests must pass (make check test succeeds)
  • Code Quality: All linting and type checking must pass
  • Documentation: Update docs if adding/changing public APIs

Commit Message Guidelines

Follow the Conventional Commits specification:

  • feat: - New features
  • fix: - Bug fixes
  • docs: - Documentation changes
  • style: - Code style changes (formatting, etc.)
  • refactor: - Code refactoring
  • test: - Adding or updating tests
  • chore: - Maintenance tasks
  • ci: - CI/CD changes

Examples:

feat: add async client support
fix: resolve connection timeout issue
docs: update installation instructions
refactor: simplify error handling
test: add integration tests for ACL

Review Process

  1. Maintainers will review your PR
  2. Address any feedback or requested changes
  3. Once approved, a maintainer will merge your PR

Code of Conduct

This project follows the Contributor Covenant Code of Conduct. Please read our Code of Conduct to understand the standards we expect from all contributors and community members.

Questions or Need Help?

Additional Resources

Thank you for contributing to pydgraph! 🎉