Skip to content

Releases: data-science-extensions/toolbox-python

v1.6.0 - Standardise Type Safety, Simplify Overloads, and Enhance Documentation

02 Jan 22:54

Choose a tag to compare

📋 Summary

Standardise the type hinting system across the entire library, transitioning from custom type aliases to native Python typing constructs. Simplify complex function overloads in the checkers.py and output.py modules, improve the robustness of type checking logic, and expand the documentation for core utility classes.

🛡️ Type Safety & Standardisation

  • Native Typing: Replace custom type aliases like any_collection, str_collection, and scalar with standard library types such as Collection[Any], Collection[str], and Any.
  • Explicit Callables: Add explicit Callable[..., bool] and Callable[..., None] type annotations to function aliases in the checkers.py module for improved static analysis.
  • Generic Collections: Refactor internal logic to use Collection[type] instead of specific list or tuple types, allowing for more flexible input handling.
  • List Standardisation: Update all __all__ export lists and internal string lists to use the native list[str] type.

⚙️ Logic & Overload Simplification

  • Overload Consolidation: Reduce the number of @overload definitions for is_all_values_of_type(), assert_value_of_type(), and list_columns() by using more general collection types.
  • Robust Type Checking: Improve the is_value_of_type() function to correctly handle non-type iterables by using a more direct not isinstance(check_type, type) check.
  • Constant Relocation: Move the log_levels Literal from collection_types.py to output.py to better align with its primary usage in logging utilities.
  • Flattening Logic: Refine the flatten() function to use Collection[Collection[Any]] for input parameters, removing the need for internal type ignores.

📖 Documentation & DX

  • Class Methods Documentation: Update the Defaults() class docstring to include a dedicated "Methods" section, documenting the .get(), ._validate_value_and_default(), and ._validate_type() methods.
  • Docstring Accuracy: Refactor parameter descriptions across multiple modules to reflect the shift from "tuple of types" to "Sequence of types" or "Collection of types".
  • Retry Metadata: Add descriptive docstrings for internal type aliases and generic type variables in the retry.py module.

🛠️ Bug Fixes & Refinements

  • Attribute Errors: Fix a missing .__name__ attribute access in error message generation for type assertions.
  • Typo Remediation: Correct several typos in docstrings and internal variable names across the checkers.py and bools.py modules.
  • Dictionary Reversal: Update dict_reverse_keys_and_values() to return explicit dict[str, Any] types, ensuring better compatibility with downstream consumers.

💪 What's Changed

  • Standardise Type Hints, Simplify Overloads, and Refactor Core Utilities by @chrimaho in #39

Full Changelog: v1.5.0...v1.6.0

v1.5.0 - Modernise Infrastructure, Refactor Core Logic, and Introduce Numeric Validation

28 Dec 10:54

Choose a tag to compare

📋 Summary

Modernise the development infrastructure, refactor core logic for better maintainability, and expand the utility suite with new validation capabilities. Transition the project to uv for package management, introduce code complexity analysis, and refactor the retry() decorator into a robust class-based implementation.

🚀 Infrastructure Modernisation

  • Package Management: Standardise on uv for all package management, environment synchronisation, and script execution tasks.
  • Dependency Consolidation: Remove the requirements/ directory and Makefile in preference for a unified pyproject.toml configuration.
  • Python 3.14 Support: Update CI/CD workflows to include Python 3.14 in the test matrix, ensuring forward compatibility.
  • Windows Compatibility: Fix UnicodeEncodeError in CI by forcing UTF-8 encoding for log output on Windows runners.

🛠️ Core Refactoring & Quality

  • Retry Logic: Refactor the retry() decorator into a helper _Retry() class, reducing cyclomatic complexity from 17 to 4 and improving modularity.
  • Metadata Management: Replace hardcoded version strings with dynamic retrieval via the metadata() function, simplifying the release process.
  • Type Checking: Replace mypy with ty for faster and more consistent type checking across the codebase.
  • Complexity Analysis: Integrate complexipy to automate code complexity monitoring and enforce maintainability standards.

🧪 New Validation Utilities

  • Validators() Class: Introduce the Validators() class to centralise logic for numeric range validation.
  • Range Checks: Implement .value_is_between() and .all_values_are_between() methods for flexible boundary checking.
  • Assertion Support: Provide .assert_value_is_between() and .assert_all_values_are_between() methods to raise descriptive AssertionError() class exceptions.

📖 Documentation & DX

  • README Overhaul: Update README.md with modernised setup guides, uv workflows, and expanded quality assurance documentation.
  • Docstring Standardisation: Migrate docstring formatting checks to a pre-commit hook using docstring-format-checker.
  • Utility Scripts: Refactor src/utils/scripts.py to use dynamic package constants and improve subprocess handling.

🔍 Technical Improvements

  • Type Hint Enhancements: Standardise type annotations using Optional, Union, and Literal for better compatibility and clarity.
  • Pylint Remediation: Resolve numerous Pylint warnings and suppress specific flags where appropriate to maintain a 10/10 quality score.
  • Exception Handling: Standardise exception storage and update return type hints (e.g., NoReturn) for better static analysis.

💪 What's Changed

  • Modernise Infrastructure, Refactor Core Logic, and Introduce Numeric Validation by @chrimaho in #38

Full Changelog: v1.4.1...v1.5.0

v1.4.1 - Documentation Infrastructure Enhancement and Automated Changelog Generation

03 Aug 00:13

Choose a tag to compare

📋 Summary

Version 1.4.1 focuses on enhancing the project's documentation infrastructure and establishing automated changelog generation capabilities. This release introduces comprehensive contribution guidelines, automated changelog workflows, and improved documentation styling to streamline the development process and enhance the contributor experience.

🔄 Automated Changelog System

  • CI/CD Integration: Automated changelog generation integrated into the continuous deployment pipeline using GitHub Actions
  • Environment Variables: Added support for GITHUB_TOKEN and REPOSITORY_NAME environment variables for secure API access
  • CLI Command: Introduced generate-changelog command for manual changelog creation and debugging purposes
  • Version Control: Automatic commit of changelog updates with version-specific commit messages and skip CI flags

📚 Contribution Framework

  • Guidelines Documentation: Created comprehensive CONTRIBUTING.md with detailed procedures for issue reporting, branch management, coding standards, and review processes
  • Documentation Integration: Added contribution guidelines to the documentation site at docs/usage/contributing.md for improved discoverability
  • Developer Onboarding: Established clear expectations and workflows to reduce contributor onboarding time

📁 Documentation Site Enhancements

🎨 Visual Improvements

  • Custom Styling: Implemented shortcode CSS system with badge support for enhanced visual consistency across documentation
  • Navigation Structure: Improved site navigation with better organisation and user experience
  • MkDocs Configuration: Enhanced site configuration with updated plugin management and table of contents depth limiting

📖 Content Organisation

  • Changelog Integration: Added dedicated changelog page (docs/usage/changelog.md) within the documentation site
  • Reference Architecture: Established standardised documentation structure for consistency across all pages

🧐 Technical Implementation

🔧 Workflow Automation

  • Release Process: Enhanced cd.yml workflow with automated changelog generation steps
  • Error Handling: Improved workflow reliability with proper dependency management and error handling
  • Version Management: Automated tracking and updating of version information throughout the release process

📦 Package Configuration

  • CLI Scripts: Updated pyproject.toml with new CLI commands for changelog generation
  • Build System: Maintained compatibility with existing build processes whilst adding new automation capabilities

🔍 Code Quality & Infrastructure

🧪 Maintenance Scripts

  • Changelog Utilities: Enhanced src/utils/changelog.py with improved formatting, error handling, and environment variable support
  • Release Automation: Streamlined release note generation with better readability and maintainability
  • Git Integration: Improved git history processing for structured changelog creation

📊 Documentation Standards

  • Formatting Consistency: Standardised formatting across all documentation files for improved readability
  • Content Structure: Established clear patterns for documentation organisation and presentation

📈 Developer Experience Improvements

🤝 Contributor Benefits

  • Clear Guidelines: Comprehensive contribution documentation reduces confusion and speeds up onboarding
  • Automated Workflows: Changelog generation eliminates manual documentation overhead for maintainers
  • Consistent Standards: Established coding and documentation standards improve code quality and review efficiency

🔧 Maintainer Benefits

  • Reduced Manual Work: Automated changelog generation significantly reduces maintenance overhead
  • Quality Assurance: Clear contribution guidelines ensure consistent quality across contributions
  • Streamlined Processes: Automated workflows improve release reliability and reduce human error

🏗️ Infrastructure Foundation

This release establishes a robust foundation for project documentation and automation. The automated changelog system ensures accurate version history tracking, whilst the comprehensive contribution guidelines provide clear expectations for all project participants.

The enhanced documentation infrastructure supports better collaboration and reduces the barrier to entry for new contributors, fostering a more inclusive and efficient development environment.

💪 What's Changed

  • Documentation Infrastructure Enhancement and Automated Changelog Generation by @chrimaho in #37

Full Changelog: v1.4.0...v1.4.1

v1.4.0 - Comprehensive Package Enhancement and Tooling Migration

13 Jul 05:02

Choose a tag to compare

📋 Overview

This release introduces significant enhancements to the toolbox-python package, including a new generators module, expanded validation capabilities, comprehensive docstring standards, and modernized CI/CD workflows. The changes span 30 commits across multiple areas of the codebase to improve functionality, maintainability, and developer experience while maintaining full backward compatibility.

🆕 New Modules and Core Functionality

Generators Module

  • New module: toolbox_python.generators - Provides functions for computing data on-the-fly based on input parameters rather than storing it in databases or files
  • New function: generate_group_cutoffs() - Divides a total number of items into specified groups, returning start and end indices for each group with proper validation and error handling
  • Documentation: Added comprehensive documentation with examples and complete test coverage

Enhanced Checkers Module

  • New constant: OPERATORS dictionary - Defines comparison operations for value validation including <, <=, >, >=, ==, !=, in, and not in
  • New function: is_valid_value() - Checks if a value is valid based on a specified operator and target value
  • New function: assert_is_valid_value() - Assert version of the validation function with proper error handling
  • Enhanced aliases: Added is_valid and assert_is_valid function aliases for consistency and convenience

Enhanced Strings Module

  • New function: str_to_list() - Converts strings to single-element lists while preserving other data types unchanged
  • Type safety: Includes comprehensive overloads and type hints for better development experience
  • Input flexibility: Provides a convenient way to normalize string inputs for functions that expect list-like objects

🔧 Function Enhancements and Type Safety

Improved Type Hints and Overloads

  • Enhanced overloads: Added @overload decorators for better type hints in:
    • toolbox_python.checkers.is_value_of_type() - Now properly handles both single types and tuple of types
    • toolbox_python.output.list_columns() - Distinguishes between print and return modes
    • toolbox_python.retry.retry() - Better handling of different parameter combinations
  • Type validation: Enhanced parameter validation across multiple functions using new assertion functions

Function Robustness Improvements

  • Parameter validation: Added comprehensive input validation using assert_is_valid functions to ensure type safety and value constraints
  • Error handling: Improved exception handling for the retry() function with better logging initialization and normalized exception parameter handling
  • Code structure: Better organization with descriptive comments separating validation, preparation, processing, and output phases
  • Column width logic: Simplified logic in list_columns() by replacing conditional assignment with min() function for cleaner code

📜 Package Scripts and Automation

Comprehensive Utility Scripts

  • Script module: src/utils/scripts.py - Extensive collection of utility functions organized into categories:
    • Syncing: uv_sync() for dependency management
    • Linting: run_black(), run_isort(), run_pycln(), run_pyupgrade(), run_blacken_docs(), and combined lint()
    • Checking: check_black(), check_mypy(), check_pytest(), check_codespell(), check_pylint(), and comprehensive check()
    • Git operations: add_git_credentials(), git_refresh_current_branch(), git_switch_to_main_branch(), git_add_coverage_report()
    • Documentation: docs_build_static(), docs_serve_versioned(), build_versioned_docs(), and version management

Project Scripts Configuration

  • Package scripts: Added extensive [project.scripts] table in pyproject.toml with organized sections:
    • Syncing: sync
    • Linting: run-black, run-isort, lint, lint-check
    • Checking: check-black, check-mypy, check-pytest, check-docstrings, check
    • Git: add-git-credentials, git-switch-to-main-branch, bump-version, git-update-version
    • Docs: docs-serve-static, build-versioned-docs, docs-check-versions
  • Enhanced version management: Updated src/utils/bump_version.py for better CLI compatibility and version tracking

📚 Docstring Standards and Validation

Automated Docstring Validation System

  • Validation infrastructure: Implemented comprehensive docstring checking with strict formatting requirements:
    • FunctionAndClassDetails() - Named tuple for tracking function and class details
    • check_docstrings_file() - Validates docstrings in individual files with detailed error reporting
    • _check_single_docstring() - Validates individual function/class docstrings against standards
    • _check_section_order() - Ensures proper section ordering (Summary → Params → Returns/Yields → Examples)
    • _validate_section_formats() - Validates specific section formats and content structure
    • check_docstrings_cli(), check_docstrings_all(), check_docstrings_dir() - Various checking interfaces

Enhanced Docstring Standards

  • Mandatory sections: Summary, Params, Returns/Yields, and Examples sections now required for all functions and classes
  • Format consistency: Fixed typos across all docstrings from !!! summary "Summary" to !!! note "Summary"
  • Examples improvements: Updated all code examples to use pycon syntax for better clarity and consistency
  • Type information: Enhanced Raises statements with proper exception types (e.g., TypeError, ValueError, LookupError) and detailed descriptions
  • Documentation quality: Improved parameter descriptions, return value specifications, and comprehensive usage examples

🚀 CI/CD Workflow Modernization

Migration to UV-Based Tooling

  • CI workflow: Complete refactoring of .github/workflows/ci.yml:
    • Replaced all make commands with uv run commands for improved consistency and performance
    • Added UV environment variables for optimal configuration
    • Streamlined dependency installation and checking processes
  • CD workflow: Comprehensive modernization of .github/workflows/cd.yml:
    • Replaced make commands with uv run commands throughout all jobs
    • Added environment variables for UV configuration (UV_LINK_MODE, UV_NATIVE_TLS, UV_NO_SYNC)
    • Improved git operations with new utility functions
    • Enhanced version management and tag handling
    • Fixed tag reference issues to ensure correct version numbers in releases
    • Streamlined package building and publishing processes

Enhanced Pre-commit Integration

  • Hook updates: Updated pre-commit hook versions for better compatibility:
    • Updated mypy from v1.15.0 to v1.16.1 with additional --allow-redefinition flag
    • Updated pyupgrade from v3.19.1 to v3.20.0
    • Updated uv-pre-commit from 0.6.12 to 0.7.20
    • Removed outdated poetry check hooks
  • New validation: Added check-docstrings hook for continuous validation during development workflow

⚙️ Configuration and Build System Updates

Project Configuration Enhancements

  • Build system: Migrated from hatchling to uv_build for better integration with UV toolchain and improved build reliability
  • MyPy configuration: Updated to include no-redef in disable_error_code list for better type checking compatibility
  • Dependencies: Added uv to the development dependency group for comprehensive toolchain integration
  • Python requirements: Updated syntax from ">3.9,<4.0" to ">=3.9,<4.0" for standard compliance

Documentation Configuration

  • MkDocs updates:
    • Added new generators module to navigation structure for complete module coverage
    • Fixed repository icon from material/gitlab to material/github for accurate branding
    • Enhanced navigation structure to reflect all available modules
  • Improved organization: Better categorization and presentation of module documentation

Enhanced Type System

  • Collection types: Expanded collection_types.py with new type aliases:
    • Added datetime collections: datetime_list, datetime_tuple, datetime_set, datetime_list_tuple
    • Added int collections: int_set, int_list_tuple
    • Enhanced dict collections: reorganized and added better documentation
    • Improved organization with clear section comments for better maintainability

🧪 Testing and Quality Assurance

Comprehensive Test Coverage

  • New test modules: Added extensive test coverage for new functionality:
    • src/tests/test_generators.py - Complete test suite for the generators module with edge cases
    • Enhanced src/tests/test_strings.py - Tests for new str_to_list() function with various input types
    • Expanded src/tests/test_checkers.py - Comprehensive tests for new validation functions including OPERATORS testing
  • Test improvements: Updated existing tests to use pytest.raises instead of pytest imports for better compatibility
  • Coverage maintenance: Maintained 100% test coverage across all modules while adding new functionality

Quality Improvements

  • Validation robustness: Enhanced function robustness through comprehensive parameter validation using new assertion functions
  • Error handling: Improved error capture and reporting in docstring checking with detailed feedback
  • Type safety: Better type hints and validation across the codebase with overloads and proper type checking
  • Code organization: Improved code structure with better separation of concerns and descriptive comments

🎯 Impact and Benefits

This release significan...

Read more

v1.3.2 - Update package config

25 May 21:44
8fc37d6

Choose a tag to compare

This release updates the pyproject.toml file to enhance metadata clarity and correct the Python version requirement syntax. The most important changes include reorganizing and expanding the classifiers section and adjusting the requires-python field.

What's Changed

  • Reorder classifiers and update Python requirement syntax in pyproject.toml by @chrimaho in #35

Metadata improvements:

  • Reorganized and expanded the classifiers section to include additional topics such as Testing :: Unit and Utilities, improving the discoverability and categorization of the project.

Syntax correction:

  • Updated the requires-python field from ">3.9,<4.0" to ">=3.9,<4.0" to align with standard version specification syntax.

Full Changelog: v1.3.1...v1.3.2

v1.3.1 - Add more collection types

26 Apr 08:28
11939a9

Choose a tag to compare

What's Changed

Full Changelog: v1.2.1...v1.3.1

v1.2.1 - Add new `@class_property` decorator

25 Apr 01:03
c5deb70

Choose a tag to compare

What's Changed

Full Changelog: v1.2.0...v1.2.1

v1.2.0 - Migrate from Poetry to UV and add new DotDict class

06 Apr 07:38
2b5386f

Choose a tag to compare

Primary Changes

  • Migrate build engine from Poetry to UV
  • Add new DotDict class to handle accessing dictionary keys using .attribute syntax
  • Add utility module bump_version for checking pyproject.toml and updating the version values across the respective files

Full Changelog: v1.1.0...v1.2.0

v1.1.0 - Enhance the `output` module

24 Feb 10:33
fef1134

Choose a tag to compare

What's Changed

  • Update list_columns() function:
    • Allow the obj input to accept different input types
    • Extend the Unit Tests
  • Update pyproject.toml file:
    • To match the structure of the latest poetry version
    • To fix the location of where the __version__ attribute is stored

Full Changelog: v1.0.4...v1.1.0

v1.0.4

13 Dec 07:54

Choose a tag to compare

What's Changed

  • Fix type definitions across all modules.

Full Changelog: v1.0.3...v1.0.4