Skip to content

README.md

Latest commit

 

History

History
79 lines (55 loc) · 2.37 KB

README.md

File metadata and controls

79 lines (55 loc) · 2.37 KB

Tests

This directory contains tests for the torchtitan project, including unit tests and integration tests.

Test Structure

  • unit_tests/: Contains unit tests for individual components
  • integration_tests/: Contains integration tests that test multiple components together
    • base_config.toml: Base configuration file for integration tests
    • features.py: Tests for torchtitan features and composability
    • ft.py: Fault-tolerance integration tests
    • h100.py: Tests cases for H100 GPUs
    • models.py: Tests for specific model architectures and configurations
  • assets/: Contains test assets and fixtures used by the tests
    • tokenizer/: Tokenizer configuration and vocabulary files for testing
    • custom_schedule.csv: Custom PP schedule for testing

Running Tests

Prerequisites

Ensure you have all development dependencies installed:

pip install -r requirements-dev.txt
pip install -r requirements.txt

Running Integration Tests

To run the integration tests:

python -m tests.integration_tests.run_tests <output_dir> [--config_path CONFIG_PATH] [--test_suite TEST_SUITE] [--test_name TEST_NAME] [--ngpu NGPU]

Arguments:

  • output_dir: (Required) Directory where test outputs will be stored
  • --test_suite: (Optional) Specific test suite to run by name (default: "features")
  • --config_path: (Optional) Path to the base config file (default: "./tests/integration_tests/base_config.toml")
  • --test_name: (Optional) Specific test to run by name (default: "all")
  • --ngpu: (Optional) Number of GPUs to use for testing (default: 8)

Examples:

# Run all model integration tests with 8 GPUs
python -m tests.integration_tests.run_tests test_output

# Run only core functionality tests for features
python -m tests.integration_tests.run_tests test_output --test_suite features

# Run a specific test with 2 GPUs
python -m tests.integration_tests.run_tests test_output --test_suite features --test_name gradient_accumulation --ngpu 2

Running Unit Tests

To run only the unit tests:

pytest -s tests/unit_tests/

Running Specific Unit Test Files

To run a specific test file:

pytest -s tests/unit_tests/test_job_config.py

Running Specific Test Functions in Unit Tests

To run a specific test function:

pytest -s tests/unit_tests/test_job_config.py::TestJobConfig::test_command_line_args