Skip to content

developer_environment.md

Latest commit

 

History

History
108 lines (73 loc) · 2.95 KB

developer_environment.md

File metadata and controls

108 lines (73 loc) · 2.95 KB

Getting Started: Developer Environment

Clone repo

git clone https://github.com/stackav-oss/conch.git
cd conch

Unless otherwise specified, all commands are intended to be run from the root of the Conch repository.

Configure Environment

Note: we assume that you have Python 3.10+ already installed on your system.

  1. Install direnv.
  2. Enable direnv.
direnv allow .

Direnv is not necessarily required, we primarily use it to manage activation/deactivation of the virtual environment for the project. If you cannot (or do not want to) install direnv, you'll just need to manually activate/deactivate the virtual environment.

Optional: User environment

Extra environment variables can be placed in tools/env/user.sh (not tracked by git). For example:

export ROCM_PATH=/opt/rocm

Install project

To install the project as an editable, clone this repository and run this command from the repo root directory.

pip install -e ".[dev]"

By default this does not install torch or triton. You can specify an extra for your platform (either cuda or rocm) to install the appropriate versions of those packages for your accelerator. For ROCm/AMD support, you'll need to add --extra-index-url https://download.pytorch.org/whl/rocm6.2.4.

pip install -e ".[dev, cuda]"
pip install -e ".[dev, rocm]" --extra-index-url https://download.pytorch.org/whl/rocm6.2.4

Testing

After installation, to run the Triton kernel tests, execute this command:

pytest

You can also specify the path to a specific test file to run one test individually, for example:

pytest tests/copy_blocks_test.py

If you have other pytest installations in your $PATH, you can also run pytest via:

python -m pytest

Benchmarks

To benchmark all Triton kernels, execute this script:

./tools/benchmarks/run_all.sh

You can also run benchmarks individually, for example:

python benchmarks/paged_attention_benchmark.py

Optional: Benchmarking against vLLM

Most unit tests/benchmarks allow comparison to CUDA implementations of operations from vLLM (rather than PyTorch-reference implementations). In order to use them, you can install vLLM (pip install vllm) and set the environment variable CONCH_ENABLE_VLLM=1.

pip install vllm==0.9.1
CONCH_ENABLE_VLLM=1 python benchmarks/paged_attention_benchmark.py

Optional: Benchmarking against Torchvision

Some unit tests/benchmarks allow comparison to CUDA implementations of operations from Torchvision (rather than PyTorch-reference implementations). In order to use them, you can install Torchvision (pip install torchvision) and set the environment variable CONCH_ENABLE_TORCHVISION=1.

# Note: add `--extra-index-url https://download.pytorch.org/whl/rocm6.2.4` on ROCm
pip install torchvision==0.22
CONCH_ENABLE_TORCHVISION=1 python benchmarks/nms_benchmark.py