[ABI Dependency] Best-Effort support for pip

[DEKHTIARJonathan]

Functional and "best effort" implementation for pip.

Should work just fine.

Implementation details

• Can not be a plugin because it needs to break plugin venv isolation.
• User controllable using export VARIANT_ABI_DEPENDENCY=packageA==1.2.3,...,packageZ==7.8.9
• Should support perfectly "sequential installs"

Should support these usecases

pip install torch
# `torch` is pre-installed and automatically fed by `variantlib`
pip install vllm flash-attn

export VARIANT_ABI_DEPENDENCY="torch==2.8.0"
pip install ${VARIANT_ABI_DEPENDENCY} vllm flash-attn

export VARIANT_ABI_DEPENDENCY="torch==2.8.0"
pip install vllm flash-attn

export VARIANT_ABI_DEPENDENCY="torch==2.8.0"
# Assuming the `requirements.txt` file does not contain torch - or pin to `==2.8.0` 
pip install -r requirements.txt

Versioning

The interface supports "sort of range support" and not exclusively major.minor.micro pinning

• export VARIANT_ABI_DEPENDENCY="torch==2.8.0" => only 2.8.0 torch release
• export VARIANT_ABI_DEPENDENCY="torch==2.8" => only >=2.8,<2.9 torch releases
• export VARIANT_ABI_DEPENDENCY="torch==2" => only >=2,<3 torch releases

Which allows developers for different package to express different level of compatibility.

It should be noted that the package maintainers can also combine abi_dependency tags to express an OR match:

• abi_depedency :: torch :: 2.8.0 => exact 2.8.0 match
• abi_depedency :: torch :: 2.9.0 => exact 2.9.0 match
• abi_depedency :: torch :: 3 => >=3,<4 match

=> Any version of torch that matches one of the above will be compatible.

[konstin]

LGTM