Skip to content

Commit 057f8f4

Browse files
grahamkinggrahamking
authored
feat: TensorRT-LLM engine (NVIDIA#317)
Engine, `tio` support and docs. Proof of concept / experimental.
1 parent 11a3665 commit 057f8f4

46 files changed

Lines changed: 4160 additions & 13 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎applications/llm/count/Cargo.lock‎

Lines changed: 76 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎examples/rust/Cargo.lock‎

Lines changed: 76 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎launch/tio/Cargo.lock‎

Lines changed: 25 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎launch/tio/Cargo.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ license = "Apache-2.0"
2525
mistralrs = ["triton-distributed-llm/mistralrs"]
2626
sglang = ["triton-distributed-llm/sglang", "dep:netlink-packet-route", "dep:rtnetlink"]
2727
llamacpp = ["triton-distributed-llm/llamacpp"]
28+
trtllm = ["triton-distributed-llm/trtllm"]
2829
cuda = ["triton-distributed-llm/cuda"]
2930
metal = ["triton-distributed-llm/metal"]
3031
vllm = ["triton-distributed-llm/vllm"]

0 commit comments

Comments
 (0)