Support LLM api of the new trtllm (0.20.0)#26
Conversation
Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
There was a problem hiding this comment.
nit: All examples in trtllm are python script. So "example_notebook" dir name can be misleading. But up to you on how LP zoo would like to reorganize examples.
There was a problem hiding this comment.
I can rename the folder to "examples"?
There was a problem hiding this comment.
Yes I think that'd be better. Can you make the change?
There was a problem hiding this comment.
General note - although in TRTLLM we have LP test w a dummy and GenLen LP, you might want to add more LP-specific tests in this repo to avoid future TRTLLM breaks any nuances in the LPs here.
There was a problem hiding this comment.
Ideally, we could have unit tests for them but it is not convenient for such task.
Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
…-processor-zoo into support-trtllm-200
hchings
left a comment
hchings
left a comment
There was a problem hiding this comment.
LGTM. I didn't run this MR locally to verify the scripts but I believe you've done that.
Thanks for the changes!
Signed-off-by: aerdem4 <ahmeterd4@gmail.com>
2 participants
No description provided.