Skip to content

Commit efeac27

Browse files
lancellyGitLab CI Bot
authored andcommitted
[None][perf] executor: avoid deepcopy of prompt_token_ids on enqueue (NVIDIA#14895)
Signed-off-by: Lanyu Liao <lancelly@users.noreply.github.com> Co-authored-by: Lanyu Liao <lancelly@users.noreply.github.com> Signed-off-by: GitLab CI Bot <gitlab-ci@nvidia.com>
1 parent f92598a commit efeac27

1 file changed

Lines changed: 1 addition & 2 deletions

File tree

tensorrt_llm/executor/base_worker.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@
1212
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1313
# See the License for the specific language governing permissions and
1414
# limitations under the License.
15-
import copy
1615
import datetime
1716
import enum
1817
import gc
@@ -449,7 +448,7 @@ def _enqueue_request(self,
449448
else:
450449
lora_config = None
451450

452-
prompt_token_ids = copy.deepcopy(request.prompt_token_ids)
451+
prompt_token_ids = list(request.prompt_token_ids)
453452
prompt_tuning_config = None
454453
if request.prompt_adapter_request is not None:
455454
self._load_prompt_adapter(request.prompt_adapter_request)

0 commit comments

Comments
 (0)