Skip to content

Commit 1c8a044

Browse files
committed
Remove memory resources constraint in A4X serving-launcher.yaml and change default number of GPUs in A4 TRTLLM inference recipe
1 parent 65ccbf4 commit 1c8a044

2 files changed

Lines changed: 2 additions & 4 deletions

File tree

inference/a4/single-host-serving/tensorrt-llm/values.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ service:
3737
workload:
3838
model:
3939
name:
40-
gpus: 4
40+
gpus: 8
4141
image:
4242
framework:
4343
configFile: serving-args.yaml

src/helm-charts/a4x/inference-templates/deployment/templates/serving-launcher.yaml

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -278,7 +278,5 @@ spec:
278278
resources:
279279
requests:
280280
nvidia.com/gpu: {{ $gpusPerNode }}
281-
memory: "800Gi"
282281
limits:
283-
nvidia.com/gpu: {{ $gpusPerNode }}
284-
memory: "1000Gi"
282+
nvidia.com/gpu: {{ $gpusPerNode }}

0 commit comments

Comments
 (0)