Skip to content

Commit 19988fd

Browse files
authored
Document MIG slice configuration for NIM models per chat message (#1324)
1 parent 67ac171 commit 19988fd

1 file changed

Lines changed: 30 additions & 0 deletions

File tree

docs/docs/extraction/quickstart-guide.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -425,6 +425,36 @@ You can specify multiple `--profile` options.
425425
| `nemotron-parse` | Advanced | Use [nemotron-parse](https://build.nvidia.com/nvidia/nemotron-parse), which adds state-of-the-art text and table extraction. For more information, refer to [Advanced Visual Parsing](nemoretriever-parse.md). |
426426
| `vlm` | Advanced | Use [llama 3.1 Nemotron 8B Vision](https://build.nvidia.com/nvidia/llama-3.1-nemotron-nano-vl-8b-v1/modelcard) for experimental image captioning of unstructured images. |
427427
428+
## Specifying MIG slices for NIM models in NV-Ingest
429+
430+
When deploying NV-Ingest with NIM models on MIG‑enabled GPUs, MIG device slices are requested and scheduled through the values.yaml file for the corresponding NIM microservice. For IBM CAS deployments, this allows NV-Ingest NIM pods to land only on nodes that expose the desired MIG profiles. [raw.githubusercontent](https://raw.githubusercontent.com/NVIDIA/nv-ingest/main/helm/README.md​)
431+
432+
To target a specific MIG profile (for example, a 3g.20gb slice on an A100) for a given NIM, configure the `resources` and `nodeSelector` under that NIM’s values path in `values.yaml`.
433+
434+
The following example shows the pattern (paths will vary by NIM, such as `nvingest.nvidiaNim`.`nemoretrieverPageElements` instead of the generic `nvingest.nim` placeholder):[catalog.ngc.nvidia](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo-microservices/helm-charts/nv-ingest)​:
435+
436+
437+
```shell
438+
nvingest:
439+
nvidiaNim:
440+
nemoretrieverPageElements:
441+
modelName: "meta/llama3-8b-instruct" # Example NIM model
442+
resources:
443+
limits:
444+
nvidia.com/mig-3g.20gb: 1 # MIG profile resource
445+
requests:
446+
nvidia.com/mig-3g.20gb: 1
447+
nodeSelector:
448+
nvidia.com/gpu.product: A100-SXM4-40GB-MIG-3g.20gb
449+
```
450+
451+
Key points:
452+
453+
* Use the appropriate NIM‑specific values path (for example, `nvingest.nvidiaNim.nemoretrieverPageElements.resources`) rather than the generic `nvingest.nim` placeholder.
454+
* Set `resources.requests` and `resources.limits` to the desired MIG resource name (for example, `nvidia.com/mig-3g.20gb`).
455+
* Use `nodeSelector` (or tolerations/affinity, if preferred) to target nodes labeled with the corresponding MIG‑enabled GPU product (for example, `nvidia.com/gpu.product: A100-SXM4-40GB-MIG-3g.20gb`).
456+
457+
This syntax and structure can be repeated for each NIM model used by CAS, ensuring that each NV-Ingest NIM pod is mapped to the correct MIG slice type and scheduled onto compatible nodes.
428458
429459
!!! important
430460

0 commit comments

Comments
 (0)