Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/docs/extraction/audio-video.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,8 @@ Instead of running the pipeline locally, you can call Parakeet through [build.nv

2. Run inference from Python with the hosted gRPC endpoint and credentials from that page (the example below uses the default hosted gRPC hostname; confirm values in the **Get API Key** flow for your deployment). Pass hosted endpoint, function ID, and API key through `ASRParams` (`audio_endpoints`, `function_id`, `auth_token`).

For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md).

```python
from nemo_retriever import create_ingestor
from nemo_retriever.common.params.models import ASRParams
Expand Down
10 changes: 5 additions & 5 deletions docs/docs/extraction/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,19 @@

These terms appear throughout NeMo Retriever Library documentation.

## Job
## Job { #job }

An **ingestion job** is a unit of work you run on input content (documents, audio, video, and other supported types). You submit jobs through the **ingestor Python API** (for example `Ingestor` task chains such as `.extract(...)`) or the **`retriever ingest` CLI**—not by posting a standalone JSON job document. Default tasks target strong recall; customize behavior with task keyword arguments (including chunking and splitting on `.extract()`) or custom UDF-style operations ([NeMo Retriever graph](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph)). Results are structured metadata and annotations (Ray Dataset, pandas `DataFrame`, or similar).

## Pipeline and tasks
## Pipeline and tasks { #pipeline-and-tasks }

NeMo Retriever Library does **not** run one static pipeline on every document. You configure **tasks** such as parsing, chunking, embedding, storage, and filtering per job. Related topics: [Extending/Customizing NeMo Retriever Library with custom code](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph).

## Extraction metadata
## Extraction metadata { #extraction-metadata }

Output is a **Ray Dataset** (Ray Data) or **pandas** `DataFrame` listing extracted objects (text regions, tables, images, and so on), processing notes, and timing or trace data. Field-level detail is in the [metadata reference](content-metadata.md).

## Embeddings and retrieval
## Embeddings and retrieval { #embeddings-and-retrieval }

Optionally, the library can compute **embeddings** for extracted content and store vectors in [LanceDB](https://lancedb.com/) for downstream semantic search in your application. For upload and retrieval APIs, see [Vector databases](vdbs.md). For multimodal (VLM) embedding options, see [Multimodal embeddings (VLM)](embedding.md).

Expand All @@ -32,7 +32,7 @@ For PDF parallelism before Ray processing (large files), see [PDF pre-splitting

Token-based splitting uses the Llama 3.2 1B tokenizer (default `meta-llama/Llama-3.2-1B`) with configurable `max_tokens` and `overlap_tokens` when you add an explicit `.split(...)` stage or when the pipeline applies the default text segmentation for unstructured text. In the shipped NeMo Retriever container, tokenizer assets are included locally, so you do not need `HF_ACCESS_TOKEN` for this default path. If your runtime loads the tokenizer from the Hugging Face Hub instead (for example, some library-only installs), set `HF_ACCESS_TOKEN` or pass `hf_access_token` in task params when the Hub requires it. Details appear in the [Python API guide](nemo-retriever-api-reference.md).

## Deployment modes
## Deployment modes { #deployment-modes }

- **Library mode** — Run without the full container stack where appropriate; see [Deployment options](deployment-options.md).
- **Kubernetes / Helm (self-hosted)** — See [Deploy (Helm chart)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) and [deployment options](deployment-options.md) for running the full microservices pipeline on your infrastructure.
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/extraction/custom-metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@ table = db.open_table("nemo_retriever_collection")

**`Retriever.query` + `where`:** LanceDB applies the predicate before ranking. For post-filter logic in Python, use a wider `top_k` first.

For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md).

```python
from nemo_retriever.graph.retriever import Retriever

Expand Down
6 changes: 3 additions & 3 deletions docs/docs/extraction/deployment-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Build and run the NeMo Retriever service image with the [Docker service image gu
3. **Published Library Helm charts (supported):** cluster install and upgrade procedures are covered in the [NeMo Retriever Library](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) — use alongside the NeMo Retriever chart README for your release
4. [Environment variables](environment-config.md) and [Troubleshoot](troubleshoot.md) as needed

**Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (see [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). See [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims).
**Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims).


For audio and video extraction in Kubernetes, set `service.installFfmpeg=true`
Expand All @@ -49,7 +49,7 @@ environments), use a custom service image that already contains `ffmpeg` and
2. [Throughput is dataset-dependent](multimodal-extraction.md#extraction-limitations-and-quality)
3. [Evaluate on your data](evaluate-on-your-data.md)

## When to use NVIDIA-hosted NIMs
## When to use NVIDIA-hosted NIMs { #when-to-use-nvidia-hosted-nims }

[NVIDIA-hosted NIMs](https://build.nvidia.com/) run inference on NVIDIA-managed infrastructure. You call models with API keys (refer to [Get your API key](api-keys.md)) without operating GPU nodes yourself.

Expand All @@ -61,7 +61,7 @@ Consider hosted NIMs when:

**Also refer to:** [NVIDIA NIM catalog](https://build.nvidia.com/)

## When to self-host NIMs
## When to self-host NIMs { #when-to-self-host-nims }

Self-hosted NIMs run on your GPUs or air-gapped hardware, typically with Kubernetes and the [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html).

Expand Down
8 changes: 6 additions & 2 deletions docs/docs/extraction/embedding.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,15 @@ The model can embed documents in the form of an image, text, or a combination of
Documents can then be retrieved given a user query in text form.
The model supports images that contain text, tables, charts, and infographics.

Parameter details for `.extract()` and `.embed()` appear in the [Python API guide](nemo-retriever-api-reference.md).

## Example with Default Text-Based Embedding

When you use the multimodal model, by default, all extracted content (text, tables, charts) is treated as plain text.
The following example provides a strong baseline for retrieval.

- The `embed` method is called with no arguments.

For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md).

```python
from nemo_retriever import create_ingestor

Expand All @@ -42,6 +42,8 @@ The following example enables the multimodal model to capture the spatial and st
- The `embed` method is configured with `embed_modality="text_image"` to embed the extracted tables and charts as images.
- This configuration is more accurate than text only, with a performance cost.

For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md).

```python
from nemo_retriever import create_ingestor

Expand All @@ -65,6 +67,8 @@ The following example extracts and embeds each page as an image.

- The `embed` method processes the page images.

For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md).

```python
from nemo_retriever import create_ingestor

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/extraction/nemo-retriever-api-reference.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# NeMo Retriever API Reference

## PDF pre-splitting for parallel ingest
## PDF pre-splitting for parallel ingest { #pdf-pre-splitting-for-parallel-ingest }

Large PDFs are split into page batches before Ray processing so extraction can run in parallel. This happens on the default ingest path; you do not need extra configuration for typical workloads.

Expand Down
10 changes: 10 additions & 0 deletions docs/docs/extraction/nimclient.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ client = create_inference_client(

To integrate a new NIM, you need to create a custom `ModelInterface` subclass that implements the required methods.

For ingest pipeline APIs used with custom NIMs, refer to the [Python API guide](nemo-retriever-api-reference.md).

### Basic Model Interface Template

```python
Expand Down Expand Up @@ -258,6 +260,8 @@ class MyCustomModelInterface(ModelInterface):

## Real-World Examples

For parameter details on ingest and pipeline APIs, refer to the [Python API guide](nemo-retriever-api-reference.md).

### Text Generation Model Interface

```python
Expand Down Expand Up @@ -377,6 +381,8 @@ class ImageAnalysisModelInterface(ModelInterface):

## Using NimClient in UDFs

For ingest pipeline APIs used in UDFs, refer to the [Python API guide](nemo-retriever-api-reference.md).

### Basic UDF with NimClient

```python
Expand Down Expand Up @@ -513,6 +519,8 @@ export NIM_MAX_RETRIES=5

### Error Handling

For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md).

```python
def robust_nim_udf(control_message: IngestControlMessage) -> IngestControlMessage:
"""UDF with comprehensive error handling."""
Expand Down Expand Up @@ -564,6 +572,8 @@ If memory issues persist, you can reduce the `NIM_TRITON_RATE_LIMIT` value — e

### Debugging Tips

For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md).

```python
import logging

Expand Down
26 changes: 19 additions & 7 deletions docs/docs/extraction/prerequisites-support-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,19 @@

Before you begin using [NeMo Retriever Library](overview.md), confirm your software stack, deployment hardware, and—if you use them—advanced features (audio and video, Nemotron Parse, VLM image captioning, reranking) against the guidance in this page.

## Software Requirements
## On this page { #on-this-page }

- [Software Requirements](#software-requirements)
- [Hardware Requirements](#hardware-requirements)
- [Core and Advanced Pipeline Features](#core-and-advanced-pipeline-features)
- [Default Helm NIMs](#default-helm-nims)
- [OCR artifacts (Helm vs local Hugging Face)](#nemotron-ocr-v2-language-mode)
- [Optional Helm NIMs (not auto-wired)](#optional-helm-nims-not-auto-wired-by-default)
- [Image captioning (26.05)](#image-captioning-2605)
- [Model Hardware Requirements](#model-hardware-requirements)
- [Related Topics](#related-topics)

## Software Requirements { #software-requirements }

- Linux operating systems (Ubuntu 22.04 or later recommended)
- [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) (NVIDIA Driver >= `580`, CUDA >= `13.0`)
Expand All @@ -23,7 +35,7 @@ Before you begin using [NeMo Retriever Library](overview.md), confirm your softw

When you use UV, create the environment with Python 3.12 — for example, `uv venv --python 3.12`. This matches the `requires-python` metadata in the library packages.

## Hardware Requirements
## Hardware Requirements { #hardware-requirements }

The full ingestion pipeline is designed to consume significant CPU and memory resources to achieve maximal parallelism.
Resource usage scales up to the limits of your deployed system.
Expand Down Expand Up @@ -60,11 +72,11 @@ For production deployments processing large volumes of documents, consider:

Ensure your deployment environment meets these specifications before running the full pipeline. Resource-constrained environments may experience performance degradation.

## Core and Advanced Pipeline Features
## Core and Advanced Pipeline Features { #core-and-advanced-pipeline-features }

The NeMo Retriever Library extraction core pipeline features run on a single A10G or better GPU.

### Default Helm NIMs
### Default Helm NIMs { #default-helm-nims }

The production Helm chart enables these NIM microservices **by default** (for example via `nimOperator.*.enabled=true`):

Expand Down Expand Up @@ -94,7 +106,7 @@ Default VL embedder container and model for release deployments:

### Optional Helm NIMs (not auto-wired) { #optional-helm-nims-not-auto-wired-by-default }

These NIM microservices are **optional** for the default extraction pipeline. The retriever service does **not** call them until you enable the matching pipeline stage (reranker, Nemotron Parse, caption, or audio). For **26.05 production**, disable keys you do not need (see [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Set `nimOperator.<key>.enabled=true` when you want that NIM reconciled. Chart keys are in the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#nim-operator-sub-stack).
These NIM microservices are **optional** for the default extraction pipeline. The retriever service does **not** call them until you enable the matching pipeline stage (reranker, Nemotron Parse, caption, or audio). For **26.05 production**, disable keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Set `nimOperator.<key>.enabled=true` when you want that NIM reconciled. Chart keys are in the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#nim-operator-sub-stack).

| Helm flag | NIM | Role |
|-----------|-----|------|
Expand Down Expand Up @@ -122,7 +134,7 @@ Optional features listed in the table above require additional GPU support, disk

For published NIM model IDs and deployment-specific constraints, use the product support matrices linked under [Related Topics](#related-topics) below.

## Model Hardware Requirements
## Model Hardware Requirements { #model-hardware-requirements }

NeMo Retriever Library supports the following GPU hardware given system constraints in the table.

Expand Down Expand Up @@ -158,7 +170,7 @@ Model repositories and NIM references are linked in [Core and Advanced Pipeline
To perform recall testing with the reranker on these GPUs, shut down the core pipeline NIM microservices
and run only the embedder, reranker, and your vector database.

## Related Topics
## Related Topics { #related-topics }

- [Troubleshooting](troubleshoot.md)
- [Release Notes](releasenotes.md)
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/extraction/workflow-document-ingestion.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This page covers extracting content from documents and turning that content into a searchable vector collection in one place so you can scroll and search (for example with Ctrl+F) instead of jumping across multiple short workflow stubs.

## Ingest and extract
## Ingest and extract { #ingest-and-extract }

Document ingestion is the step where NeMo Retriever Library reads your files (PDFs, Office documents, images, and other [supported formats](multimodal-extraction.md#supported-file-types-and-formats)), runs extraction and optional enrichment, and returns structured content you can embed and index.

Expand All @@ -16,7 +16,7 @@ Pipeline concepts and stage overview appear in [Key concepts](concepts.md). Defa

`create_ingestor(...)` returns a `GraphIngestor`, which chains `.extract()`, `.embed()`, and `.vdb_upload()` into one graph. The Python example below stops after `.embed()` so you can inspect chunks first; append `.vdb_upload(vdb_op="lancedb", vdb_kwargs={...})` before `.ingest()` to write directly to LanceDB (refer to [Vector databases](vdbs.md)). For directory-scale corpus ingest, the `graph_pipeline` CLI below is the canonical path used in evaluation workflows.

## Choose how you call the library
## Choose how you call the library { #choose-how-you-call-the-library }

The following examples match the [NeMo Retriever Library README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/README.md). They assume a checkout of the [NeMo Retriever](https://github.com/NVIDIA/NeMo-Retriever) repository and the `batch` run mode with local GPU inference unless you configure remote NIMs.

Expand Down
Loading
Loading