diff --git a/docs/docs/extraction/agentic-retrieval-concept.md b/docs/docs/extraction/agentic-retrieval-concept.md index b2bfd09cfd..12689ada61 100644 --- a/docs/docs/extraction/agentic-retrieval-concept.md +++ b/docs/docs/extraction/agentic-retrieval-concept.md @@ -7,4 +7,4 @@ NeMo Retriever Library focuses on document ingestion, embeddings, vector stores, **Related** - [Semantic retrieval](vdbs.md#semantic-retrieval) -- Framework examples: [Starter kits](starter-kits.md) +- Framework examples: [Starter kits](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md) diff --git a/docs/docs/extraction/concepts.md b/docs/docs/extraction/concepts.md index 4f18ba6c26..87586de017 100644 --- a/docs/docs/extraction/concepts.md +++ b/docs/docs/extraction/concepts.md @@ -36,6 +36,6 @@ Token-based splitting uses the Llama 3.2 1B tokenizer (default `meta-llama/Llama - **Library mode** — Run without the full container stack where appropriate; refer to [Deployment options](deployment-options.md). - **Kubernetes / Helm (self-hosted)** — Refer to [Deploy (Helm chart)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) and [deployment options](deployment-options.md) for running the full microservices pipeline on your infrastructure. -- **Notebooks** — [Jupyter examples](starter-kits.md) for experimentation and RAG demos. +- **Notebooks** — [Jupyter examples](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md) for experimentation and RAG demos. For a concise comparison, refer to [Deployment options](deployment-options.md). diff --git a/docs/docs/extraction/deployment-options.md b/docs/docs/extraction/deployment-options.md index b57404685a..cec011c555 100644 --- a/docs/docs/extraction/deployment-options.md +++ b/docs/docs/extraction/deployment-options.md @@ -24,11 +24,11 @@ Build and run the NeMo Retriever service image with the [Docker service image gu **Core NIMs for the default extraction pipeline:** `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims). -For audio and video extraction in Kubernetes, set `service.installFfmpeg=true` so the service container installs `ffmpeg` and `ffprobe` at startup. This runtime install requires package-repository network egress, a writable root filesystem, and security policy that allows the image's scoped sudo use. If your cluster blocks startup package installation, use a custom service image that already contains `ffmpeg` and `ffprobe`, then set `service.image.repository` and `service.image.tag`. For Parakeet ASR chart values, OpenShift-specific Helm configuration, and air-gapped alternatives, refer to [Audio and video (Parakeet ASR)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-video-parakeet) and [OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/openshift.md) in the Helm chart directory. +For audio and video extraction in Kubernetes, refer to [Audio and video](audio-video.md). ### I want examples and notebooks -1. [Jupyter Notebooks](starter-kits.md) +1. [Jupyter Notebooks](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md) ### I need API details and keys @@ -73,7 +73,7 @@ On a staging host with internet access, pull from NGC, retag to your private reg !!! warning "Audio and video extraction" - [Audio and video](audio-video.md) need **`ffmpeg` and `ffprobe` on `PATH`**. The bundled image omits them. Do **not** use `service.installFfmpeg=true` in an air gap (startup install needs package-repo egress). Build a custom service image on a connected staging host, mirror it, and set `service.image.repository` / `service.image.tag`. Skip this step if you do not use audio/video. + Audio and video workflows require `ffmpeg` and `ffprobe` on `PATH`; runtime package installation is not suitable for air-gapped clusters. Refer to [Audio and video](audio-video.md) and the Helm chart [air-gapped deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#air-gapped-deployment) guide. Skip this if you do not use audio or video. For offline image captioning, deploy the in-cluster [Nemotron 3 Nano Omni](prerequisites-support-matrix.md#image-captioning) NIM and point your pipeline caption endpoint at the in-cluster HTTP URL instead of `integrate.api.nvidia.com` or other hosted APIs. diff --git a/docs/docs/extraction/getting-started-about.md b/docs/docs/extraction/getting-started-about.md index 8f3a8ee69f..e737d091f1 100644 --- a/docs/docs/extraction/getting-started-about.md +++ b/docs/docs/extraction/getting-started-about.md @@ -8,7 +8,7 @@ Typical order: 2. Confirm the [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md) for your OS, GPU, and software stack. 3. Deploy using one of: - [Deployment options](deployment-options.md) for library, hosted NIMs, and Kubernetes paths - - **Supported:** [Helm chart](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) for Kubernetes cluster install and upgrade -4. Explore [Jupyter Notebooks](starter-kits.md) for end-to-end examples. + - **Supported:** [Helm chart](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) for Kubernetes, plus [NeMo Retriever Library install docs](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) for the published charts +4. Explore [Jupyter Notebooks](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md) for end-to-end examples. If you are new to the product, read [What is NeMo Retriever Library?](overview.md) and [Concepts](concepts.md) under **Introduction** first. diff --git a/docs/docs/extraction/overview.md b/docs/docs/extraction/overview.md index f4837ae63b..04385adefd 100644 --- a/docs/docs/extraction/overview.md +++ b/docs/docs/extraction/overview.md @@ -48,5 +48,6 @@ NeMo Retriever Library supports the following file types: - [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md) - [Deployment options](deployment-options.md) — library, Helm, hosted vs self-hosted NIMs in one place - [Deploy on Kubernetes with Helm](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) -- [Notebooks](starter-kits.md) -- [NVIDIA AI Blueprints catalog](https://build.nvidia.com/explore/discover) — solution cards, enterprise RAG blueprints, and end-to-end patterns (including [Enterprise RAG — multimodal PDF data extraction](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag)); for integration pathways, refer to [Starter kits](starter-kits.md). +- [NeMo Retriever Library — prerequisites / deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) (supported Helm charts) +- [Notebooks](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md) +- [NVIDIA AI Blueprints catalog](https://build.nvidia.com/explore/discover) — solution cards, enterprise RAG blueprints, and end-to-end patterns (including [Enterprise RAG — multimodal PDF data extraction](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag)); for integration pathways, refer to [Starter kits](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md). diff --git a/docs/docs/extraction/starter-kits.md b/docs/docs/extraction/starter-kits.md deleted file mode 100644 index 08b366a6eb..0000000000 --- a/docs/docs/extraction/starter-kits.md +++ /dev/null @@ -1,24 +0,0 @@ -# Starter Kits for NeMo Retriever Library - -To get started using [NeMo Retriever Library](overview.md), you can try one of the ready-made notebooks that are available. - -## Dataset Downloads for Benchmarking - -If you plan to run benchmarking or evaluation tests, you must download the [Benchmark Datasets (Bo20, Bo767, Bo10k)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/evaluation/digital_corpora_download.ipynb) from Digital Corpora. This is a prerequisite for all benchmarking operations. - -## Getting Started - -To get started with the basics, try one of the following guides or notebooks: - -- [Quickstart: retriever CLI](../reference/retriever-cli-quickstart.md) -- [Workflow: Ingest documents](workflow-document-ingestion.md) -- [Adding Custom Metadata for Filtered Search/Retrieval](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/nemo_retriever_retriever_query_metadata_filter.ipynb) — also summarized on [Vector databases — Metadata and filtering](vdbs.md#metadata-and-filtering) - - -For more advanced scenarios, try one of the following notebooks: - -- [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb) -- [Try Enterprise RAG Blueprint](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag) -- [Evaluate bo767 retrieval recall accuracy with NeMo Retriever Library](https://github.com/NVIDIA/NeMo-Retriever/blob/main/evaluation/bo767_recall.ipynb) -- [Multimodal RAG with LangChain](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/langchain_multimodal_rag.ipynb) -- [Multimodal RAG with LlamaIndex](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/llama_index_multimodal_rag.ipynb) diff --git a/docs/docs/extraction/workflow-e2e-blueprints.md b/docs/docs/extraction/workflow-e2e-blueprints.md index 93203a6374..7123f02207 100644 --- a/docs/docs/extraction/workflow-e2e-blueprints.md +++ b/docs/docs/extraction/workflow-e2e-blueprints.md @@ -5,4 +5,4 @@ Use these external resources for end-to-end RAG implementations with NeMo Retrie - [Enterprise RAG - multimodal PDF data extraction](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag) - [NVIDIA AI Blueprints catalog](https://build.nvidia.com/explore/discover) -For framework-specific integration patterns, refer to [Starter kits](starter-kits.md). +For framework-specific integration patterns, refer to [Starter kits](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md). diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 4e23c7d0a6..915fcec187 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -101,7 +101,7 @@ nav: - "8. Customize & extend": - Extending/Customizing NeMo Retriever Library with custom code: https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph - "9. Integrations & ecosystem": - - "Starter kits": extraction/starter-kits.md + - "Starter kits": https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md - "10. Evaluation & benchmarks": - "Evaluate on your own documents": extraction/evaluate-on-your-data.md - "11. Reference": @@ -156,11 +156,12 @@ plugins: extraction/hosted-nims-when-to-use.md: extraction/deployment-options.md extraction/releasenotes-nv-ingest.md: extraction/releasenotes.md extraction/ngc-api-key.md: extraction/api-keys.md - extraction/notebooks/index.md: extraction/starter-kits.md - extraction/notebooks.md: extraction/starter-kits.md + extraction/notebooks/index.md: https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md + extraction/notebooks.md: https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md + extraction/starter-kits.md: https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md extraction/data-store.md: extraction/vdbs.md extraction/custom-metadata.md: extraction/vdbs.md#metadata-and-filtering - extraction/integrations-langchain-llamaindex-haystack.md: extraction/starter-kits.md + extraction/integrations-langchain-llamaindex-haystack.md: https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md extraction/nemoretriever-parse.md: extraction/multimodal-extraction.md#text-and-layout-extraction extraction/supported-file-types.md: extraction/multimodal-extraction.md#supported-file-types-and-formats extraction/text-layout-extraction.md: extraction/multimodal-extraction.md#text-and-layout-extraction diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000000..684ab20226 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,23 @@ +# Starter Kits for NeMo Retriever Library + +Explore ready-made Jupyter notebooks and guides for [NeMo Retriever Library](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/). + +## Dataset Downloads for Benchmarking + +If you run benchmarking or evaluation tests, download the [Benchmark Datasets (Bo20, Bo767, Bo10k)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/evaluation/digital_corpora_download.ipynb) from Digital Corpora first. This prerequisite applies to all benchmarking operations. + +## Getting Started + +Start with these guides and notebooks: + +- [Quickstart: retriever CLI](https://docs.nvidia.com/nemo/retriever/latest/reference/retriever-cli-quickstart/) +- [Workflow: Ingest documents](https://docs.nvidia.com/nemo/retriever/latest/extraction/workflow-document-ingestion/) +- [Adding custom metadata for filtered search and retrieval](nemo_retriever_retriever_query_metadata_filter.ipynb) — also summarized on [Vector databases — Metadata and filtering](https://docs.nvidia.com/nemo/retriever/latest/extraction/vdbs/#metadata-and-filtering) + +For advanced scenarios, use these notebooks: + +- [Build a custom vector database operator](building_vdb_operator.ipynb) +- [Try Enterprise RAG Blueprint](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag) +- [Evaluate bo767 retrieval recall accuracy with NeMo Retriever Library](https://github.com/NVIDIA/NeMo-Retriever/blob/main/evaluation/bo767_recall.ipynb) +- [Multimodal RAG with LangChain](langchain_multimodal_rag.ipynb) +- [Multimodal RAG with LlamaIndex](llama_index_multimodal_rag.ipynb)