Skip to content

Commit 09071b6

Browse files
authored
Add root service query mode (#2228)
1 parent 7e367bf commit 09071b6

19 files changed

Lines changed: 1079 additions & 492 deletions

docs/docs/extraction/deployment-options.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ Build and run the NeMo Retriever service image with the [Docker service image gu
2424

2525
**Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims).
2626

27-
For audio and video dependencies and OpenShift-specific Helm configuration, refer to [Audio and video (Parakeet ASR)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-video-parakeet) and [OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/openshift.md) in the Helm chart directory.
27+
For audio and video extraction in Kubernetes, set `service.installFfmpeg=true` so the service installs `ffmpeg` and `ffprobe` at container startup. This runtime install requires package-repository egress and cluster policy that allows the image's scoped sudo use. For Helm chart procedures, OpenShift-specific configuration, and air-gapped alternatives, refer to [Audio and video (Parakeet ASR)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#audio-video-parakeet) and [OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/openshift.md) in the Helm chart directory.
2828

2929
### I want examples and notebooks
3030

nemo_retriever/docs/cli/README.md

Lines changed: 34 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ ingest and retrieval.
66
For product-facing examples, prefer these commands:
77

88
- `retriever ingest` - ingest supported documents and media into a Retriever index.
9-
- `retriever query` - query a local LanceDB table written by ingest.
9+
- `retriever query` - query a local LanceDB table written by local or batch ingest.
10+
- `retriever query service` - query a Retriever service deployment.
1011

1112
`retriever pipeline run` remains available as a development and compatibility
1213
command for legacy pipeline workflows, evaluation, intermediate artifacts, and
@@ -69,9 +70,8 @@ retriever query "What is in this document?" \
6970
By default, local ingest writes to `lancedb/nemo-retriever` and `retriever query`
7071
reads from the same table.
7172

72-
The `retriever query` examples below apply to local and batch ingest output
73-
written to LanceDB. Service ingest sends documents to a remote service; querying
74-
service-owned storage is handled by that service deployment.
73+
The plain `retriever query` examples below apply to local and batch ingest output
74+
written to LanceDB. Use `retriever query service` to query a Retriever service.
7575

7676
### Ingest a larger corpus with batch mode
7777

@@ -94,8 +94,13 @@ retriever ingest service ./data/pdf_corpus \
9494
```
9595

9696
Use `--service-api-token` or `NEMO_RETRIEVER_API_TOKEN` when the service requires
97-
a bearer token. Service ingest does not expose `--lancedb-uri`; the service owns
98-
its vector database configuration.
97+
a bearer token. Service ingest does not expose `--lancedb-uri`; the service
98+
configures its vector database. Query the service with:
99+
100+
```bash
101+
retriever query service "What is in this corpus?" \
102+
--service-url http://localhost:7670
103+
```
99104

100105
### Route ingest to hosted or self-hosted NIM endpoints
101106

@@ -131,9 +136,10 @@ retriever query "What is in this document?" \
131136

132137
### Query result controls
133138

134-
`retriever query` returns compact JSON hits with `source`, `page_number`, and
135-
`text`. Use `--candidate-k`, `--page-dedup`, and `--content-types` to shape the
136-
result set after vector retrieval:
139+
Both `retriever query` and `retriever query service` return compact JSON hits
140+
with `source`, `page_number`, and `text`. Use `--candidate-k`, `--page-dedup`,
141+
and `--content-types` to control how results are selected after vector
142+
retrieval:
137143

138144
```bash
139145
retriever query "annual revenue by region" \
@@ -142,19 +148,27 @@ retriever query "annual revenue by region" \
142148
--content-types table
143149
```
144150

145-
`--top-k` is the final number of hits returned. `--candidate-k` is the wider
146-
candidate pool retrieved before page deduplication, content-type filtering, and
147-
final truncation. It must be greater than or equal to `--top-k`, and should
148-
usually be larger when page deduplication or content-type filtering might
149-
otherwise remove too many of the top retrieved rows. Page deduplication and
150-
content-type filtering are applied after vector retrieval, preserving the
151-
retriever's ranking order and truncating the final output to `--top-k`.
152-
When querying a table ingested with an explicit embedding model, pass the same
153-
`--embed-model-name` to `retriever query`.
151+
`--top-k` is the final number of results to return after filtering and
152+
deduplication. `--candidate-k` is the number of raw results to retrieve from
153+
LanceDB or the Retriever service before filtering, page deduplication, and
154+
final truncation. If omitted, the candidate pool is the same size as
155+
`--top-k`. Set `--candidate-k` larger than `--top-k` when page deduplication
156+
or content-type filtering might remove too many of the nearest retrieved rows.
157+
It must always be greater than or equal to `--top-k`.
158+
159+
Page deduplication and content-type filtering are applied after vector
160+
retrieval, preserving retriever ranking order and truncating the final output to
161+
`--top-k`. When querying a local table ingested with an explicit embedding
162+
model, pass the same `--embed-model-name` to `retriever query`.
163+
154164
`--content-types` accepts comma-separated content types such as `text`, `table`,
155165
`chart`, `image`, and `infographic`. `images` is accepted as an alias for
156-
captioned image rows emitted by ingest. Hits with missing or unknown content
157-
types are excluded while `--content-types` is active.
166+
captioned image rows emitted by ingest. This option filters by content-type
167+
metadata only; it does not filter by source, page, or other metadata
168+
predicates. Hits with missing or unknown content-type metadata are excluded
169+
while `--content-types` is active. In service mode, results must include
170+
content-type metadata to match this filter. Default display values in the JSON
171+
output are not used for content-type matching.
158172

159173
### Agentic retrieval
160174

0 commit comments

Comments
 (0)