LLM Based Task

docs/docs/extraction/api-keys.md

@@ -27,6 +27,18 @@ For a full list of related variables, see [Environment configuration variables](
 
     The `NVIDIA_API_KEY` from build.nvidia.com is not the same string as your NGC personal key used for Helm and `nvcr.io` access. Do not substitute one for the other unless your tooling explicitly documents that mapping.
 
+## Credential references in persisted graphs
+
+Persisted pipeline graphs never contain literal API keys. Configure a graph with an explicit worker-side environment reference such as:
+
+```python
+api_key="os.environ/NVIDIA_API_KEY"
+```
+
+Use the provider's own variable name, for example `os.environ/OPENAI_API_KEY` for an OpenAI model. The reference is stored in graph JSON and resolved only when the operator is constructed or invoked on the worker.
+
+Literal keys remain available for non-persisted local execution, but attempting to serialize one raises an error. This prevents graph persistence from silently substituting an NVIDIA credential for another provider's key.
+
 ## NGC personal key (Helm and `nvcr.io`)
 
 Many public assets on NGC can be used without authentication. For a Kubernetes deployment, the cluster must still pull NIM and microservice images from `nvcr.io` and may need NGC API access; the Helm chart expects credentials derived from an NGC personal key.

docs/docs/extraction/nemo-retriever-api-reference.md

@@ -8,6 +8,32 @@ To tune splitter throughput from the CLI, use `--pdf-split-batch-size` (Ray acto
 
 **Python client (`pdf_split_config`):** Only `create_ingestor(run_mode="service")` implements `.pdf_split_config(pages_per_chunk=...)`, which records page-chunking settings in the request pipeline spec for the remote gateway. Local graph ingest (`run_mode="inprocess"` or `"batch"`) raises `NotImplementedError` if you call this method; PDFs are split automatically on the default ingest path without client-side configuration.
 
+## One-shot text generation
+
+`TextGenerationOperator` is the reusable base for synchronous, one-request-per-row text generation. It is a provisional text-only API: it does not support tool calls, agent loops, streaming, multiple choices, or structured domain results.
+
+Concrete operators construct an immutable `GenerationTask` and provide reconstructible constructor state. Runtime task and client objects must not be included in graph constructor arguments. A custom completion client must be safe for concurrent calls or report that it does not support concurrent calls so the operator serializes access.
+
+Embedding and captioning remain separate operator families because they use modality grouping, native batching, and specialized CPU/GPU lifecycles.
+
+## Persisted graphs are trusted configuration
+
+Graph loading imports operator classes and invokes their constructors. Load graph JSON only from trusted sources; do not expose graph payloads, callable references, or class names as model- or user-controlled agent tools.
+
+Version 2 graph files preserve shared-node DAG identity and reject cycles. Constructor state must consist of supported JSON-native values, typed Pydantic models, paths, sets and tuples, or importable type/callable references. Runtime data such as DataFrames and opaque client objects is not persistable.
+
+API keys are never written into graph JSON. Use an explicit environment reference in persisted configuration:
+
+```python
+QAGenerationOperator(
+    model="openai/gpt-4o-mini",
+    api_key="os.environ/OPENAI_API_KEY",
+)
+```
+
+Serializing a graph containing a literal API key fails with a contextual error instead of guessing which provider credential should be used on a worker.
+
+
 ::: nemo_retriever.ingestor
     options:
       filters:

nemo_retriever/src/nemo_retriever/common/params/__init__.py

@@ -20,6 +20,7 @@
 from nemo_retriever.common.params.models import LanceDbParams
 from nemo_retriever.common.params.models import LLMInferenceParams
 from nemo_retriever.common.params.models import LLMRemoteClientParams
+from nemo_retriever.common.params.models import LLMSamplingOverrides
 from nemo_retriever.common.params.models import ModelRuntimeParams
 from nemo_retriever.common.params.models import OcrParams
 from nemo_retriever.common.params.models import PageElementsParams
@@ -30,6 +31,7 @@
 from nemo_retriever.common.params.models import TabularExtractParams
 from nemo_retriever.common.params.models import TableParams
 from nemo_retriever.common.params.models import TextChunkParams
+from nemo_retriever.common.params.models import TextGenerationParams
 from nemo_retriever.common.params.models import MetaJoinKey
 from nemo_retriever.common.params.models import VdbUploadParams
 from nemo_retriever.common.params.models import VideoFrameParams
@@ -58,6 +60,7 @@
     "LanceDbParams",
     "LLMInferenceParams",
     "LLMRemoteClientParams",
+    "LLMSamplingOverrides",
     "ModelRuntimeParams",
     "OcrParams",
     "PageElementsParams",
@@ -69,6 +72,7 @@
     "TabularExtractParams",
     "TableParams",
     "TextChunkParams",
+    "TextGenerationParams",
     "MetaJoinKey",
     "VdbUploadParams",
     "VideoFrameParams",