Chat UI answers section-specific questions correctly, but Chat API ignores indexed nodes and falls back to generic assistant

[parthpaliwal-49]

Hi PageIndex team,

I’m integrating PageIndex via the Python SDK and I’m seeing a consistent difference between the Chat UI and the Chat API (chat_completions) behavior.

What works in the UI:

Upload a PDF (e.g., an air-conditioner operating manual)

Ask a section-specific question like “Give me OPERATING TIPS from the document”

The UI correctly retrieves and summarizes the exact section

What fails via the API:

The same document is uploaded and fully processed (status=completed)

get_tree() shows nodes with titles, page indexes, and extracted text (e.g., OPERATING TIPS)

Calling chat_completions(messages, doc_id=...) with the same question

The model calls recent_documents, claims the document is missing, and answers generically

From MCP traces, it appears that when retrieval confidence is low, the Chat API silently falls back to a generic assistant and ignores the document — even though the node content exists and is retrievable via get_tree().

Questions:

Is this difference between the Chat UI and Chat API expected behavior?

Does the UI use additional retrieval orchestration (retries, node/page binding, forced grounding) that is not exposed via the API?

Is there a documented or recommended way to force the Chat API to answer only from indexed document content or specific nodes?

If not, should section-level or procedural Q&A be implemented using the retrieval endpoints instead of chat_completions?

I’m trying to understand the intended usage boundaries here

Thanks in advance for clarification.

[Manoj-Gujare]

Hi @parthpaliwal-49 ,

The Chat UI and Chat API are different surfaces. The Chat API (beta) is an agentic loop over PageIndex's name-based MCP tools — it discovers a document, then reads it. Your recent_documents → "document missing" → generic answer is the discovery step never binding your doc, so the agent fell back to general knowledge. Your index is fine (get_tree() proves it); the agent just never read it.

Q1 & Q2 (expected? hidden orchestration?): Yes. The hosted UI binds the open document into the agent's scope so discovery can't wander off; the API exposes the raw agent and leaves that to you. Same primitives underneath.

Q3 (force document-only answers?): No single "grounded-only" flag, but combine:

• pass doc_id
• set temperature=0
• add a system message: "Answer only from the selected document; don't browse other docs or use general knowledge; if it's not there, say so"
• set enable_citations=True (no citations = not grounded)
• while streaming, watch mcp_tool_use_start — get_page_content is the right path, recent_documents is the wrong one
• pip install -U pageindex (recent_documents is deprecated/hidden in current SDKs, so seeing it suggests an old build)
• confirm the doc was uploaded under the same API key and isn't in a different folder scope

Q4 (use retrieval instead for section Q&A?): Yes — for deterministic lookups like "OPERATING TIPS," skip the agent and retrieve directly:

import time
if pi_client.is_retrieval_ready(doc_id):
    r = pi_client.submit_query(doc_id, "OPERATING TIPS", thinking=True)
    res = pi_client.get_retrieval(r["retrieval_id"])
    while res.get("status") != "completed":          # retrieval is async
        time.sleep(1); res = pi_client.get_retrieval(r["retrieval_id"])
    for node in res["retrieved_nodes"]:
        for c in node["relevant_contents"]:           # note: nested array
            print(node["title"], c["page_index"], c["relevant_content"])

Or just pick the node from get_tree() and feed its text to your own LLM (vectorless RAG).

Rule of thumb: agentic Chat API for open-ended multi-doc chat; Retrieval API / tree-node selection for precise, must-be-grounded sections.