feat: add language detection functionality with robust parsing and structured output

Author: a-klos

libs/README.md

@@ -90,12 +90,14 @@ Uploaded documents are required to contain the following metadata:
 | composed_retriever | [`rag_core_api.retriever.retriever.Retriever`](./rag-core-api/src/rag_core_api/retriever/retriever.py) | [`rag_core_api.impl.retriever.composite_retriever.CompositeRetriever`](./rag-core-api/src/rag_core_api/impl/retriever/composite_retriever.py) | Handles retrieval, re-ranking, etc. |
 | large_language_model | `langchain_core.language_models.llms.BaseLLM` | `langchain_community.llms.vllm.VLLMOpenAI`, `langchain_community.llms.Ollama` or `langchain_community.llms.FakeListLLM` | The LLm that is used for all LLM tasks. The default depends on the value of `rag_core_lib.impl.settings.rag_class_types_settings.RAGClassTypeSettings.llm_type`. The FakeListLLM is used for testing |
 | prompt | `str` | [`rag_core_api.prompt_templates.answer_generation_prompt.ANSWER_GENERATION_PROMPT`](./rag-core-api/src/rag_core_api/prompt_templates/answer_generation_prompt.py) | The prompt used for answering the question. |
-| rephrasing_prompt | `str` |  [`rag_core_api.prompt_templates.question_rephrasing_prompt.ANSWER_REPHRASING_PROMPT`](./rag-core-api/src/rag_core_api/prompt_templates/question_rephrasing_prompt.py) | The prompt used for rephrasing the question. The rephrased question (and the *original* question are both used for retrival of the documents)|
+| rephrasing_prompt | `str` |  [`rag_core_api.prompt_templates.question_rephrasing_prompt.ANSWER_REPHRASING_PROMPT`](./rag-core-api/src/rag_core_api/prompt_templates/question_rephrasing_prompt.py) | The prompt used for rephrasing the question. The rephrased question (and the *original* question are both used for retrival of the documents) |
+| language_detection_prompt | `str` | [`rag_core_api.prompt_templates.language_detection_prompt.LANGUAGE_DETECTION_PROMPT`](./rag-core-api/src/rag_core_api/prompt_templates/language_detection_prompt.py) | Prompt for detecting input language. Enforces structured JSON output `{ "language": "<iso639-1>" }` and defaults to `en` when uncertain. |
 | langfuse_manager | [`rag_core_lib.impl.langfuse_manager.langfuse_manager.LangfuseManager`](./rag-core-lib/src/rag_core_lib/impl/langfuse_manager/langfuse_manager.py) | [`rag_core_lib.impl.langfuse_manager.langfuse_manager.LangfuseManager`](./rag-core-lib/src/rag_core_lib/impl/langfuse_manager/langfuse_manager.py) | Retrieves additional settings, as well as the prompt from langfuse if available. |
-| answer_generation_chain | [`rag_core_lib.chains.async_chain.AsyncChain[rag_core_api.impl.graph.graph_state.graph_state.AnswerGraphState, str]`](./rag-core-lib/src/rag_core_lib/chains/async_chain.py) | [`rag_core_api.impl.answer_generation_chains.answer_generation_chain.AnswerGenerationChain`](./rag-core-api/src/rag_core_api/impl/answer_generation_chains/answer_generation_chain.py) | LangChain chain used for answering the question. Is part of the *chat_graph*, |
-| rephrasing_chain | [`rag_core_lib.chains.async_chain.AsyncChain[rag_core_api.impl.graph.graph_state.graph_state.AnswerGraphState, str]`](./rag-core-lib/src/rag_core_lib/chains/async_chain.py) | [`rag_core_api.impl.answer_generation_chains.rephrasing_chain.RephrasingChain`](./rag-core-api/src/rag_core_api/impl/answer_generation_chains/rephrasing_chain.py) | LangChain chain used for rephrasing the question. Is part of the *chat_graph*. |
+| answer_generation_chain | [`rag_core_lib.chains.runnables.AsyncRunnable[rag_core_api.impl.graph.graph_state.graph_state.AnswerGraphState, str]`](./rag-core-lib/src/rag_core_lib/runnables/async_runnable.py) | [`rag_core_api.impl.answer_generation_chains.answer_generation_chain.AnswerGenerationChain`](./rag-core-api/src/rag_core_api/impl/answer_generation_chains/answer_generation_chain.py) | LangChain chain used for answering the question. Is part of the *chat_graph*, |
+| rephrasing_chain | [`rag_core_lib.chains.runnables.AsyncRunnable[rag_core_api.impl.graph.graph_state.graph_state.AnswerGraphState, str]`](./rag-core-lib/src/rag_core_lib/runnables/async_runnable.py) | [`rag_core_api.impl.answer_generation_chains.rephrasing_chain.RephrasingChain`](./rag-core-api/src/rag_core_api/impl/answer_generation_chains/rephrasing_chain.py) | LangChain chain used for rephrasing the question. Is part of the *chat_graph*. |
+| language_detection_chain | [`rag_core_lib.chains.runnables.AsyncRunnable[rag_core_api.impl.graph.graph_state.graph_state.AnswerGraphState, str]`](./rag-core-lib/src/rag_core_lib/runnables/async_runnable.py) | [`rag_core_api.impl.answer_generation_chains.language_detection_chain.LanguageDetectionChain`](./rag-core-api/src/rag_core_api/impl/answer_generation_chains/language_detection_chain.py) | Detects the language of the question and returns an ISO 639-1 code (e.g., `en`, `de`). Uses structured-output guidance and robust parsing with fallback to `en`. Part of the *chat_graph*. |
 | chat_graph | [`rag_core_api.graph.graph_base.GraphBase`](./rag-core-api/src/rag_core_api/graph/graph_base.py) | [`rag_core_api.impl.graph.chat_graph.DefaultChatGraph`](./rag-core-api/src/rag_core_api/impl/graph/chat_graph.py) | Langgraph graph that contains the entire logic for question answering. |
-| traced_chat_graph | [`rag_core_lib.chains.async_chain.AsyncChain[Any, Any]`](./rag-core-lib/src/rag_core_lib/chains/async_chain.py)| [`rag_core_lib.impl.tracers.langfuse_traced_chain.LangfuseTracedGraph`](./rag-core-lib/src/rag_core_lib/impl/tracers/langfuse_traced_chain.py) | Wraps around the *chat_graph* and add langfuse tracing. |
+| traced_chat_graph | [`rag_core_lib.chains.runnables.AsyncRunnable[Any, Any]`](./rag-core-lib/src/rag_core_lib/runnables/async_runnable.py)| [`rag_core_lib.impl.tracers.langfuse_traced_chain.LangfuseTracedGraph`](./rag-core-lib/src/rag_core_lib/impl/tracers/langfuse_traced_chain.py) | Wraps around the *chat_graph* and add langfuse tracing. |
 | evaluator | [`rag_core_api.impl.evaluator.langfuse_ragas_evaluator.LangfuseRagasEvaluator`](./rag-core-api/src/rag_core_api/impl/evaluator/langfuse_ragas_evaluator.py) | [`rag_core_api.impl.evaluator.langfuse_ragas_evaluator.LangfuseRagasEvaluator`](./rag-core-api/src/rag_core_api/impl/evaluator/langfuse_ragas_evaluator.py) | The evaulator used in the evaluate endpoint. |
 | chat_endpoint | [`rag_core_api.api_endpoints.chat.Chat`](./rag-core-api/src/rag_core_api/api_endpoints/chat.py) | [`rag_core_api.impl.api_endpoints.default_chat.DefaultChat`](./rag-core-api/src/rag_core_api/impl/api_endpoints/default_chat.py) | Implementation of the chat endpoint. Default implementation just calls the *traced_chat_graph* |
 | ragas_llm | `langchain_core.language_models.chat_models.BaseChatModel` | `langchain_openai.ChatOpenAI` or `langchain_ollama.ChatOllama` | The LLM used for the ragas evaluation. |
@@ -191,7 +193,7 @@ The extracted information will be summarized using LLM. The summary, as well as
 | langfuse_manager | [`rag_core_lib.impl.langfuse_manager.langfuse_manager.LangfuseManager`](./rag-core-lib/src/rag_core_lib/impl/langfuse_manager/langfuse_manager.py) | [`rag_core_lib.impl.langfuse_manager.langfuse_manager.LangfuseManager`](./rag-core-lib/src/rag_core_lib/impl/langfuse_manager/langfuse_manager.py) | Retrieves additional settings, as well as the prompt from langfuse if available. |
 | summarizer |  [`admin_api_lib.summarizer.summarizer.Summarizer`](./admin-api-lib/src/admin_api_lib/summarizer/summarizer.py) | [`admin_api_lib.impl.summarizer.langchain_summarizer.LangchainSummarizer`](./admin-api-lib/src/admin_api_lib/impl/summarizer/langchain_summarizer.py) | Creates the summaries. Uses the shared retry decorator with optional per-summarizer overrides (see 2.4). |
 | untraced_information_enhancer |[`admin_api_lib.information_enhancer.information_enhancer.InformationEnhancer`](./admin-api-lib/src/admin_api_lib/information_enhancer/information_enhancer.py) | [`admin_api_lib.impl.information_enhancer.general_enhancer.GeneralEnhancer`](./admin-api-lib/src/admin_api_lib/impl/information_enhancer/general_enhancer.py) |  Uses the *summarizer* to enhance the extracted documents. |
-| information_enhancer |  [`rag_core_lib.chains.async_chain.AsyncChain[Any, Any]`](./rag-core-lib/src/rag_core_lib/chains/async_chain.py)| [`rag_core_lib.impl.tracers.langfuse_traced_chain.LangfuseTracedGraph`](./rag-core-lib/src/rag_core_lib/impl/tracers/langfuse_traced_chain.py) |Wraps around the *untraced_information_enhancer* and adds langfuse tracing. |
+| information_enhancer |  [`rag_core_lib.chains.runnables.AsyncRunnable[Any, Any]`](./rag-core-lib/src/rag_core_lib/runnables/async_runnable.py)| [`rag_core_lib.impl.tracers.langfuse_traced_chain.LangfuseTracedGraph`](./rag-core-lib/src/rag_core_lib/impl/tracers/langfuse_traced_chain.py) |Wraps around the *untraced_information_enhancer* and adds langfuse tracing. |
 | document_deleter |[`admin_api_lib.api_endpoints.document_deleter.DocumentDeleter`](./admin-api-lib/src/admin_api_lib/api_endpoints/document_deleter.py) | [`admin_api_lib.impl.api_endpoints.default_document_deleter.DefaultDocumentDeleter`](./admin-api-lib/src/admin_api_lib/impl/api_endpoints/default_document_deleter.py) |  Handles deletion of sources. |
 | documents_status_retriever |  [`admin_api_lib.api_endpoints.documents_status_retriever.DocumentsStatusRetriever`](./admin-api-lib/src/admin_api_lib/api_endpoints/documents_status_retriever.py) | [`admin_api_lib.impl.api_endpoints.default_documents_status_retriever.DefaultDocumentsStatusRetriever`](./admin-api-lib/src/admin_api_lib/impl/api_endpoints/default_documents_status_retriever.py) |Handles return of source status. |
 | source_uploader | [`admin_api_lib.api_endpoints.source_uploader.SourceUploader`](./admin-api-lib/src/admin_api_lib/api_endpoints/source_uploader.py) | [`admin_api_lib.impl.api_endpoints.default_source_uploader.DefaultSourceUploader`](./admin-api-lib/src/admin_api_lib/impl/api_endpoints/default_source_uploader.py)| Handles data loading and extraction from various non-file sources. |

libs/rag-core-api/src/rag_core_api/dependency_container.py

@@ -19,6 +19,7 @@
     AnswerGenerationChain,
 )
 from rag_core_api.impl.answer_generation_chains.rephrasing_chain import RephrasingChain
+from rag_core_api.impl.answer_generation_chains.language_detection_chain import LanguageDetectionChain
 from rag_core_api.impl.api_endpoints.default_chat import DefaultChat
 from rag_core_api.impl.api_endpoints.default_information_pieces_remover import (
     DefaultInformationPiecesRemover,
@@ -57,6 +58,7 @@
 from rag_core_api.prompt_templates.question_rephrasing_prompt import (
     QUESTION_REPHRASING_PROMPT,
 )
+from rag_core_api.prompt_templates.language_detection_prompt import LANGUAGE_DETECTION_PROMPT
 from rag_core_lib.impl.data_types.content_type import ContentType
 from rag_core_lib.impl.langfuse_manager.langfuse_manager import LangfuseManager
 from rag_core_lib.impl.llms.llm_factory import chat_model_provider
@@ -180,6 +182,7 @@ class DependencyContainer(DeclarativeContainer):
 
     prompt = ANSWER_GENERATION_PROMPT
     rephrasing_prompt = QUESTION_REPHRASING_PROMPT
+    language_detection_prompt = LANGUAGE_DETECTION_PROMPT
 
     langfuse = Singleton(
         Langfuse,
@@ -194,6 +197,7 @@ class DependencyContainer(DeclarativeContainer):
         managed_prompts={
             AnswerGenerationChain.__name__: prompt,
             RephrasingChain.__name__: rephrasing_prompt,
+            LanguageDetectionChain.__name__: language_detection_prompt,
         },
         llm=large_language_model,
     )
@@ -208,10 +212,16 @@ class DependencyContainer(DeclarativeContainer):
         langfuse_manager=langfuse_manager,
     )
 
+    language_detection_chain = Singleton(
+        LanguageDetectionChain,
+        langfuse_manager=langfuse_manager,
+    )
+
     chat_graph = Singleton(
         DefaultChatGraph,
         composed_retriever=composed_retriever,
         rephrasing_chain=rephrasing_chain,
+        language_detection_chain=language_detection_chain,
         mapper=information_piece_mapper,
         answer_generation_chain=answer_generation_chain,
         error_messages=error_messages,

libs/rag-core-api/src/rag_core_api/impl/answer_generation_chains/language_detection_chain.py

@@ -0,0 +1,128 @@
+"""Language detection chain.
+
+# Assumptions:
+# - We want a robust return of a 2-letter ISO 639-1 language code (lowercase),
+#   defaulting to "en" when unsure or parsing fails.
+# - We instruct the LLM to produce structured JSON and include format instructions,
+#   but we still parse defensively to handle non-conforming outputs.
+"""
+
+import json
+import re
+from typing import Any, Optional
+
+from langchain_core.output_parsers import StrOutputParser
+from langchain_core.runnables import Runnable, RunnableConfig
+
+from rag_core_api.impl.graph.graph_state.graph_state import AnswerGraphState
+from rag_core_lib.runnables.async_runnable import AsyncRunnable
+from rag_core_lib.impl.langfuse_manager.langfuse_manager import LangfuseManager
+from pydantic import BaseModel, field_validator
+from langchain_core.output_parsers import PydanticOutputParser
+
+
+RunnableInput = AnswerGraphState
+RunnableOutput = str
+
+
+from rag_core_api.utils.utils import (
+    strip_code_fences,
+    norm_lang,
+    extract_lang_from_parsed,
+    JSONISH_DQ_RE,
+    JSONISH_SQ_RE,
+    JSONISH_LOOSE_RE,
+)
+
+
+class LanguageDetectionChain(AsyncRunnable[RunnableInput, RunnableOutput]):
+    """Base class for language detection of the input question."""
+
+    def __init__(self, langfuse_manager: LangfuseManager):
+        """Initialize LanguageDetectionChain with LangfuseManager."""
+        self._langfuse_manager = langfuse_manager
+
+    async def ainvoke(
+        self, chain_input: RunnableInput, config: Optional[RunnableConfig] = None, **kwargs: Any
+    ) -> RunnableOutput:
+        """
+        Asynchronously detects the language and returns an ISO 639-1 code.
+
+        Returns
+        -------
+        RunnableOutput
+            Two-letter ISO 639-1 language code in lowercase (e.g., "en", "de").
+        """
+        raw = await self._create_chain().ainvoke(chain_input, config=config)
+        return self._extract_language_code(raw)
+
+    def _create_chain(self) -> Runnable:
+        # Provide format instructions to the prompt using a minimal Pydantic schema.
+        class _LangSchema(BaseModel):
+            language: str
+
+            @field_validator("language")
+            @classmethod
+            def two_letter_lower(cls, v: str) -> str:
+                v = v.strip().lower()
+                if not re.fullmatch(r"[a-z]{2}", v):
+                    raise ValueError("language must be a two-letter ISO 639-1 code")
+                return v
+
+        # We don't use PydanticOutputParser directly in the chain to avoid hard failures
+        # when the model doesn't follow instructions. Instead, we keep a string parser and
+        # parse defensively in ainvoke(). We still inject format instructions to guide the LLM.
+        try:
+            fmt_instructions = PydanticOutputParser(
+                pydantic_object=_LangSchema
+            ).get_format_instructions()
+        except Exception:
+            fmt_instructions = (
+                "Return a strict JSON object with a single field 'language' as a "
+                "lowercase two-letter ISO 639-1 code, e.g. {\"language\":\"de\"}."
+            )
+
+        prompt = self._langfuse_manager.get_base_prompt(self.__class__.__name__).partial(
+            format_instructions=fmt_instructions
+        )
+
+        return prompt | self._langfuse_manager.get_base_llm(self.__class__.__name__) | StrOutputParser()
+
+    @staticmethod
+    def _extract_language_code(raw_output: Any) -> str:
+        """Extract a two-letter ISO 639-1 language code; default to 'en' on failure."""
+
+        # 1) Already parsed structures
+        c = extract_lang_from_parsed(raw_output)
+        if c:
+            return c
+
+        # 2) Strings (common case)
+        if isinstance(raw_output, str):
+            text = raw_output.strip().lstrip("\ufeff")
+            text = strip_code_fences(text)
+
+            # a) Direct 'xx' or 'xx-YY'
+            c = norm_lang(text)
+            if c:
+                return c
+
+            # b) Try strict JSON parsing
+            try:
+                data = json.loads(text)
+                c = extract_lang_from_parsed(data)
+                if c:
+                    return c
+            except Exception:
+                pass
+
+            # c) JSON-ish fallbacks (handles single quotes, loose key/value, etc.)
+            for pattern in (JSONISH_DQ_RE, JSONISH_SQ_RE, JSONISH_LOOSE_RE):
+                m = pattern.search(text)
+                if m:
+                    c = norm_lang(m.group(1))
+                    if c:
+                        return c
+
+        # Fallback
+        return "en"

libs/rag-core-api/src/rag_core_api/impl/graph/chat_graph.py

@@ -20,6 +20,7 @@
     AnswerGenerationChain,
 )
 from rag_core_api.impl.answer_generation_chains.rephrasing_chain import RephrasingChain
+from rag_core_api.impl.answer_generation_chains.language_detection_chain import LanguageDetectionChain
 from rag_core_api.impl.graph.graph_state.graph_state import AnswerGraphState
 from rag_core_api.impl.retriever.no_or_empty_collection_error import (
     NoOrEmptyCollectionError,
@@ -71,6 +72,7 @@ def __init__(
         self,
         answer_generation_chain: AnswerGenerationChain,
         rephrasing_chain: RephrasingChain,
+        language_detection_chain: LanguageDetectionChain,
         composed_retriever: Retriever,
         mapper: InformationPieceMapper,
         error_messages: ErrorMessages,
@@ -100,6 +102,7 @@ def __init__(
         self._mapper = mapper
         self._chat_history_settings = chat_history_settings
         self._rephrasing_chain = rephrasing_chain
+        self._language_detection_chain = language_detection_chain
         self._error_messages = error_messages
         self._rephrase_node_builder = partial(self._rephrase_node)
         self._generate_node_builder = partial(self._generate_node)
@@ -200,7 +203,14 @@ def draw_graph(self, relative_dir_path: Optional[str] = None) -> None:
     #########
     async def _determine_language_node(self, state: dict, config: Optional[RunnableConfig] = None) -> dict:
         question = state["question"]
-        question_language = langdetect.detect(question)
+        # Prefer the LLM-based language detection; fallback to langdetect if needed inside the chain.
+        try:
+            question_language = await self._language_detection_chain.ainvoke(state, config=config)
+        except Exception:
+            try:
+                question_language = langdetect.detect(question)
+            except Exception:
+                question_language = "en"
         logger.debug('Detected langauge for question "%s": %s', question, question_language)
         return {"language": question_language}
 

libs/rag-core-api/src/rag_core_api/prompt_templates/language_detection_prompt.py

@@ -0,0 +1,23 @@
+from langchain.prompts import ChatPromptTemplate, SystemMessagePromptTemplate, HumanMessagePromptTemplate
+
+# Generic LangChain ChatPromptTemplate - works with any LLM
+LANGUAGE_DETECTION_PROMPT = ChatPromptTemplate.from_messages(
+    [
+        SystemMessagePromptTemplate.from_template(
+            """You are a helpful assistant that detects the language of the user's question.
+Return your answer as a strict JSON object with a single field 'language' containing the ISO 639-1 language code in lowercase.
+If you cannot determine the language with reasonable certainty, set 'language' to 'en'.
+
+{format_instructions}
+
+Examples (input -> output):
+- "What is the capital of Germany?" -> {{"language": "en"}}
+- "Was ist die Hauptstadt von Deutschland?" -> {{"language": "de"}}
+- "¿Cuál es la capital de Alemania?" -> {{"language": "es"}}
+- "Quelle est la capitale de l'Allemagne ?" -> {{"language": "fr"}}
+- "計算できません!!!" (ambiguous/gibberish) -> {{"language": "en"}}
+"""
+        ),
+        HumanMessagePromptTemplate.from_template("""Question: {question}"""),
+    ]
+)