How handle history with extern API.

[Malagurti]

Issue: LLM Not Accessing History Through Function

I'm facing issues when using history with the score endpoint. The function provides the history for the endpoint, but the LLM can't access the information. When we test the app in multimodal mode, it works fine. The issue only occurs when we pass the history through the function.

Has anyone faced the same issue or has any ideas on how I could fix this?

---

JSON Sent by the Function to the Score Endpoint

{
    "question": "Faça uma simulação para Jose",
    "chat_history": [
        {
            "inputs": {
                "question": "Faça uma simulação para Jose"
            },
            "outputs": {
                "answer": "Ei, só para confirmar, está faltando: Data de nascimento ou idade e Profissão."
            }
        },
        {
            "inputs": {
                "question": "Faça uma simulação para Jose que é Caminhoneiro e nasceu em 12 de jan de 88"
            },
            "outputs": {
                "answer": "Aqui estão as informações do seu futuro seguro de vida: Perfil do Cliente e Produtos Selecionados."
            }
        }
    ],
    "context": {
        "usuario": {
            "cpf": "XXXXXXXXXXX",
            "nome": "Nome do Usuário",
            "codigo": "XXXXXX"
        }
    }
}

[KeepALifeUS]

The issue is likely how the history is being formatted for the LLM. Here's what typically works:

Check the Message Format

The LLM expects messages in a specific format:

messages = [
    {"role": "user", "content": "Previous question"},
    {"role": "assistant", "content": "Previous answer"},
    {"role": "user", "content": "Current question"}
]

If your external API is passing history differently, the LLM might not recognize it as conversation context.

Common Issues

1. History as a string — LLMs need structured messages, not concatenated text
2. Wrong role names — Must be "user"/"assistant", not custom names
3. History in wrong position — Must come before the current message

Alternative: External State

If the API makes message formatting difficult, consider managing history externally:

# Store history in your own state
state = {
    "conversation_id": "abc123",
    "history": [...],
    "current_query": "..."
}

# Build messages properly before calling LLM
messages = build_messages_from_state(state)

For PromptFlow

Check if your flow is:

1. Properly converting the history JSON to the expected message format
2. Including history in the messages array, not as a separate parameter
3. Using the correct role values

Could you share how the messages look when they reach the LLM?