Skip to content

Bug_192_EVALUATE: CHAT-L3-QA-002 — Event log records original message, DB saves effective message — inconsistent on attachments #413

@steadhac

Description

@steadhac

Component: finbot/agents/chat.py → ChatAssistantBase.stream_response (lines 296, 449)

Root cause:

stream_response builds effective_message by prepending a FinDrive file reference
prefix when attachments are present. The DB record uses effective_message (with the
prefix). The response_complete event emitted to Redis uses user_message (the
original, without the prefix):

# line 296 — message_received event
"user_message": user_message,   # original

# line 449 — response_complete event
"user_message": user_message,   # original — should be effective_message

Steps to reproduce:

  1. Send a message with one attachment: message="Review this",
    attachments=[{"filename": "invoice.pdf", "file_id": "fd_001"}].
  2. Capture the response_complete event emitted to the event bus.
  3. Compare event_data["user_message"] against the value saved to the DB.

Expected: event_data["user_message"] ==
"[User attached FinDrive files: invoice.pdf (file_id: fd_001)]\n\nReview this"
Actual: event_data["user_message"] == "Review this"

How to execute:

pytest tests/integration/agents/test_chat_layer3.py::TestL3QAFindings::test_chat_l3_qa_002_event_data_logs_original_message_not_effective -v

Proposed fix: Replace user_message with effective_message in the
response_complete event payload on line 449.

Impact: The event log and the DB record are inconsistent for any session where
attachments were sent. A compliance audit that compares event logs to DB records will
find a mismatch. Debugging a session replay using event logs will show a different
message than what was actually processed by the LLM, making root-cause analysis
unreliable.

Acceptance criteria:

  • test_chat_l3_qa_002_event_data_logs_original_message_not_effective passes
  • response_complete event user_message matches effective_message when attachments present
  • No change to message_received event (that correctly logs user_message)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions