Hi,
After the reading 4.7 and its subsection I think the explanation of short and long term memory is missing some clarification. I think its better to explain first type of data an agent can consume and then what considered as short term and what considered as long term.
From the section 4.7.1:
“Without short-term memory, the agent cannot resolve the second question because it has no context that “it” refers to the Eiffel Tower. In our agent implementation, short-term memory is implemented as message history that stores recent messages”
Excerpt From
Designing Multi-Agent Systems
Victor Dibia
This material may be protected by copyright.
But this statement create an intuition that any user and assistant turn for a history is a short term memory and stored locally. But reality is also each session is persisted with each user assistant turn into a database not locally. But From that N number of messages may be last [-10:] is used with new information (tool call or rag search etc.) which is working context only will be used to answer the next question. So tagging a history as short-term memory is miss leading from my perspective.
I know you did not explained memory this way, but reading these lines create a intuition that short term memory means user conversation history. Rather its better we can categorize the type of data or information and then can explain how a piece of information being end up into long or short term memory.
For example here is few example of data/information which an agent can encounter with:
- System Instruction
- Latest question is being asked
- Previous history
- Gathered information from external tool calls
- Gathered information from internal tool calls (enterprise data index as vector sore, structured data)
- Learned preferences while interacting with the user
- Like and Dislike
- User feedback when disliked
etc.
Then we can explain how they can end up into either long term or short term memory.
Here is a possible example how a conversation history can end up as short term memory and long term memory.
Turn 1 – User:
Hi, I’m planning a trip to Europe this year.
Turn 2 – Assistant:
That’s exciting! Which countries or cities are you considering?
Turn 3 – User:
I’m thinking about France and Italy, mainly major tourist cities.
Turn 4 – Assistant:
Great choices. Cities like Paris, Rome, and Florence are very popular. Do you have specific landmarks in mind?
Turn 5 – User:
Yes, I definitely want to see famous monuments and historical places.
Turn 6 – Assistant:
In France, people often visit landmarks like the Louvre, Notre-Dame, and the Eiffel Tower. In Italy, the Colosseum and Vatican are highlights.
Turn 7 – User:
I love photography, so iconic structures are very important to me.
Turn 8 – Assistant:
Then Paris would be excellent for you—especially for architectural and cityscape photography.
Turn 9 – User:
I’ve seen that tall iron structure so many times in pictures.
Turn 10 – User (Final Question):
Is that landmark located in the capital city of France?
Then we can choose last few turns to pass as context for the LLM to answer the question last question:
Turn 7 – User:
I love photography, so iconic structures are very important to me.
Turn 8 – Assistant:
Then Paris would be excellent for you—especially for architectural and cityscape photography.
Turn 9 – User:
I’ve seen that tall iron structure so many times in pictures.
Turn 10 – User:
Is that landmark located in the capital city of France?
Which explain this as short term memory.
Then we can produce long term memory from this conversation as facts:
Candidate1 (User Preference): User values iconic structures and photography when traveling.
Candidate2 (world knowledge): Eiffel Tower is located in Paris, France.
Then we can embeeded the text:
“The user enjoys photography and prioritizes iconic landmarks when choosing travel destinations.”
and store into vector store which become long term memory.
There is also challenges on Long term memory as Vector Index as query with top k will always pull some item from the vector index even with very low score which might not be relavant as context and how to tackle such usecase.
I am just providing my openions. If you do not like this way to send feedback you can let me know some other way to send the feedback.
I gaveup on hands on coding with the book as codes are not complete and there is no version tag in the git which I can move around to see the older implementation. I am reading the book like a literature knowledge source to understand the topics.
Hi,
After the reading 4.7 and its subsection I think the explanation of short and long term memory is missing some clarification. I think its better to explain first type of data an agent can consume and then what considered as short term and what considered as long term.
From the section 4.7.1:
But this statement create an intuition that any user and assistant turn for a history is a short term memory and stored locally. But reality is also each session is persisted with each user assistant turn into a database not locally. But From that N number of messages may be last [-10:] is used with new information (tool call or rag search etc.) which is working context only will be used to answer the next question. So tagging a history as short-term memory is miss leading from my perspective.
I know you did not explained memory this way, but reading these lines create a intuition that short term memory means user conversation history. Rather its better we can categorize the type of data or information and then can explain how a piece of information being end up into long or short term memory.
For example here is few example of data/information which an agent can encounter with:
etc.
Then we can explain how they can end up into either long term or short term memory.
Here is a possible example how a conversation history can end up as short term memory and long term memory.
Then we can choose last few turns to pass as context for the LLM to answer the question last question:
Which explain this as short term memory.
Then we can produce long term memory from this conversation as facts:
Then we can embeeded the text:
and store into vector store which become long term memory.
There is also challenges on Long term memory as Vector Index as query with top k will always pull some item from the vector index even with very low score which might not be relavant as context and how to tackle such usecase.
I am just providing my openions. If you do not like this way to send feedback you can let me know some other way to send the feedback.
I gaveup on hands on coding with the book as codes are not complete and there is no version tag in the git which I can move around to see the older implementation. I am reading the book like a literature knowledge source to understand the topics.