Long-term Memory for AI Agents

Why Vector Databases are insufficient for Memory Management of Agentic AI Systems?

Fig. Agentic AI Memory Management (Image by Author)

1. Introduction to Agentic AI Systems

AI agents are the current hype. I have written about them, and others are also discussing them. Overall, however, it does mean that there is a lot of confusion reg. what are agentic AI systems? How are they different from generative AI (Gen AI), or large language model (LLM) agents?

Fig. 1: Agentic AI evolution (Image by Author)

In this section, we try to bring some clarity to this debate by highlighting the functional / non-functional requirements of an agentic AI system with respect to implementing an actual marketing use-case — illustrated in Fig. 2.

Fig. 2: Agentic AI execution of a Marketing use-case (Image by Author)

Given a user task, the goal of an agent platform is to identify an agent (group of agents) capable to executing that task. So the first component we need is an orchestration layer capable of decomposing a task into sub-tasks, with execution of the resp. agents orchestrated by an orchestration engine.

As of today, we prompt an LLM for the task decomposition. So this is the overlap with Gen AI. Unfortunately, this also means that agentic AI today is limited by the reasoning capabilities of large language models (LLMs).

For ex., the GPT4 task decomposition of the prompt

Generate a tailored email campaign to achieve sales of USD 1 Million in 1 month, The applicable products and their performance metrics are available at [url]. Connect to CRM system [integration] for customer names, email addresses, and demographic details.

is detailed in Fig. 2: (Analyze products) — (Identify target audience) — (Create tailored email campaign).

It then monitors the execution / environment and adapts autonomously as needed. In this case, the agent realised that it is not going to achieve its sales goal and autonomously added the tasks: (Find alternative products) — (Utilize customer data) — (Perform A/B testing).

It is also important to mention that integration with enterprise systems (e.g., CRM in this case) will be needed for most use-cases. For example, refer to the Model Context Protocol (MCP) proposed by Anthropic recently to connect AI agents to external systems where enterprise data resides.

Given the long-running nature of such complex tasks, memory management is key for Agentic AI systems. Once the initial email campaign is launched, the agent needs to monitor the campaign for 1-month.

This entails both context sharing between tasks and maintaining execution context over long periods.

The current solution is to use vector databases (Vector DBs) to store the agent memory externally — making data items accessible as needed. In the sequel, we deep dive into the details of

• how agentic memory is managed using Vector DBs,

• their corresponding data quality issues.

We then show that vector databases (while sufficient for conversational memory — Q&A pairs) are insufficient for agentic tasks given their need to manage additional memory types:

• semantic memory (general knowledge),

• episodic memory (personal experiences),

• procedural memory (skills and task procedures),

• emotional memory (feelings tied to experiences);

and highlight the need for alternative formalisms (e.g., knowledge graphs, finite state machines) to manage the same effectively.

2. Conversational Memory Management using Vector DBs