When you talk to an AI assistant, it may be as if you missed the previous things.
But large language models (LLM) do not actually have memory themselves. They do not remember the conversation unless they are given the information again.
So, how do they miss things?
The answer is something that is called a vector store – and will learn about you in this article.
The table of contents
What is a vector store?
A vector store is a special type of database. Instead of storing regular databases such as text or numbers, it stores the vector.
There is a list of vector numbers that represents the meaning of a piece of text. You get these vectors using a process called embedid.
The model takes a phrase and turns it into a high -dimensional point. In this place, similar meanings are close to each other.
For example, if I embed to “Sushi”, it can be close to “Sushi is my favorite food” in the vector instead of the vector. This embedded AI agent helps to find ideas related to the AI agent, even if the exact words are different.
How does the embedded work work
We say that the user tells an assistant:
“I live in Austin, Texas.”
The model turns this phrase into a vector:
(0.23, -0.41, 0.77, ..., 0.08)
This vector does not mean much for us, but for AI, this is a way to achieve the meaning of the sentence. The vector vector is stored in the database, as well as some additional information – that the time stamp or a note that comes from this user.
Later, if the user says:
“Book a flight to my hometown.”
The model transforms this new phrase into a new vector. It then searches the vector database to find similar stored vectors.
The nearest match can be “I live in Austin, Texas.” Now AI knows what you may mean by “my hometown”.
This ability to find the inputs of the past based on the meaning – not only is the keyword similar – that is the one that gives the LLM a form of memory.
Why are vector stores important for memory
The language of the LLMS process using the context window. This is the text in which they can “see” together.
For GPT -4 Turbo, the window can handle up to 128,000 tokens, which looks huge -but even it is full. You cannot keep the whole conversation forever.
Instead, you use the vector store as a long -term memory. You save embedded and useful information.
Then, when needed, you ask the vector store, recover the related pieces above, and feed them back to the LLM. Thus, the model remembers enough to work smart-keep everything in its short-term memory.
Popular vector stores
There are several famous vector databases in use. Everyone has their own strength.
FAST (Facebook AI searching for matching)
Foss is an open source library developed by the meta. It is fast and works well for local or on -premises applications.
If you want full control and do not need cloud hosting, then the foss is great. It supports millions of vectors and provides index and search tools tools with high performance.
This is how you can use the fossils:
from sentence_transformers import SentenceTransformer
import faiss
import numpy as np
# Load a pre-trained sentence transformer model that converts sentences to numerical vectors (embeddings)
model = SentenceTransformer('all-MiniLM-L6-v2')
# Define the input sentence we want to store in memory
sentence = "User lives in Austin, Texas"
# Convert the sentence into a dense vector (embedding)
embedding = model.encode(sentence)
# Get the dimensionality of the embedding vector (needed to create the FAISS index)
dimension = embedding.shape(0)
# Create a FAISS index for L2 (Euclidean) similarity search using the embedding dimension
index = faiss.IndexFlatL2(dimension)
# Add the sentence embedding to the FAISS index (this is our "memory")
index.add(np.array((embedding)))
# Encode a new query sentence that we want to match against the stored memory
query = model.encode("Where is the user from?")
# Search the FAISS index for the top-1 most similar vector to the query
D, I = index.search(np.array((query)), k=1)
# Print the index of the most relevant memory (in this case, only one item in the index)
print("Most relevant memory index:", I(0)(0))
This code has been used to convert phrases such as “User Austin, Texas” into embellishment.
It saves this embellishness in the Fas Index. When you ask a question like “Where is the user from?” , The code converts this question to another embedded and finds the index to find stored phrases that are the most similar in meaning.
Finally, it prints the most relevant sentence position (index) in memory.
Fuus is effective, but it has not been hosted. This means that you need to manage your infrastructure.
Panicone
Pancon is a local vector database from a cloud. It is organized for you, which makes it the best of the production system.
You do not have to worry about scaling or maintaining servers. Pinkan handles billions of vector and offers filtering, metad data support, and fast questions. It is well connected with tools like Langchen and Open.
How does a basic pancone setup work here:
import pinecone
from sentence_transformers import SentenceTransformer
# Initialize Pinecone with your API key and environment
pinecone.init(api_key="your-api-key", environment="us-west1-gcp")
# Connect to or create a Pinecone index named "memory-store"
index = pinecone.Index("memory-store")
# Load a pre-trained sentence transformer model to convert text into embeddings
model = SentenceTransformer('all-MiniLM-L6-v2')
# Convert a fact/sentence into a numerical embedding (vector)
embedding = model.encode("User prefers vegetarian food")
# Store (upsert) the embedding into Pinecone with a unique ID
index.upsert((("user-pref-001", embedding.tolist())))
# Encode the query sentence into an embedding
query = model.encode("What kind of food does the user like?")
# Search Pinecone to find the most relevant stored embedding for the query
results = index.query(queries=(query.tolist()), top_k=1)
# Print the ID of the top matching memory
print("Top match ID:", results('matches')(0)('id'))
Pancone is ideal if you want to scale and use it without having to manage hardware.
Other famous vector stores include:
weveate – connects vector search with the graph of knowledge. Hybrid offers a strong meaningful search with key word support.
Chrome – Easy to use and good for prototying. Often used in personal apps or demo.
qdrant -Made to search for high -performance vector with open source and filtering.
Each of them has its own place, depending on whether you need speed, scale, simplicity, or special features.
Finding to be smart with a generation of recovery to AI
This whole system-usure inputs are embedded, storing them in vector database, and later recovering them.
The AI still does not have a brain, but it can work as if it is. You choose what to remember, when to remember it, and how to feed it in the conversation.
If the AI helps the user in the track project updates, you can store each project description as a vector. When the user later asks, “What is the design of the design phase?” You find your memory database, take highly relevant notes, and let the LLM stitch them in a helpful response.
The limits of memory based on vector
Although vector stores provide AI agents a powerful way to imitate memory, this approach comes with some important limits.
The search for vector is based on the match, not the correct understanding. This means that the most stored embedids may not always be the most relevant or helpful in the context. For example, two sentences can be close to mathematics in the vector space but have very different meanings. As a result, AI can sometimes level confusion or topic results, especially when a newborn or emotional accent involves.
Another challenge is that embellishments are stable snap shots. Once stored, they are not ready or compromised unless it is clearly updated. If a user changes his mind or provides new information, the system “cannot learn” until the original vector is removed or changed. Unlike human memory, which makes itself a shield and improves over time, the vector -based memory is frozen until the developer is actively managing it.
There are some ways you can reduce these challenges.
One is to add more contexts to the recovery process, such as a time stamp, titles, or user’s intentions such as filtering through metadata. This helps to narrow the results, which is really relevant right now.
Another point of view is to re -process or regenerate old memories from time to time, making sure that the information reflects the recent understanding of user needs or preferences.
Beyond the technical boundaries, vector stores also give rise to privacy and moral concerns. Key questions are: Who decides what is saved? How long should this memory be maintained? And is the user’s control over the one who remembers or forgets?
For example, these decisions should not be fully made by the developer or the system. Explaining the memory is another thought. Let users select what you miss. For example, by marking some inputs as “important”, it involves a layer of consent and transparency. Similarly, the time to maintain memory should be appropriate where appropriate, the expiry policies are based on how long the information is useful.
It is equally important for users to see, manage or delete their stored data. Whether through a simple interface or programming API, memory management tools are essential to the essential of confidence. As the use of vector stores spreads, it is also expected that the AI system will respect the user agency and privacy.
The wider AI community is still forming the best ways of these issues. But one thing is clear: artificial memory should not only be designed for accuracy and performance, but also accountability. By combining strong defaults with user control, developers can ensure that the vector -based memory system is both smart and responsible.
Conclusion
Vector stores provide AI agents a way of fake memory – and they do it well. By embedded the text in vector and using tools like FAES or Pinks, we give the strength to remember the models what is important. This is not real memory. But this makes the AI system feel more personal, more helpful and more human.
Since these tools are more advanced, so does the illusion. But behind every smart AI, there is a simple system of vector and similarities. If you can master it, you can create assistants who can remember, learn and improve over time.
Hope you enjoy this article. Linked to contact me on them.