What's a Silo?
This entity is designed to store and retrieve chunks, the result of embeddings of documents. It is the basic building block of dRAGon.
A Silo is composed of three main components :
How does it work?
The Ingestor Loader is responsible for loading the documents into the Silo : it can be done by reading files in a directory or by providing URLs.
After the documents are loaded they are split into chunks (eg. paragraphs, sentences, etc.) by the Silo Splitter.
Each chunk is then transformed into a vector by the Embedding Model, which is then stored in the Vector Store for later retrieval.
Diagrams
VectorStoreSettings
PersistInMemoryEmbeddingStoreSettings
PGVectorEmbeddingStoreSettings
EmbeddingSettings
IngestorSettings
DefaultIngestorLoaderSettings
Last updated
