Skip to main content

​
Overview

Retrieval-Augmented Generation (RAG) is a powerful paradigm that enhances large language models by providing them with relevant information from external knowledge sources. This approach has become essential for enterprise AI applications that need to work with specific, up-to-date, or domain-specific information that wasn’t part of the model’s training data. RAG addresses key limitations of traditional LLMs:
  • Knowledge cutoffs - Access the most current information
  • Domain expertise - Integrate specialized knowledge bases
  • Factual accuracy - Reduce hallucinations with grounded responses
  • Scalability - Work with vast document collections efficiently
Enterprises rely on RAG for applications like customer support, document analysis, knowledge management, and intelligent search systems.
Location within the framework: beeai_framework/rag.
RAG is most effective when document chunking and retrieval strategies are tailored to your specific problem domain. It’s recommended to experiment with different configurations such as chunk sizes, overlap settings, and retrieval parameters. Future releases of BeeAI will provide enhanced capabilities to streamline this optimization process.

​
Philosophy

BeeAI Framework’s approach to RAG emphasizes integration over invention. Rather than building RAG components from scratch, we provide seamless adapters for proven, production-ready solutions from leading platforms like LangChain and Llama-Index. This philosophy offers several advantages:
  • Leverage existing expertise - Use battle-tested implementations
  • Faster time-to-market - Focus on your application logic, not infrastructure
  • Community support - Benefit from extensive documentation and community
  • Flexibility - Switch between providers as needs evolve

​
Installation

To use RAG components, install the framework with the RAG extras: 
pip install "beeai-framework[rag]"

​
RAG Components

The following table outlines the key RAG components available in the BeeAI Framework:
ComponentDescriptionCompatibilityFuture Compatibility
Document LoadersResponsible for loading content from different formats and sources such as PDFs, web pages, and structured text filesLangChainBeeAI
Text SplittersSplits long documents into workable chunks using various strategies, e.g. fixed length or preserving contextLangChainBeeAI
DocumentThe basic data structure to house text content, metadata, and relevant scores for retrieval operationsBeeAI-
Vector StoreUsed to store document embeddings and retrieve them based on semantic similarity using embedding distanceLangChainBeeAI, Llama-Index
Document ProcessorsUsed to process and refine documents during the retrieval-generation lifecycle including reranking and filteringLlama-Index-

​
Dynamic Module Loading

BeeAI Framework provides a dynamic module loading system that allows you to instantiate RAG components using string identifiers. This approach enables configuration-driven architectures and easy provider switching. The from_name method uses the format provider:ClassName where:
  • provider identifies the integration module (e.g., “beeai”, “langchain”)
  • ClassName specifies the exact class to instantiate
Dynamic loading enables you to switch between different vector store implementations without changing your application code - just update the configuration string.

​
BeeAI Vector Store

Python
import asyncio
import sys
import traceback

from beeai_framework.adapters.beeai.backend.vector_store import TemporalVectorStore
from beeai_framework.adapters.langchain.mappers.documents import lc_document_to_document
from beeai_framework.backend.embedding import EmbeddingModel
from beeai_framework.backend.vector_store import VectorStore
from beeai_framework.errors import FrameworkError

# LC dependencies - to be swapped with BAI dependencies
try:
    from langchain_community.document_loaders import UnstructuredMarkdownLoader
    from langchain_text_splitters import RecursiveCharacterTextSplitter
except ModuleNotFoundError as e:
    raise ModuleNotFoundError(
        "Optional modules are not found.\nRun 'pip install \"beeai-framework[rag]\"' to install."
    ) from e


async def main() -> None:
    embedding_model = EmbeddingModel.from_name("watsonx:ibm/slate-125m-english-rtrvr-v2", truncate_input_tokens=500)

    # Document loading
    loader = UnstructuredMarkdownLoader(file_path="docs/modules/agents.mdx")
    docs = loader.load()
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=1000)
    all_splits = text_splitter.split_documents(docs)
    documents = [lc_document_to_document(document) for document in all_splits]
    print(f"Loaded {len(documents)} documents")

    vector_store: TemporalVectorStore = VectorStore.from_name(
        name="beeai:TemporalVectorStore", embedding_model=embedding_model
    )  # type: ignore[assignment]
    _ = await vector_store.add_documents(documents=documents)


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except FrameworkError as e:
        traceback.print_exc()
        sys.exit(e.explain())
Native BeeAI modules can be loaded directly by importing and instantiating the module, e.g. from beeai_framework.adapters.beeai.backend.vector_store import TemporalVectorStore.

​
Supported Provider’s Vector Store

# LangChain integration
vector_store = VectorStore.from_name(
    name="langchain:InMemoryVectorStore",
    embedding_model=embedding_model
)
For production deployments, consider implementing document caching and index optimization to improve response times.

​
RAG as Tools

Vector store population (loading and chunking documents) is typically handled offline in production applications, making Vector Store the prominent RAG building block utilized as a tool. VectorStoreSearchTool enables any agent to perform semantic search against a pre-populated vector store. This provides flexibility for agents that need retrieval capabilities alongside other functionalities.
The VectorStoreSearchTool can be dynamically instantiated using VectorStoreSearchTool.from_vector_store_name("beeai:TemporalVectorStore", embedding_model=embedding_model), see RAG with RequirementAgent example for the full code.

​
Examples

Python RAG Agent

Complete RAG agent implementation with document loading and processing

RAG with RequirementAgent

Example showing how to use VectorStoreSearchTool with RequirementAgent for RAG capabilities