Backend

Overview

Backend is an umbrella module that encapsulates a unified way to work with the following functionalities:

Chat Models (ChatModel class)
Embedding Models (EmbeddingModel class)
Audio Models (coming soon)
Image Models (coming soon)

BeeAI framework’s backend is designed with a provider-based architecture, allowing you to switch between different AI service providers while maintaining a consistent API.

Supported in Python and TypeScript.

Supported providers

The following table depicts supported providers. Each provider requires specific configuration through environment variables. Ensure all required variables are set before initializing a provider.

Name	Chat	Embedding	Environment Variables
Ollama	✅	✅	OLLAMA_CHAT_MODEL OLLAMA_BASE_URL
OpenAI	✅	✅	OPENAI_CHAT_MODEL OPENAI_EMBEDDING_MODEL OPENAI_API_BASE OPENAI_API_KEY OPENAI_ORGANIZATION OPENAI_API_HEADERS
Watsonx	✅	✅	WATSONX_CHAT_MODEL WATSONX_API_KEY WATSONX_PROJECT_ID WATSONX_SPACE_ID WATSONX_TOKEN WATSONX_ZENAPIKEY WATSONX_URL WATSONX_REGION
Anthropic	✅	✅	ANTHROPIC_CHAT_MODEL ANTHROPIC_API_KEY ANTHROPIC_API_HEADERS
Groq	✅	✅	GROQ_CHAT_MODEL GROQ_EMBEDDING_MODEL GROQ_API_KEY
Amazon Bedrock	✅	✅	AWS_CHAT_MODEL AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_REGION AWS_API_HEADERS
Google Vertex	✅	✅	GOOGLE_VERTEX_CHAT_MODEL GOOGLE_VERTEX_PROJECT GOOGLE_APPLICATION_CREDENTIALS GOOGLE_APPLICATION_CREDENTIALS_JSON GOOGLE_CREDENTIALS GOOGLE_VERTEX_API_HEADERS
Azure OpenAI	✅	✅	AZURE_OPENAI_CHAT_MODEL AZURE_OPENAI_API_KEY AZURE_OPENAI_API_BASE AZURE_OPENAI_API_VERSION AZURE_AD_TOKEN AZURE_API_TYPE AZURE_API_HEADERS
xAI	✅	✅	XAI_CHAT_MODEL XAI_API_KEY
Google Gemini	✅	✅	GEMINI_CHAT_MODEL GEMINI_API_KEY GEMINI_API_HEADERS
MistralAI	✅	✅	MISTRALAI_CHAT_MODEL MISTRALAI_EMBEDDING_MODEL MISTRALAI_API_KEY MISTRALAI_API_BASE
Transformers	✅	✅	TRANSFORMERS_CHAT_MODEL HF_TOKEN

If you don’t see your provider raise an issue here. Meanwhile, you can use the Ollama for local models in Python or TypeScript or the Langchain adapter for hosted providers.

Google Gemini, MistralAI, and Transformers are supported in Python only. The Transformers chat model does not support tool calling.

Backend initialization

The Backend class serves as a central entry point to access models from your chosen provider. This example illustrate how to leverage the framework’s unified interface for different provider model operations by showcasing various interaction patterns including:

Basic chat completion
Streaming responses with abort functionality
Structured output generation
Real-time response parsing
Tool calling with external APIs
Text embedding generation

Explore more provider examples in Python or TypeScript

import asyncio
import datetime
import sys
import traceback

from pydantic import BaseModel, Field

from beeai_framework.adapters.openai import OpenAIChatModel, OpenAIEmbeddingModel
from beeai_framework.backend import (
    ChatModel,
    ChatModelNewTokenEvent,
    ChatModelParameters,
    MessageToolResultContent,
    ToolMessage,
    UserMessage,
)
from beeai_framework.emitter import EventMeta
from beeai_framework.errors import AbortError, FrameworkError
from beeai_framework.parsers.field import ParserField
from beeai_framework.parsers.line_prefix import LinePrefixParser, LinePrefixParserNode
from beeai_framework.tools.weather import OpenMeteoTool, OpenMeteoToolInput
from beeai_framework.utils import AbortSignal


async def openai_from_name() -> None:
    llm = ChatModel.from_name("openai:gpt-4.1-mini")
    user_message = UserMessage("what states are part of New England?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def openai_granite_from_name() -> None:
    llm = ChatModel.from_name("openai:gpt-4.1-mini")
    user_message = UserMessage("what states are part of New England?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def openai_sync() -> None:
    llm = OpenAIChatModel("gpt-4.1-mini")
    user_message = UserMessage("what is the capital of Massachusetts?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def openai_stream() -> None:
    llm = OpenAIChatModel("gpt-4.1-mini")
    user_message = UserMessage("How many islands make up the country of Cape Verde?")
    response = await llm.run([user_message], stream=True)
    print(response.get_text_content())


async def openai_stream_abort() -> None:
    llm = OpenAIChatModel("gpt-4.1-mini")
    user_message = UserMessage("What is the smallest of the Cape Verde islands?")

    try:
        response = await llm.run([user_message], stream=True, signal=AbortSignal.timeout(0.5))

        if response is not None:
            print(response.get_text_content())
        else:
            print("No response returned.")
    except AbortError as err:
        print(f"Aborted: {err}")


async def openai_structure() -> None:
    class TestSchema(BaseModel):
        answer: str = Field(description="your final answer")

    llm = OpenAIChatModel("gpt-4.1-mini")
    user_message = UserMessage("How many islands make up the country of Cape Verde?")
    response = await llm.run([user_message], response_format=TestSchema, stream=True)
    print(response.output_structured)


async def openai_stream_parser() -> None:
    llm = OpenAIChatModel("gpt-4.1-mini")

    parser = LinePrefixParser(
        nodes={
            "test": LinePrefixParserNode(
                prefix="Prefix: ", field=ParserField.from_type(str), is_start=True, is_end=True
            )
        }
    )

    async def on_new_token(data: ChatModelNewTokenEvent, event: EventMeta) -> None:
        await parser.add(data.value.get_text_content())

    user_message = UserMessage("Produce 3 lines each starting with 'Prefix: ' followed by a sentence and a new line.")
    await llm.run([user_message], stream=True).observe(lambda emitter: emitter.on("new_token", on_new_token))
    result = await parser.end()
    print(result)


async def openai_tool_calling() -> None:
    llm = ChatModel.from_name("openai:gpt-4.1-mini", ChatModelParameters(stream=True, temperature=0))
    user_message = UserMessage(f"What is the current weather in Boston? Current date is {datetime.datetime.today()}.")
    weather_tool = OpenMeteoTool()
    response = await llm.run([user_message], tools=[weather_tool])
    tool_call_msg = response.get_tool_calls()[0]
    print(tool_call_msg.model_dump())
    tool_response = await weather_tool.run(OpenMeteoToolInput(location_name="Boston"))
    tool_response_msg = ToolMessage(
        MessageToolResultContent(
            result=tool_response.get_text_content(),
            tool_name=weather_tool.name,
            tool_call_id=response.get_tool_calls()[0].id,
        )
    )
    print(tool_response_msg.to_plain())
    final_response = await llm.run([user_message, *response.output, tool_response_msg], tools=[])
    print(final_response.get_text_content())


async def openai_embedding() -> None:
    embedding_llm = OpenAIEmbeddingModel()

    response = await embedding_llm.create(["Text", "to", "embed"])

    for row in response.embeddings:
        print(*row)


async def openai_file_example() -> None:
    llm = ChatModel.from_name("openai:gpt-4.1-mini")
    data_uri = "data:application/pdf;base64,JVBERi0xLjQKMSAwIG9iago8PC9UeXBlIC9DYXRhbG9nCi9QYWdlcyAyIDAgUgo+PgplbmRvYmoKMiAwIG9iago8PC9UeXBlIC9QYWdlcwovS2lkcyBbMyAwIFJdCi9Db3VudCAxCj4+CmVuZG9iagozIDAgb2JqCjw8L1R5cGUgL1BhZ2UKL1BhcmVudCAyIDAgUgovTWVkaWFCb3ggWzAgMCA1OTUgODQyXQovQ29udGVudHMgNSAwIFIKL1Jlc291cmNlcyA8PC9Qcm9jU2V0IFsvUERGIC9UZXh0XQovRm9udCA8PC9GMSA0IDAgUj4+Cj4+Cj4+CmVuZG9iago0IDAgb2JqCjw8L1R5cGUgL0ZvbnQKL1N1YnR5cGUgL1R5cGUxCi9OYW1lIC9GMQovQmFzZUZvbnQgL0hlbHZldGljYQovRW5jb2RpbmcgL01hY1JvbWFuRW5jb2RpbmcKPj4KZW5kb2JqCjUgMCBvYmoKPDwvTGVuZ3RoIDUzCj4+CnN0cmVhbQpCVAovRjEgMjAgVGYKMjIwIDQwMCBUZAooRHVtbXkgUERGKSBUagpFVAplbmRzdHJlYW0KZW5kb2JqCnhyZWYKMCA2CjAwMDAwMDAwMDAgNjU1MzUgZgowMDAwMDAwMDA5IDAwMDAwIG4KMDAwMDAwMDA2MyAwMDAwMCBuCjAwMDAwMDAxMjQgMDAwMDAgbgowMDAwMDAwMjc3IDAwMDAwIG4KMDAwMDAwMDM5MiAwMDAwMCBuCnRyYWlsZXIKPDwvU2l6ZSA2Ci9Sb290IDEgMCBSCj4+CnN0YXJ0eHJlZgo0OTUKJSVFT0YK"

    file_message = UserMessage.from_file(file_data=data_uri, format="text")
    print(file_message.to_plain())
    response = await llm.run([UserMessage("Read content of the file."), file_message])
    print(response.get_text_content())


async def main() -> None:
    await openai_file_example()
    return

    print("*" * 10, "openai_from_name")
    await openai_from_name()
    print("*" * 10, "openai_granite_from_name")
    await openai_granite_from_name()
    print("*" * 10, "openai_sync")
    await openai_sync()
    print("*" * 10, "openai_stream")
    await openai_stream()
    print("*" * 10, "openai_stream_abort")
    await openai_stream_abort()
    print("*" * 10, "openai_structure")
    await openai_structure()
    print("*" * 10, "openai_stream_parser")
    await openai_stream_parser()
    print("*" * 10, "openai_tool_calling")
    await openai_tool_calling()
    print("*" * 10, "openai_embedding")
    await openai_embedding()


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except FrameworkError as e:
        traceback.print_exc()
        sys.exit(e.explain())

See the events documentation for more information on standard emitter events.

Chat model

The ChatModel class represents a Chat Language Model and provides methods for text generation, streaming responses, and more. You can initialize a chat model in multiple ways: Method 1: Using the from_name method

from beeai_framework.backend.chat import ChatModel

model = ChatModel.from_name("ollama:llama3.1")

Method 2: Directly specifying the provider class

from beeai_framework.adapters.ollama.backend.chat import OllamaChatModel

model = OllamaChatModel("llama3.1")

File / Document Inputs (PDF etc.)

You can attach files (e.g. PDFs) to a UserMessage using the MessageFileContent part or the convenience factory UserMessage.from_file. Provide either a remote file_id/URL or an inline base64 data URI (file_data). Optionally specify a MIME format.

Python

from beeai_framework.backend import UserMessage, MessageFileContent

# Using a remote / previously uploaded file URL or id (flattened API)
msg_with_file_id = UserMessage([
  MessageFileContent(
    file_id="https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
    format="application/pdf",
  ),
  "What's this file about?",
])

# Same using the factory helper
msg_with_file_id_factory = UserMessage.from_file(
  file_id="https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
  format="application/pdf",
)

# Using inline base64 data (shortened example)
msg_with_file_data = UserMessage([
  MessageFileContent(
    file_data="data:application/pdf;base64,AAA...",
    format="application/pdf",
  ),
  "Summarize the document",
])

# Inline base64 with factory
msg_with_file_data_factory = UserMessage.from_file(
  file_data="data:application/pdf;base64,AAA...",
  format="application/pdf",
)

These content parts serialize to the flattened schema (legacy nested { "file": {...} } removed):

{
  "type": "file",
  "file_id": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
  "format": "application/pdf"
}

If neither file_id nor file_data is supplied a validation error is raised.

Chat model configuration

You can configure various parameters for your chat model.

import asyncio
import sys
import traceback

from beeai_framework.adapters.ollama import OllamaChatModel
from beeai_framework.backend import UserMessage
from beeai_framework.errors import FrameworkError
from examples.helpers.io import ConsoleReader


async def main() -> None:
    llm = OllamaChatModel("llama3.1")

    #  Optionally one may set llm parameters
    llm.parameters.max_tokens = 10000  # high number yields longer potential output
    llm.parameters.top_p = 0.1  # higher number yields more complex vocabulary, recommend only changing p or k
    llm.parameters.frequency_penalty = 0  # higher number yields reduction in word reptition
    llm.parameters.temperature = 0  # higher number yields greater randomness and variation
    llm.parameters.top_k = 0  # higher number yields more variance, recommend only changing p or k
    llm.parameters.n = 1  # higher number yields more choices
    llm.parameters.presence_penalty = 0  # higher number yields reduction in repetition of words
    llm.parameters.seed = 10  # can help produce similar responses if prompt and seed are always the same
    llm.parameters.stop_sequences = ["q", "quit", "ahhhhhhhhh"]  # stops the model on input of any of these strings
    llm.parameters.stream = False  # determines whether or not to use streaming to receive incremental data

    reader = ConsoleReader()

    for prompt in reader:
        response = await llm.run([UserMessage(prompt)])
        reader.write("LLM 🤖 (txt) : ", response.get_text_content())
        reader.write("LLM 🤖 (raw) : ", "\n".join([str(msg.to_plain()) for msg in response.output]))


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except FrameworkError as e:
        traceback.print_exc()
        sys.exit(e.explain())

Text generation

The most basic usage is to generate text responses:

from beeai_framework.adapters.ollama.backend.chat import OllamaChatModel
from beeai_framework.backend.message import UserMessage

model = OllamaChatModel("llama3.1")
response = await model.create(
    messages=[UserMessage("what states are part of New England?")]
)

print(response.get_text_content())

Execution parameters (those passed to model.create({...})) take precedent over ones defined via config.

Streaming responses

For applications requiring real-time responses:

from beeai_framework.adapters.ollama.backend.chat import OllamaChatModel
from beeai_framework.backend.message import UserMessage

llm = OllamaChatModel("llama3.1")
user_message = UserMessage("How many islands make up the country of Cape Verde?")
response = await llm.create(messages=[user_message], stream=True)
  .on(
    "new_token",
    lambda data, event: print(data.value.get_text_content()))
  )
)
print("Full response", response.get_text_content())

Structured generation

Generate structured data according to a schema:

import asyncio
import json
import sys
import traceback

from pydantic import BaseModel, Field

from beeai_framework.backend import ChatModel, UserMessage
from beeai_framework.errors import FrameworkError


async def main() -> None:
    model = ChatModel.from_name("ollama:llama3.1")

    class ProfileSchema(BaseModel):
        first_name: str = Field(..., min_length=1)
        last_name: str = Field(..., min_length=1)
        address: str
        age: int
        hobby: str

    class ErrorSchema(BaseModel):
        error: str

    class SchemUnion(ProfileSchema, ErrorSchema):
        pass

    response = await model.run(
        [UserMessage("Generate a profile of a citizen of Europe.")],
        response_format=SchemUnion,
    )
    assert isinstance(response.output_structured, ProfileSchema)
    print(json.dumps(response.output_structured.model_dump(), indent=4))


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except FrameworkError as e:
        traceback.print_exc()
        sys.exit(e.explain())

Tool calling

Integrate external tools with your AI model:

import asyncio
import json
import re
import sys
import traceback

from beeai_framework.backend import (
    AnyMessage,
    ChatModel,
    ChatModelParameters,
    MessageToolResultContent,
    SystemMessage,
    ToolMessage,
    UserMessage,
)
from beeai_framework.errors import FrameworkError
from beeai_framework.tools import AnyTool, ToolOutput
from beeai_framework.tools.search.duckduckgo import DuckDuckGoSearchTool
from beeai_framework.tools.weather.openmeteo import OpenMeteoTool


async def main() -> None:
    model = ChatModel.from_name("ollama:llama3.1", ChatModelParameters(temperature=0))
    tools: list[AnyTool] = [DuckDuckGoSearchTool(), OpenMeteoTool()]
    messages: list[AnyMessage] = [
        SystemMessage("You are a helpful assistant. Use tools to provide a correct answer."),
        UserMessage("What's the fastest marathon time?"),
    ]

    while True:
        response = await model.run(
            messages,
            tools=tools,
        )

        tool_calls = response.get_tool_calls()
        messages.extend(response.output)

        tool_results: list[ToolMessage] = []

        for tool_call in tool_calls:
            print(f"-> running '{tool_call.tool_name}' tool with {tool_call.args}")
            tool: AnyTool = next(tool for tool in tools if tool.name == tool_call.tool_name)
            assert tool is not None
            res: ToolOutput = await tool.run(json.loads(tool_call.args))
            result = res.get_text_content()
            print(f"<- got response from '{tool_call.tool_name}'", re.sub(r"\s+", " ", result)[:256] + " (truncated)")
            tool_results.append(
                ToolMessage(
                    MessageToolResultContent(
                        result=result,
                        tool_name=tool_call.tool_name,
                        tool_call_id=tool_call.id,
                    )
                )
            )

        messages.extend(tool_results)

        answer = response.get_text_content()

        if answer:
            print(f"Agent: {answer}")
            break


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except FrameworkError as e:
        traceback.print_exc()
        sys.exit(e.explain())

Embedding model

The EmbedingModel class provides functionality for generating vector embeddings from text.

Embedding model initialization

You can initialize an embedding model in multiple ways: Method 1: Using the from_name method

from beeai_framework.backend.embedding import EmbeddingModel

model = EmbeddingModel.from_name("ollama:nomic-embed-text")

Method 2: Directly specifying the provider class

from beeai_framework.adapters.ollama.backend import OllamaEmbeddingModel

model = OllamaEmbeddingModel("nomic-embed-text")

Embedding model usage

Generate embeddings for one or more text strings:

from beeai_framework.backend.embedding import EmbeddingModel

model = EmbeddingModel.from_name("ollama:nomic-embed-text")

response = await model.create(["Hello world!", "Hello Bee!"])
console.log(response.values)
console.log(response.embeddings)

Adding a Provider Using the LangChain Adapter

If your preferred provider isn’t directly supported, you can use the LangChain adapter as a bridge as long as that provider has LangChain compatibility.

import asyncio
import json
import sys
import traceback
from datetime import UTC, datetime

from pydantic import BaseModel, Field

from beeai_framework.adapters.langchain.backend.chat import LangChainChatModel
from beeai_framework.backend import (
    AnyMessage,
    ChatModelNewTokenEvent,
    MessageToolResultContent,
    SystemMessage,
    ToolMessage,
    UserMessage,
)
from beeai_framework.emitter import EventMeta
from beeai_framework.errors import AbortError, FrameworkError
from beeai_framework.parsers.field import ParserField
from beeai_framework.parsers.line_prefix import LinePrefixParser, LinePrefixParserNode
from beeai_framework.tools.weather import OpenMeteoTool
from beeai_framework.utils import AbortSignal

# prevent import error for langchain_ollama (only needed in this context)
cur_dir = sys.path.pop(0)
while cur_dir in sys.path:
    sys.path.remove(cur_dir)

from langchain_ollama.chat_models import ChatOllama as LangChainOllamaChat  # noqa: E402


async def langchain_ollama_from_name() -> None:
    langchain_llm = LangChainOllamaChat(model="granite3.3:8b")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("what states are part of New England?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def langchain_ollama_granite_from_name() -> None:
    langchain_llm = LangChainOllamaChat(model="granite3.3:8b")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("what states are part of New England?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def langchain_ollama_sync() -> None:
    langchain_llm = LangChainOllamaChat(model="granite3.3:8b")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("what is the capital of Massachusetts?")
    response = await llm.run([user_message])
    print(response.get_text_content())


async def langchain_ollama_stream() -> None:
    langchain_llm = LangChainOllamaChat(model="granite3.3:8b")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("How many islands make up the country of Cape Verde?")
    response = await llm.run([user_message], stream=True)
    print(response.get_text_content())


async def langchain_ollama_stream_abort() -> None:
    langchain_llm = LangChainOllamaChat(model="granite3.3:8b")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("What is the smallest of the Cape Verde islands?")

    try:
        response = await llm.run([user_message], stream=True, signal=AbortSignal.timeout(0.5))

        if response is not None:
            print(response.get_text_content())
        else:
            print("No response returned.")
    except AbortError as err:
        print(f"Aborted: {err}")


async def langchain_ollama_structure() -> None:
    class TestSchema(BaseModel):
        answer: str = Field(description="your final answer")

    langchain_llm = LangChainOllamaChat(model="granite3.3:8b")
    llm = LangChainChatModel(langchain_llm)
    user_message = UserMessage("How many islands make up the country of Cape Verde?")
    response = await llm.run([user_message], response_format=TestSchema)
    print(response.output_structured)


async def langchain_ollama_stream_parser() -> None:
    langchain_llm = LangChainOllamaChat(model="granite3.3:8b")
    llm = LangChainChatModel(langchain_llm)

    parser = LinePrefixParser(
        nodes={
            "test": LinePrefixParserNode(
                prefix="Prefix: ", field=ParserField.from_type(str), is_start=True, is_end=True
            )
        }
    )

    async def on_new_token(data: ChatModelNewTokenEvent, event: EventMeta) -> None:
        await parser.add(data.value.get_text_content())

    user_message = UserMessage("Produce 3 lines each starting with 'Prefix: ' followed by a sentence and a new line.")
    await llm.run([user_message], stream=True).observe(lambda emitter: emitter.on("new_token", on_new_token))
    result = await parser.end()
    print(result)


async def langchain_ollama_tool_calling() -> None:
    langchain_llm = LangChainOllamaChat(model="granite3.3:8b")
    llm = LangChainChatModel(langchain_llm)
    llm.parameters.stream = True
    weather_tool = OpenMeteoTool()
    messages: list[AnyMessage] = [
        SystemMessage(
            f"""You are a helpful assistant that uses tools to provide answers.
Current date is {datetime.now(tz=UTC).date()!s}
"""
        ),
        UserMessage("What is the current weather in Berlin?"),
    ]
    response = await llm.run(messages, tools=[weather_tool], tool_choice="required")
    messages.extend(response.output)
    tool_call_msg = response.get_tool_calls()[0]
    print(tool_call_msg.model_dump())
    tool_response = await weather_tool.run(json.loads(tool_call_msg.args))
    tool_response_msg = ToolMessage(
        MessageToolResultContent(
            result=tool_response.get_text_content(), tool_name=tool_call_msg.tool_name, tool_call_id=tool_call_msg.id
        )
    )
    print(tool_response_msg.to_plain())
    final_response = await llm.run([*messages, tool_response_msg], tools=[])
    print(final_response.get_text_content())


async def main() -> None:
    print("*" * 10, "langchain_ollama_from_name")
    await langchain_ollama_from_name()
    print("*" * 10, "langchain_ollama_granite_from_name")
    await langchain_ollama_granite_from_name()
    print("*" * 10, "langchain_ollama_sync")
    await langchain_ollama_sync()
    print("*" * 10, "langchain_ollama_stream")
    await langchain_ollama_stream()
    print("*" * 10, "langchain_ollama_stream_abort")
    await langchain_ollama_stream_abort()
    print("*" * 10, "langchain_ollama_structure")
    await langchain_ollama_structure()
    print("*" * 10, "langchain_ollama_stream_parser")
    await langchain_ollama_stream_parser()
    print("*" * 10, "langchain_ollama_tool_calling")
    await langchain_ollama_tool_calling()


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except FrameworkError as e:
        traceback.print_exc()
        sys.exit(e.explain())

Troubleshooting

Common issues and their solutions:

Authentication errors: Ensure all required environment variables are set correctly
Model not found: Verify that the model ID is correct and available for the selected provider

Introduction

Core Concepts

Integrations

Guides

Community

Overview

Supported providers

Backend initialization

Chat model

File / Document Inputs (PDF etc.)

Chat model configuration

Text generation

Streaming responses

Structured generation

Tool calling

Embedding model

Embedding model initialization

Embedding model usage

Adding a Provider Using the LangChain Adapter

Troubleshooting

Examples

Python

TypeScript

Introduction

Core Concepts

Integrations

Guides

Community

​Overview

​Supported providers

​Backend initialization

​Chat model

​File / Document Inputs (PDF etc.)

​Chat model configuration

​Text generation

​Streaming responses

​Structured generation

​Tool calling

​Embedding model

​Embedding model initialization

​Embedding model usage

​Adding a Provider Using the LangChain Adapter

​Troubleshooting

​Examples

Python

TypeScript

Overview

Supported providers

Backend initialization

Chat model

File / Document Inputs (PDF etc.)

Chat model configuration

Text generation

Streaming responses

Structured generation

Tool calling

Embedding model

Embedding model initialization

Embedding model usage

Adding a Provider Using the LangChain Adapter

Troubleshooting

Examples