Modules
The following section describes essential modules that are important to understand the whole concept.Event
An event refers to an action created by some component.- Has a name (e.g.,
start,success,error, …,fetch_data). - Has a data payload (typically a Pydantic model).
- Has associated metadata about the context where the event was fired.
Python
Emitter
The core component through which events are emitted and observed. An emitter is typically attached to some class but can be used alone. An emitter instance is typically the child of a root emitter to which all events are propagated. Emitters can be nested (one can be a child of another), hence they internally create a tree hierarchy. The emitter instance has the following properties:namespacein which the emitter operates (eg:agents.requirement,tool.open_meteo, …).creatorclass which the given emitter belongs to.context(dictionary which is attached to all events emitted via the given emitter).tracemetadata (such as currentid,run_idandparent_id)
onfor registering a new event listener- The method takes
matcher(event name, callback, regex),callback(sync/async function), andoptions(priority, etc.) - The method can be used as a decorator or as a standalone function
- The method takes
offfor deregistering an event listenerpipefor propagating all captured events to another emitterchildfor creating a child emitter
The event’s
path attribute is created by concatenating namespace with an event name (eg: backend.chat.ollama.start).- defines a data object for the
fetch_dataevent, - creates an emitter from the root one,
- registers a callback listening to the
fetch_dataevent, which modifies its content, - fires the
fetch_dataevent, - logs the modified event’s data.
Python
The
emitter.on can be used directly and not just as a decorator. Example: emitter.on("fetch_data", callback).Run (Context)
TheRun class acts as a wrapper of the target implementation with its own lifecycle (an emitter with a set of events) and context (data that gets propagated to all events).
The RunContext class is a container that contains information about the current execution context.
Why is it useful?
Such abstraction allows you to:
- modify
inputto the given target (listen to astartevent and modify the content of theinputproperty) - modify
outputfrom the given target (listen to asuccessevent and modify the content of theoutputproperty) - does premature stop (listen to a
startevent and set theoutputproperty to not aNonevalue) - propagate
context(dictionary) to whatever component of your system you’d like - cancel the execution in an arbitrary place
- observability
run method gets called on a framework class that can be executed (eg, ChatModel, Agent, …).
The run object has the following methods:
- The
onmethod allows registering a callback to its emitter. - The
middlewarefor registering middleware (a function that takesRunContextas a first parameter or a class with abindmethod that takes theRunContextas a first parameter). - The
contextallows data to be set for a given execution. That data will then be propagated as metadata in every event that gets emitted.
RunContext), which internally forms a hierarchical tree structure that shares the same context.
In simpler terms, when you call one runnable (e.g., ChatModel) from within another runnable (e.g., Agent), the inner call (ChatModel) is attached to the context of the outer one (Agent).
Runnable
TheRunnable[R] class was created to unify common objects that can be executed and observed. It is an abstract class with the following traits:
- It has an abstract
runmethod that executes the class and returns aRun[R](Ris bound to theRunnableOutput). - It has an abstract
emittergetter. - It has a
middlewaresgetter that lists the existing middlewares.
RunnableOptions):
signal(an instance ofAbortSignal) — allows aborting the execution.context(a dictionary) — used to propagate additional data.
RunnableOutput has the following properties:
output: a list of messages (can be empty)context: a dictionary that can store additional datalast_message(getter): returns the last message if it exists, or creates an emptyAssistantMessageotherwise
Python
Working with events
The following sections show you how to use the emitter to capture and modify events.Scopes
Events can be generally observed on three different levels. 1. Global Level Every emitter provided by the out-of-the-box modules is a child of the root emitter, which means we can listen to all events directly from the root emitter.Python
Listeners that are bound “closer” to the source are executed earlier. For those that reside at the same level, the order can be altered by setting a
priority value which is part of the EmitterOptions class.Python
Python
run method).
The run instance has its own emitter, which is a direct child of the class emitter.
Thanks to this, one can apply modifications to a single run instead of the whole class.
Config
With a larger number of callbacks, we might want to ensure that some run before the others or that some run exclusively. To address these needs, we can use the optional keyword argument calledoptions of type EmitterOptions.
Example
Python
matcher parameter (the one that is used to match the event), the framework
decides whether to include/exclude nested events (events created from children emitters or from piping).
The default value of the match_nested depends on the matcher value. Note that the value can be set directly as shown in the example above.
| Matcher Type | Default match_nested |
|---|---|
String without . (event name) | False |
String with . (event path) | True |
"*" (match all top-level events) | False |
"*.*" (match all events) | True |
| Regex | True |
| Function | False |
If two events have the same priority, they are executed in the order they were added.
Lifecycle
When a framework component is executed, it creates a run context, which is a wrapper around the target handler that gives us the ability to modify its input and output (learn more). How it works? Once aRun instance is executed (i.e., awaited), the lifecycle begins:
- The
startevent is emitted. - The target implementation is executed.
- Depending on the outcome, either a
successorerrorevent is emitted. - Finally, the
finishevent is emitted.
| Event | Data Type | Description |
|---|---|---|
start | RunContextStartEvent | Triggered when the run starts. |
success | RunContextSuccessEvent | Triggered when the run succeeds. |
error | FrameworkError | Triggered when an error occurs. |
finish | RunContextFinishEvent | Triggered when the run finishes. |
Python
In this example we are using
create_internal_event_matcher which correctly matches the correct event.Debugging
To see which events get emitted in your case, the best way is to register a log-all callback for your run.Python
Reusability
We just showed you how you can alter the component’s behavior by listening to events that the target emits and modifying them. To make this concept more general and reusable, we can group all relevant listeners into a class called middleware. Similarly, callbacks are registered via theon method, and middlewares are registered via the middleware method. Sometimes they can also be set on the class itself via a constructor.
The middleware is a function that takes the RunContext where one can attach desired callbacks. Middleware can also be a class with a public bind method that retrieves the RunContext.
Python
Note that this concept is built on top of the run instance and is not available on the standalone emitter.
Piping
In some cases, one might want to propagate all events from one emitter to another (for instance when creating a child emitter).Existing middlewares
The following section showcases built-in middlewares that you can start using right away.Global Trajectory
The fastest way to see what’s happening in your code (who is calling what) is by using theGlobalTrajectoryMiddleware. It captures all events (even deeply nested) and prints them to the console while visualizing nested calls with indentation.
Example
Python
| Parameter | Description |
|---|---|
target | Specify a file or stream to which to write the trajectory (pass False to disable). |
included | List of classes to include in the trajectory. |
excluded | List of classes to exclude from the trajectory. |
pretty | Use pretty formatting for the trajectory. |
prefix_by_type | Customize how instances of individual classes should be printed. |
exclude_none | Exclude None values from the printing. |
enabled | Enable/Disable the logging. |
match_nested | Whether to observe trajectories of nested run contexts. |
emitter_priority | Setting higher priority may result in capturing events without any modifications from other middlewares. |
Python
Tool Call Streaming
Middleware for handling streaming tool calls in a ChatModel. This middleware observes and listens to Chat Model stream updates and parses the tool calls on demand so that they can be consumed as soon as possible. The middleware works even without streaming enabled. It just emits theupdate event at the very end.
Example
Python
| Parameter | Description |
|---|---|
target | The tool that we are waiting for to be called. |
key | Refers to the name of the attribute in the tool’s schema that we want to stream. |
match_nested | Whether the middleware should be applied only to the top level. |
force_streaming | Sets the stream flag on the ChatModel. |
Events glossary
The following sections list all events that can be observed for built-in components. Note that your tools/agents/etc. can emit additional events.Tools
The following events can be observed when callingTool.run(...).
| Event | Data Type | Description |
|---|---|---|
start | ToolStartEvent | Triggered when a tool starts executing. |
success | ToolSuccessEvent | Triggered when a tool completes execution successfully. |
error | ToolErrorEvent | Triggered when a tool encounters an error. |
retry | ToolRetryEvent | Triggered when a tool operation is being retried. |
finish | None | Triggered when tool execution finishes (regardless of success or error). |
Chat Models
The following events can be observed when callingChatModel.run(...).
| Event | Data Type | Description |
|---|---|---|
start | ChatModelStartEvent | Triggered when model generation begins. |
new_token | ChatModelNewTokenEvent | Triggered when a new token is generated during streaming. Streaming must be enabled. |
success | ChatModelSuccessEvent | Triggered when the model generation completes successfully. |
error | ChatModelErrorEvent | Triggered when model generation encounters an error. |
finish | None | Triggered when model generation finishes (regardless of success or error). |
Requirement Agent
| Event | Data Type | Description |
|---|---|---|
start | RequirementAgentStartEvent | Triggered when the agent begins execution. |
success | RequirementAgentSuccessEvent | Triggered when the agent successfully completes execution. |
final_answer | RequirementAgentFinalAnswerEvent | Triggered with intermediate chunks of the final answer when streaming is enabled. |
ToolCalling Agent
The following events can be observed by callingToolCallingAgent.run(...).
| Event | Data Type | Description |
|---|---|---|
start | ToolCallingAgentStartEvent | Triggered when the agent begins execution. |
success | ToolCallingAgentSuccessEvent | Triggered when the agent successfully completes execution. |
ReAct Agent
The following events can be observed by callingReActAgent.run(...).
| Event | Data Type | Description |
|---|---|---|
start | ReActAgentStartEvent | Triggered when the agent begins execution. |
error | ReActAgentErrorEvent | Triggered when the agent encounters an error. |
retry | ReActAgentRetryEvent | Triggered when the agent is retrying an operation. |
success | ReActAgentSuccessEvent | Triggered when the agent successfully completes execution. |
update and partial_update | ReActAgentUpdateEvent | Triggered when the agent updates its state. |
Workflow
The following events can be observed when callingWorkflow.run(...).
| Event | Data Type | Description |
|---|---|---|
start | WorkflowStartEvent | Triggered when a workflow step begins execution. |
success | WorkflowSuccessEvent | Triggered when a workflow step completes successfully. |
error | WorkflowErrorEvent | Triggered when a workflow step encounters an error. |
LinePrefixParser
The following events are caught internally by theLinePrefixParser.
| Event | Data Type | Description |
|---|---|---|
update | LinePrefixParserUpdate | Triggered when an update occurs. |
partial_update | LinePrefixParserUpdate | Triggered when a partial update occurs. |
StreamToolCallMiddleware
The following events are caught internally by theStreamToolCallMiddleware.
| Event | Data Type | Description |
|---|---|---|
update | StreamToolCallMiddlewareUpdateEvent | Triggered when an update occurs. |
GlobalTrajectoryMiddleware
The following events are handled internally by theGlobalTrajectoryMiddleware:
| Event | Data Type | Description |
|---|---|---|
start | GlobalTrajectoryMiddlewareStartEvent | Triggered when a target begins execution. |
success | GlobalTrajectoryMiddlewareSuccessEvent | Triggered when a target completes successfully. |
error | GlobalTrajectoryMiddlewareErrorEvent | Triggered when an error occurs during target execution. |
finish | GlobalTrajectoryMiddlewareFinishEvent | Triggered after a target has finished execution, regardless of success or failure. |
GlobalTrajectoryMiddlewareEvent class.
Python
origin attribute is the original event (e.g., start → RunContextStartEvent, etc.) that comes from the RunContext.
RunContext
Special events that are emitted before the target’s handler gets executed. A run event contains.run. in its event’s path and has internal set to true in the event’s context object.
| Event | Data Type | Description |
|---|---|---|
start | RunContextStartEvent | Triggered when the run starts. Has input (positional/keyword argument with which the function was run) and output property. Set the output property to prevent the execution of the target handler. |
success | RunContextSuccessEvent | Triggered when the run succeeds. |
error | FrameworkError | Triggered when an error occurs. |
finish | RunContextFinishEvent | Triggered when the run finishes. |