> ## Documentation Index
> Fetch the complete documentation index at: https://daily-docs-pr-5482.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# System Frames

> Reference for SystemFrame types: pipeline lifecycle, interruptions, speaking state, input, and diagnostics

SystemFrames have higher priority than DataFrames and ControlFrames and are never cancelled during user interruptions. They are queued and processed in order with other SystemFrames. They carry signals that must always be delivered: pipeline startup and teardown, error notifications, user input, and speaking state changes. See the [frames overview](/api-reference/server/frames/overview) for base class details, mixin fields, and frame properties common to all frames.

## Pipeline Lifecycle

### StartFrame

The first frame pushed into a pipeline, marking the point at which frames start flowing. Every processor receives this before any DataFrames or ControlFrames arrive; a processor holds anything that reaches it earlier and delivers it, in arrival order, once the `StartFrame` has passed.

Pipeline configuration reaches processors through [`FrameProcessorSetup`](/pipecat/fundamentals/custom-frame-processor#frameprocessorsetup) in `setup()`, which runs before this frame.

<Warning>
  Reading `audio_in_sample_rate`, `audio_out_sample_rate`, `enable_metrics`,
  `enable_tracing`, `enable_usage_metrics`, `report_only_initial_ttfb`, or
  `tracing_context` off a `StartFrame` is deprecated since v1.8.0 and warns.
  These fields are removed in 2.0.0 — read them from the `FrameProcessorSetup`
  passed to `setup()` instead:

  ```python theme={null}
  async def setup(self, setup: FrameProcessorSetup):
      await super().setup(setup)
      self._sample_rate = setup.audio_out_sample_rate
  ```
</Warning>

### CancelFrame

Stops the pipeline immediately, skipping any queued non-SystemFrames. Use this when you need to abort without waiting for pending work to drain. For example, when the user has left the session.

<ParamField path="reason" type="Any | None" default="None">
  Optional reason for the cancellation.
</ParamField>

## Errors

### ErrorFrame

Carries an error notification, typically pushed upstream so earlier processors can react.

<ParamField path="error" type="str" required>
  Human-readable error message.
</ParamField>

<ParamField path="category" type="ErrorCategory | None" default="None">
  What kind of failure this was — a rejected API key (`AUTHENTICATION`) versus a
  provider outage (`SERVER`), for example. See [Error
  Handling](/pipecat/fundamentals/error-handling#errorcategory).
</ParamField>

<ParamField path="processor" type="FrameProcessor | None" default="None">
  The processor that raised the error. Read `processor.is_usable` to tell an
  error the processor can carry on from apart from one that has finished it.
</ParamField>

<ParamField path="exception" type="Exception | None" default="None">
  The underlying exception, if one was caught.
</ParamField>

<ParamField path="fatal" type="bool" default="False" deprecated>
  *Deprecated in v1.8.0. Will be removed in 2.0.0.* Check `processor.is_usable`
  instead, and set the pipeline's
  [`processor_unusable_policy`](/pipecat/fundamentals/error-handling#deciding-what-the-pipeline-does)
  to decide what an unusable processor does to the run.
</ParamField>

### FatalErrorFrame

<Warning>
  Deprecated in v1.8.0, removed in 2.0.0. Report the error with `push_error(...,
      force_treat_as_permanent=True)` when it leaves its processor unable to work,
  or push an `EndWorkerFrame` after a plain `ErrorFrame` when the pipeline
  should stop for a reason that isn't about a processor's state. See [Migrating
  from fatal
  errors](/pipecat/fundamentals/error-handling#migrating-from-fatal-errors).
</Warning>

An unrecoverable error requiring the bot to shut down. The `fatal` field is always `True`.

Inherits from `ErrorFrame`.

## Processor Pause/Resume

Resuming always travels the high-priority input queue, so a paused processor is reached regardless of what has queued up behind the pause. Pausing comes in two forms: [`FrameProcessorPauseFrame`](/api-reference/server/frames/control-frames#frameprocessorpauseframe) is a `ControlFrame` that takes effect in order, after the frames ahead of it, while `FrameProcessorPauseUrgentFrame` takes effect immediately.

### FrameProcessorResumeFrame

Resumes a previously paused processor, releasing all buffered frames for processing in the order they arrived.

<ParamField path="processor" type="FrameProcessor">
  The processor to resume.
</ParamField>

### FrameProcessorPauseUrgentFrame

Pauses a processor immediately, without waiting for queued frames to drain first.

<ParamField path="processor" type="FrameProcessor">
  The processor to pause.
</ParamField>

### FrameProcessorResumeUrgentFrame

Equivalent to `FrameProcessorResumeFrame`. Either resumes a paused processor and releases its buffered frames.

<ParamField path="processor" type="FrameProcessor">
  The processor to resume.
</ParamField>

## Interruptions

### InterruptionFrame

Interrupts the pipeline, discarding pending DataFrames and ControlFrames. Typically triggered when the user starts speaking during a bot response.

## User Speaking State

### UserStartedSpeakingFrame

Indicates that a user turn has begun. By this point, transcriptions are usually already flowing through the pipeline.

### UserStoppedSpeakingFrame

Marks the end of a user turn. The bot's response is triggered separately by the turn detection system.

### ProposedUserStartedSpeakingFrame

Proposes that a user turn has started. Emitted by a component with its own turn detection, typically an STT or realtime LLM service whose provider reports speech boundaries. It is a proposal rather than a decision: an [`ExternalUserTurnStartStrategy`](/api-reference/server/utilities/turn-management/user-turn-strategies#externaluserturnstartstrategy) resolves it into a `UserStartedSpeakingFrame` and broadcasts the interruption.

Its end-of-turn counterpart, [`ProposedUserStoppedSpeakingFrame`](/api-reference/server/frames/control-frames#proposeduserstoppedspeakingframe), is a `ControlFrame`. Resolving a start proposal broadcasts an interruption, which has to reach the pipeline ahead of whatever is queued; resolving a stop proposal has to stay ordered against the final `TranscriptionFrame`, since the strategy needs that text in hand to close the turn on.

### UserSpeakingFrame

Emitted by the VAD processor while the user is actively speaking. Useful for UI feedback or suppressing idle timeouts.

### UserTurnInferenceCompletedFrame

Indicates that the user turn is semantically complete. Emitted by any component that can judge conversational turn completeness — for example an LLM with turn-completion markers, an STT service with built-in turn detection, or a dedicated end-of-turn classifier. Stop strategies that gate the user-turn-stop event on an external completeness signal (e.g. [`LLMTurnCompletionUserTurnStopStrategy`](/api-reference/server/utilities/turn-management/user-turn-strategies#llmturncompletionuserturnstopstrategy)) consume this frame to finalize the turn. An absence of this frame means the turn is not yet considered complete.

### UserMuteStartedFrame

Broadcast when one or more [user mute strategies](/api-reference/server/utilities/turn-management/user-mute-strategies) activate. User mute temporarily suppresses user input while the bot is speaking to prevent interruptions. While muted, the `LLMUserAggregator` drops incoming user frames (`InputAudioRawFrame`, `TranscriptionFrame`, `InterimTranscriptionFrame`, `UserStartedSpeakingFrame`, `UserStoppedSpeakingFrame`, `ProposedUserStartedSpeakingFrame`, `ProposedUserStoppedSpeakingFrame`, VAD signals, and `InterruptionFrame`). Lifecycle frames (`StartFrame`, `EndFrame`, `CancelFrame`) are never muted.

### UserMuteStoppedFrame

Broadcast when all active [user mute strategies](/api-reference/server/utilities/turn-management/user-mute-strategies) deactivate, allowing user input to be processed again.

## VAD Events

These frames are emitted directly by the Voice Activity Detection (VAD) processor and carry timing metadata. Higher-level speaking-state frames (`UserStartedSpeakingFrame`, `UserStoppedSpeakingFrame`) are derived from these.

### VADUserStartedSpeakingFrame

VAD confirmed that speech has started.

<ParamField path="start_secs" type="float" default="0.0">
  Timestamp in seconds when speech onset was detected.
</ParamField>

<ParamField path="timestamp" type="float" default="time.time()">
  Wall-clock time when the frame was created.
</ParamField>

### VADUserStoppedSpeakingFrame

VAD confirmed that speech has ended.

<ParamField path="stop_secs" type="float" default="0.0">
  Timestamp in seconds when speech ended.
</ParamField>

<ParamField path="timestamp" type="float" default="time.time()">
  Wall-clock time when the frame was created.
</ParamField>

### SpeechControlParamsFrame

Notifies processors that VAD or turn detection parameters have changed at runtime.

<ParamField path="vad_params" type="VADParams | None" default="None">
  Updated VAD parameters.
</ParamField>

<ParamField path="turn_params" type="BaseTurnParams | None" default="None">
  Updated turn detection parameters.
</ParamField>

## Bot Speaking State

### BotStartedSpeakingFrame

Emitted by the output transport when the bot begins speaking. Broadcast in both directions so processors on either side of the transport can react.

### BotStoppedSpeakingFrame

Emitted by the output transport when the bot finishes speaking. Also broadcast in both directions.

### BotSpeakingFrame

Emitted continuously while the bot is speaking. Processors can use this to suppress idle timeouts or drive visual indicators.

## Connection Status

### BotConnectedFrame

The bot has joined the transport room. Only relevant for SFU-based transports: Daily, LiveKit, HeyGen, and Tavus.

### ClientConnectedFrame

A client or participant has connected to the transport.

## Input Frames

Input frames carry raw data from transport sources into the pipeline. As `SystemFrame`s, they are never discarded during interruptions. Incoming user data must always be processed.

### InputAudioRawFrame

Raw audio received from the transport. Inherits the `audio`, `sample_rate`, `num_channels`, and `num_frames` fields from the [`AudioRawFrame`](/api-reference/server/frames/overview#audiorawframe) mixin.

Inherits from `AudioRawFrame`.

### UserAudioRawFrame

Audio from a specific user in a multi-participant session.

Inherits from `InputAudioRawFrame`.

<ParamField path="user_id" type="str" default="&#x22;&#x22;">
  Identifier for the user who produced this audio.
</ParamField>

### InputImageRawFrame

Raw image received from the transport. Inherits `image`, `size`, and `format` from the [`ImageRawFrame`](/api-reference/server/frames/overview#imagerawframe) mixin.

Inherits from `ImageRawFrame`.

### UserImageRawFrame

An image from a specific user, optionally tied to a pending image request.

Inherits from `InputImageRawFrame`.

<ParamField path="user_id" type="str" default="&#x22;&#x22;">
  Identifier for the user who produced this image.
</ParamField>

<ParamField path="text" type="str | None" default="None">
  Optional text associated with the image.
</ParamField>

<ParamField path="append_to_context" type="bool | None" default="None">
  Whether to append this image to the LLM context.
</ParamField>

<ParamField path="request" type="UserImageRequestFrame | None" default="None">
  The original request frame that triggered this image capture.
</ParamField>

### InputTextRawFrame

Text received from the transport, such as a user typing in a chat interface. Inherits the `text` field from `TextFrame`.

Inherits from `TextFrame`.

## DTMF Input

### InputDTMFFrame

A DTMF keypress received from the transport. Inherits the `button` field from the `DTMFFrame` mixin.

Inherits from `DTMFFrame`.

### OutputDTMFUrgentFrame

A DTMF keypress for immediate output, bypassing the normal frame queue.

Inherits from `DTMFFrame`.

## Transport Messages

### InputTransportMessageFrame

A message received from an external transport. The message format is transport-specific.

<ParamField path="message" type="Any" required>
  The transport message payload.
</ParamField>

### OutputTransportMessageUrgentFrame

An outbound transport message that bypasses the normal queue for immediate delivery.

<ParamField path="message" type="Any" required>
  The transport message payload.
</ParamField>

## Function Calling

### FunctionCallsStartedFrame

Signals that one or more function calls are about to begin executing.

<ParamField path="function_calls" type="Sequence[FunctionCallFromLLM]" required>
  Sequence of function calls that will be executed.
</ParamField>

### FunctionCallCancelFrame

Signals that a function call was cancelled, typically due to user interruption when the function's `cancel_on_interruption` flag is set.

<ParamField path="function_name" type="str" required>
  Name of the function that was cancelled.
</ParamField>

<ParamField path="tool_call_id" type="str" required>
  Unique identifier for the cancelled function call.
</ParamField>

## User Interaction

### UserImageRequestFrame

Requests an image from a specific user, typically to capture a camera frame for vision processing.

<ParamField path="user_id" type="str" required>
  Identifier for the user to capture from.
</ParamField>

<ParamField path="text" type="str | None" default="None">
  Optional text prompt associated with the image request.
</ParamField>

<ParamField path="append_to_context" type="bool | None" default="None">
  Whether to append the resulting image to the LLM context.
</ParamField>

<ParamField path="video_source" type="str | None" default="None">
  Specific video source to capture from.
</ParamField>

<ParamField path="function_name" type="str | None" default="None">
  Function name if this request originated from a tool call.
</ParamField>

<ParamField path="tool_call_id" type="str | None" default="None">
  Tool call identifier if this request originated from a tool call.
</ParamField>

<ParamField path="result_callback" type="Any | None" default="None">
  Callback to invoke with the captured image result.
</ParamField>

### STTMuteFrame

Mutes or unmutes the STT service. While muted, incoming audio is not sent to the STT provider.

<ParamField path="mute" type="bool" required>
  `True` to mute, `False` to unmute.
</ParamField>

### UserIdleTimeoutUpdateFrame

Updates the user idle timeout at runtime. Setting the timeout to `0` disables idle detection, and setting a positive value enables it. Updates apply immediately: a running idle timer restarts with the new duration, and if the bot is waiting for the user to speak, the timer is armed right away.

<ParamField path="timeout" type="float" required>
  New idle timeout in seconds. `0` disables detection.
</ParamField>

## Diagnostics

### MetricsFrame

Performance metrics collected from processors. Emitted when metrics reporting is enabled in the pipeline configuration.

<ParamField path="data" type="List[MetricsData]" required>
  List of metrics data entries.
</ParamField>

## Service Metadata

### ServiceMetadataFrame

Base metadata frame broadcast by services at startup, providing information about service capabilities and configuration.

<ParamField path="service_name" type="str" required>
  Name of the service that emitted this metadata.
</ParamField>

<ParamField path="user_turn_strategies" type="UserTurnStrategies | None" default="None">
  The turn strategies the service recommends, when it has an opinion — a service
  with its own turn detection uses this to ask for
  [`ExternalUserTurnStrategies`](/api-reference/server/utilities/turn-management/external-turn-management).
</ParamField>

### STTMetadataFrame

Metadata from an STT service, including latency characteristics used for turn detection tuning.

Inherits from `ServiceMetadataFrame`.

<ParamField path="ttfs_p99_latency" type="float" required>
  P99 latency in seconds for time-to-final-segment. Used by turn detectors to
  calibrate wait times.
</ParamField>

### LLMServiceMetadataFrame

Broadcast at pipeline start by an LLM service. Downstream processors — notably `LLMContextAggregatorPair` — read `is_realtime_service` to tell whether a realtime (speech-to-speech) service is in the pipeline and configure themselves accordingly. See [Realtime (Speech-to-Speech) Services](/api-reference/server/utilities/turn-management/external-turn-management#realtime-speech-to-speech-services).

Inherits from `ServiceMetadataFrame`.

<ParamField path="is_realtime_service" type="bool" default="False">
  Whether the broadcasting service is a realtime (speech-to-speech) LLM service.
</ParamField>

## RTVI

Frames for the [Real-Time Voice Interface (RTVI)](/api-reference/server/rtvi/introduction) protocol, which bridges clients and the pipeline. These frames handle custom messaging between the client and server.

### RTVIServerMessageFrame

Sends a server message to the connected client.

<ParamField path="data" type="Any" required>
  The message data to send to the client.
</ParamField>

### RTVIClientMessageFrame

A message received from the client, expecting a server response via `RTVIServerResponseFrame`.

<ParamField path="msg_id" type="str" required>
  Unique identifier for the client message.
</ParamField>

<ParamField path="type" type="str" required>
  The message type.
</ParamField>

<ParamField path="data" type="Any | None" default="None">
  Optional message data from the client.
</ParamField>

### RTVIServerResponseFrame

Responds to an `RTVIClientMessageFrame`. Include the original client message frame to ensure the response is properly correlated. Set the `error` field to respond with an error instead of a normal response.

<ParamField path="client_msg" type="RTVIClientMessageFrame" required>
  The original client message this response is for.
</ParamField>

<ParamField path="data" type="Any | None" default="None">
  Response data to send to the client.
</ParamField>

<ParamField path="error" type="str | None" default="None">
  Error message. When set, the client receives an `error-response` instead of a
  `server-response`.
</ParamField>

## Pipeline Worker Frames

Pipeline worker frames provide a system-priority mechanism for requesting pipeline actions from outside the normal frame flow. They are converted into their corresponding standard frames when processed.

### WorkerSystemFrame

Base class for system-priority worker frames.

### CancelWorkerFrame

Requests immediate pipeline cancellation. Converted to a `CancelFrame` when processed by the pipeline.

Inherits from `WorkerSystemFrame`.

<ParamField path="reason" type="Any | None" default="None">
  Optional reason for the cancellation request.
</ParamField>

### InterruptionWorkerFrame

Requests a pipeline interruption. Converted to an `InterruptionFrame` when processed.

Inherits from `WorkerSystemFrame`.