RTVI: ignore messages from another bot

[vipyne]

Please describe the changes in your PR. If it is addressing an issue, please reference that as well.

Ignore RTVI messages from another bot

Currently, RTVI observer only handles RTVI messages from a client.
When a bot talks to another bot*, it receives messages from a bot and it does not aniticipate these types of messages. This was throwing lots and lots of errors and interrupting recordings.
This change ignores these messages.

*a bot talking to another bot is a common test scenario.

[mattieruth]

unfortunately, our message types are not this consistent. I'm working on docs for the new client and in doing so, documented the RTVI Protocol. You can check out that full spec (which of note: is what WILL be the spec, there are a bunch of messages in v1 that i did not document since they go away) here.

So this feels a bit icky. I'd prefer to be explicit and list out the types we wish to ignore.

[mattieruth]

Also, what error is being generated? All the messages should pass that model_validate

[vipyne]

Definitely pro explicitly listing all the messages, thank you for linking to the spec!
I will perhaps group the messages into current/staying and soon-going-away.

[vipyne]

Also, what error is being generated? All the messages should pass that model_validate

the error is:

| WARNING  | pipecat.processors.frameworks.rtvi:_handle_transport_message:805 - Invalid RTVI transport message '{'type': 'bot-llm-started', 'label': 'rtvi-ai'}': 1 validation error for RTVIMessage
id
  Field required [type=missing, input_value={'type': 'bot-llm-started', 'label': 'rtvi-ai'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing

it seems the bot* and user* messages are missing id.

additionally, along with the warning message in _handle_transport_message, we call
send_error, which calls
_send_error_frame, which calls
_push_transport_message, which pushes a
TransportMessageUrgentFrame, which gets
_handle_transport_message called again,
which starts the cycle over, as this message also does not have an id.

for example:

...
2025-06-20 09:59:58.624 | WARNING  | pipecat.processors.frameworks.rtvi:_handle_transport_message:805 - Invalid RTVI transport message '{'type': 'bot-llm-stopped', 'label': 'rtvi-ai'}': 1 validation error for RTVIMessage
id
  Field required [type=missing, input_value={'type': 'bot-llm-stopped', 'label': 'rtvi-ai'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
2025-06-20 09:59:58.624 | WARNING  | pipecat.processors.frameworks.rtvi:_handle_transport_message:805 - Invalid RTVI transport message '{'label': 'rtvi-ai', 'type': 'bot-started-speaking'}': 1 validation error for RTVIMessage
id
  Field required [type=missing, input_value={'label': 'rtvi-ai', 'typ... 'bot-started-speaking'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
2025-06-20 09:59:58.706 | WARNING  | pipecat.processors.frameworks.rtvi:_handle_transport_message:805 - Invalid RTVI transport message '{'label': 'rtvi-ai', 'type': 'error', 'data': {'error': "Invalid RTVI transport message: 1 validation error for RTVIMessage\nid\n  Field required [type=missing, input_value={'type': 'bot-llm-started', 'label': 'rtvi-ai'}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.10/v/missing", 'fatal': False}}': 1 validation error for RTVIMessage
id
  Field required [type=missing, input_value={'label': 'rtvi-ai', 'typ...ssing", 'fatal': False}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
2025-06-20 09:59:58.740 | WARNING  | pipecat.processors.frameworks.rtvi:_handle_transport_message:805 - Invalid RTVI transport message '{'data': {'text': 'Hi!'}, 'type': 'bot-tts-text', 'label': 'rtvi-ai'}': 1 validation error for RTVIMessage
id
  Field required [type=missing, input_value={'data': {'text': 'Hi!'},...xt', 'label': 'rtvi-ai'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
2025-06-20 09:59:58.794 | WARNING  | pipecat.processors.frameworks.rtvi:_handle_transport_message:805 - Invalid RTVI transport message '{'label': 'rtvi-ai', 'data': {'error': "Invalid RTVI transport message: 1 validation error for RTVIMessage\nid\n  Field required [type=missing, input_value={'data': {'text': 'Hi'}, ... 'type': 'bot-llm-text'}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.10/v/missing", 'fatal': False}, 'type': 'error'}': 1 validation error for RTVIMessage
id
  Field required [type=missing, input_value={'label': 'rtvi-ai', 'dat...False}, 'type': 'error'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
2025-06-20 09:59:58.794 | WARNING  | pipecat.processors.frameworks.rtvi:_handle_transport_message:805 - Invalid RTVI transport message '{'data': {'error': "Invalid RTVI transport message: 1 validation error for RTVIMessage\nid\n  Field required [type=missing, input_value={'type': 'bot-llm-text', ...!'}, 'label': 'rtvi-ai'}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.10/v/missing", 'fatal': False}, 'type': 'error', 'label': 'rtvi-ai'}': 1 validation error for RTVIMessage
id
  Field required [type=missing, input_value={'data': {'error': "Inval...or', 'label': 'rtvi-ai'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
...

so it is a compounded issue and all the logs overwhelm the system. the call sounds ok, but under the hood the machine is struggling, and that was causing issues recording clean audio.

[vipyne]

(in the logs above, I have added exactly what the message is that triggers the error. in current code it only prints the error, ie | WARNING | pipecat.processors.frameworks.rtvi:_handle_transport_message:805 | 60668b54-6626-4108-899f-6323fd9474bb - Invalid RTVI transport message: 1 validation error for RTVIMessage)

[vipyne]

oof, ok, after some more investigation, it seems we can turn off these messages with RTVIObserverParams

    bot_llm_enabled: bool = True
    bot_tts_enabled: bool = True
    bot_speaking_enabled: bool = True
    user_llm_enabled: bool = True
    user_speaking_enabled: bool = True
    user_transcription_enabled: bool = True
    metrics_enabled: bool = True
    errors_enabled: bool = True

Adding a comment here to say these should be set to False if a bot is talking to another bot and hopefully save a future dev time.

These might need to be updated with the new spec?

[mattieruth]

i like this approach for now. If more people start doing multi-bot or multi-client apps, we're going to need to expand rtvi. right now it (like pipecat itself) is really is designed for 1:1 user-bot communication.

[mattieruth]