Skip to content

Delayed H264 decoding on browsers when using VideoDecoder for macOS devices #899

@mycodingneeds

Description

@mycodingneeds

I have been trying to put together an end to end H264 real-time streaming implementation from macOS devices to browsers. The macOS implementation uses native hardware accelerated APIs to encoder the frames, convert the encoded buffers from AVCC format to AnnexB format and send it across over the wire, on the other end the browser receives the stream and feeds it to the VideoDecoder WebCodecs API. I am using hardwareAcceleration: 'no-preference' setting on the VideoDecoder so the browser uses hardware accelerated decoder provided by the operating system.

The problem I am facing is the decoder takes about couple of seconds(around 3 seconds most of the times) to decode the frames, irrespective of the profile I use, I have tried with High, Main and Baseline. As I am trying to achieve real time streaming even a more than few hundreds of milli seconds overall is not acceptable. I did verify the SPS, PPS, I and P frames and they all look good to me. Below is the config I am using on the browser to create the decoder

const config = {
  codec: "avc1.64001F",
  avc: { format: "annexb" },
  codecWidth: 1440,
  codecHeight: 900,
  hardwareAcceleration: 'no-preference',
  optimizeForLatency: true
}

I am also attaching a encoded bitstream for your reference of 17 frames (15 P-Frames separated by 2 I-Frames), can anyone help me to understand what is going on with the decoder/encoder here?

data_0_0_2025-07-23 0:23:41.1310.dat.zip

PS: When I decoded the same encoded bitstream using native decoder API, I do not see any issues, I can play the saved decoded video without any issues. I see issue only with browsers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions