-
Notifications
You must be signed in to change notification settings - Fork 151
Description
I have been trying to put together an end to end H264 real-time streaming implementation from macOS devices to browsers. The macOS implementation uses native hardware accelerated APIs to encoder the frames, convert the encoded buffers from AVCC format to AnnexB format and send it across over the wire, on the other end the browser receives the stream and feeds it to the VideoDecoder WebCodecs API. I am using hardwareAcceleration: 'no-preference'
setting on the VideoDecoder so the browser uses hardware accelerated decoder provided by the operating system.
The problem I am facing is the decoder takes about couple of seconds(around 3 seconds most of the times) to decode the frames, irrespective of the profile I use, I have tried with High, Main and Baseline. As I am trying to achieve real time streaming even a more than few hundreds of milli seconds overall is not acceptable. I did verify the SPS, PPS, I and P frames and they all look good to me. Below is the config I am using on the browser to create the decoder
const config = {
codec: "avc1.64001F",
avc: { format: "annexb" },
codecWidth: 1440,
codecHeight: 900,
hardwareAcceleration: 'no-preference',
optimizeForLatency: true
}
I am also attaching a encoded bitstream for your reference of 17 frames (15 P-Frames separated by 2 I-Frames), can anyone help me to understand what is going on with the decoder/encoder here?
data_0_0_2025-07-23 0:23:41.1310.dat.zip
PS: When I decoded the same encoded bitstream using native decoder API, I do not see any issues, I can play the saved decoded video without any issues. I see issue only with browsers.