Skip to content

Achieve higher efficiency and quality in low-bandwidth and high-latency environments #152

Open
@ehfd

Description

@ehfd

In low-bandwidth environments (under 2 mbps), while still acceptable, there is a cross in terms of performance and quality (starting from 1920x1080 FHD) compared to VNC.

This is because:

  1. We do not use XDamage with ximagesrc, which can differentiate between updated parts of the screen and those that are not updated (but leads to issues in higher bandwidth videos and thus disabled by default).
  2. We do not adapt encoder settings in low bandwidth environments to prefer not blurring while sacrificing performance with fullscreen refreshes (which leads back to number one).
  3. MJPEG, which is suited for partial screen refresh and is the core codec for various VNC implementations, is not applicable with WebRTC MediaStream (but may be applicable with WebRTC DataChannels, WebTransport or WebSocket, combined with nvjpegenc and vajpegenc hardware encoders).
  4. Most importantly, keyframes (also called IDR frames in H.264 and H.265), which are required to reestablish the video when the stream is down, lead to congestion because of their notably large size (often multitudes) compared to normal P-frames. No longer default.

(WebP is essentially VP8 and VP9, so it technically is already incorporated, but not optimized enough for the above scenarios.)

This should technically be possible within GStreamer, provided that various parameters are explored.

Metadata

Metadata

Assignees

No one assigned

    Labels

    encodingAudio or video encoders but not the OS interfacesenhancementNew feature or requesthelp wantedExternal contribution is requiredinterfaceOS input, display, or audio interfacesperformancePerformance or latency issues, not critical but impacts usagetransportUnderlying media or data transport protocolswebWeb components including gst-web

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions