Merge pull request #1241 from prominenceai/v0.30.a.alpha

V0.30.a.alpha

Author: rjhowell44

Makefile

@@ -39,7 +39,7 @@ else
 endif
 
 CXX_VERSION:=c++17
-DSL_VERSION:='L"v0.30.alpha"'
+DSL_VERSION:='L"v0.30.a.alpha"'
 GLIB_VERSION:=2.0
 
 GSTREAMER_VERSION:=1.0

README.md

@@ -36,7 +36,25 @@ The DeepStream SDK and DSL use the open source [GStreamer](https://gstreamer.fre
 ---
 
 ## Important Bulletins
-The latest release `v0.30.alpha` was developed to support DeepSteam 6.4 and 7.0 on Ubuntu 22.04. 
+The latest release [v0.30.a.alpha](/Release%20Notes/v0.30.a.alpha.md) is a patch for the v0.30.alpha release that fixes a [critical bug](https://github.com/prominenceai/deepstream-services-library/issues/1238) in the V4L2 Sink. 
+
+The [v0.30.alpha](/Release%20Notes/v0.30.alpha.md) release was developed to support DeepSteam 6.4 and 7.0 on Ubuntu 22.04. 
+
+> WARNING! There is a cricical error in the DeepStream 7.0 Installation Instructions.
+
+Under the section [Install librdkafka](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Installation.html#install-librdkafka-to-enable-kafka-protocol-adaptor-for-message-broker), the following instructions
+```bash
+$ sudo mkdir -p /opt/nvidia/deepstream/deepstream/lib
+$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream/lib
+```
+Must be replaced with.
+```bash
+$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-7.0/lib
+$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-7.0/lib
+```
+See [Error in DeepStream 7.0 installation instructions - symlink fails to create](https://forums.developer.nvidia.com/t/error-in-deepstream-7-0-installation-instructions-symlink-fails-to-create/296026) for more information.
+
+---
 
 > WARNING! There is a cricical error in the DeepStream 7.0 Installation Instructions.
 

Release Notes/dsl-releases.md

@@ -2,11 +2,12 @@
 
 | Release                                             | Date        |
 | --------------------------------------------------- | ----------- |
+| [v0.30.a.alpha (patch)](/Release%20Notes/v0.30.a.alpha.md)  | 07/14/2024  |
 | [v0.30.alpha](/Release%20Notes/v0.30.alpha.md)      | 05/28/2024  |
 | [v0.29.alpha](/Release%20Notes/v0.29.alpha.md)      | 03/23/2024  |
 | [v0.28.alpha](/Release%20Notes/v0.28.alpha.md)      | 01/29/2024  |
-| [v0.27.b.alpha](/Release%20Notes/v0.27.b.alpha.md)  | 11/08/2023  |
-| [v0.27.a.alpha](/Release%20Notes/v0.27.a.alpha.md)  | 10/24/2023  |
+| [v0.27.b.alpha (patch)](/Release%20Notes/v0.27.b.alpha.md)  | 11/08/2023  |
+| [v0.27.a.alpha (patch)](/Release%20Notes/v0.27.a.alpha.md)  | 10/24/2023  |
 | [v0.27.alpha](/Release%20Notes/v0.27.alpha.md)      | 09/13/2023  |
 | [v0.26.a.alpha](/Release%20Notes/v0.26.a.alpha.md)  | 06/07/2023  |
 | [v0.26.alpha](/Release%20Notes/v0.26.alpha.md)      | 05/03/2023  |

Release Notes/v0.30.a.alpha.md

@@ -0,0 +1,14 @@
+# v0.30.a.alpha (patch) Release Notes
+**Important!**
+* `v0.30.a.alpha` is a **patch** release (patch `a` for the `v0.30.alpha` release).
+* The public/client API in DslApi.h has not been changed - i.e there are no new services.
+* There is one critical bug fix, 2 new examples, minor documentation updates, and more test coverage.
+
+## Issues closed in this release
+### Bugs closed in this release
+*  Fix and complete the V4L2 Sink - revert previous invalid change, and add test coverage and examples [#1238](https://github.com/prominenceai/deepstream-services-library/issues/1238).
+
+## New Examples in this release
+* [1file_pgie_iou_tracker_osd_window_v4l2.py](/examples/python/1file_pgie_iou_tracker_osd_window_v4l2.py)
+* [1file_pgie_iou_tracker_osd_window_v4l2.cpp](/examples/python/1file_pgie_iou_tracker_osd_window_v4l2.cpp)
+

docs/api-sink.md

@@ -81,6 +81,31 @@ As a general rule
 * <b id="f1">1</b> _The NVIDIA Smart Recording Bin - used by the Record Sink - does not support/extern any of the common sink properties._ [↩](#a1)
 * <b id="f2">2</b> _The rtspclientsink plugin is not derived from the GStreamer basesink which implements the common sink properties._ [↩](#a2)
 
+## Using the V4L2 Sink with V4L2 Loopback
+From the [GStream documentation](https://gstreamer.freedesktop.org/documentation/video4linux2/v4l2sink.html?gi-language=c#v4l2sink-page):
+> _"The V4L2 Sink can be used to display video to V4L2 capatible video devices (screen overlays provided by the graphics hardware, tv-out, etc)."_ 
+
+[V4L2 Loopback](https://github.com/umlaeute/v4l2loopback) can be used to create "virtual V4L2 video devices" allowing applications to read the virtual devices as V4L2 input sources. See: https://github.com/umlaeute/v4l2loopback for more information.
+
+Applicable DSL examples:
+* [1file_pgie_iou_tracker_osd_window_v4l2.py](/examples/python/1file_pgie_iou_tracker_osd_window_v4l2.py)
+* [1file_pgie_iou_tracker_osd_window_v4l2.cpp](/examples/python/1file_pgie_iou_tracker_osd_window_v4l2.cpp)
+
+You can install v4l2loopback with the command below. Depending on your device, there may be extra steps to install a kernel module. Follow the prompts as directed.
+```bash
+   $ sudo apt-get install v4l2loopback-dkms
+```
+
+Run the following to setup '/dev/video3' (used by the examples above)
+```bash
+    $ sudo modprobe v4l2loopback video_nr=3
+```
+
+You can use the following GStreamer launch command to test the loopback device when the example pipeline is running.
+```bash
+   $ gst-launch-1.0 v4l2src device=/dev/video3 ! videoconvert  ! xvimagesink
+```
+
 ## Sink API
 **Types:**
 * [`dsl_recording_info`](#dsl_recording_info)

examples/cpp/1file_pgie_iou_tracker_osd_window_v4l2.cpp

@@ -0,0 +1,220 @@
+/*
+The MIT License
+
+Copyright (c) 2024, Prominence AI, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in-
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+*/
+
+/* ##############################################################################
+#
+# This simple example demonstrates how to create a set of Pipeline components, 
+# specifically:
+#   - A File Source
+#   - Primary GST Inference Engine (PGIE)
+#   - IOU Tracker
+#   - On-Screen Display
+#   - Window Sink
+#   - V4L2 Sink
+# ...and how to add them to a new Pipeline and play.
+#
+# The V4L2 Sink is used to display video to v4l2 video devices. 
+# 
+# V4L2 Loopback can be used to create "virtual video devices". Normal (v4l2) 
+# applications will read these devices as if they were ordinary video devices.
+# See: https://github.com/umlaeute/v4l2loopback for more information.
+#
+# You can install v4l2loopback with
+#    $ sudo apt-get install v4l2loopback-dkms
+#
+# Run the following to setup '/dev/video3'
+#    $ sudo modprobe v4l2loopback video_nr=3
+#
+# When the script is running, you can use the following GStreamer launch 
+# command to test the loopback
+#    $ gst-launch-1.0 v4l2src device=/dev/video3 ! videoconvert  ! xvimagesink
+#
+# The example registers handler callback functions for:
+#   - key-release events
+#   - delete-window events
+#   - end-of-stream EOS events
+#   - Pipeline change-of-state events
+#  
+############################################################################## */
+
+#include <iostream>
+#include <glib.h>
+#include <gst/gst.h>
+#include <gstnvdsmeta.h>
+#include <nvdspreprocess_meta.h>
+
+#include "DslApi.h"
+
+std::wstring file_path(
+    L"/opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h265.mp4");
+
+// Config and model-engine files 
+std::wstring primary_infer_config_file(
+    L"/opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-preprocess-test/config_infer.txt");
+std::wstring primary_model_engine_file(
+    L"/opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet18_trafficcamnet.etlt_b8_gpu0_int8.engine");
+
+// Config file used by the IOU Tracker    
+std::wstring iou_tracker_config_file(
+    L"/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_tracker_IOU.yml");
+
+// V4L2 Sink device location to stream to
+std::wstring DEVICE_LOCATION = L"/dev/video3";
+
+uint WINDOW_WIDTH = 1280;
+uint WINDOW_HEIGHT = 720;
+
+// 
+// Function to be called on XWindow KeyRelease event
+// 
+void xwindow_key_event_handler(const wchar_t* in_key, void* client_data)
+{   
+    std::wstring wkey(in_key); 
+    std::string key(wkey.begin(), wkey.end());
+    std::cout << "key released = " << key << std::endl;
+    key = std::toupper(key[0]);
+    if(key == "P"){
+        dsl_pipeline_pause(L"pipeline");
+    } else if (key == "R"){
+        dsl_pipeline_play(L"pipeline");
+    } else if (key == "Q" or key == "" or key == ""){
+        std::cout << "Main Loop Quit" << std::endl;
+        dsl_pipeline_stop(L"pipeline");
+        dsl_main_loop_quit();
+    }
+}
+
+// 
+// Function to be called on XWindow Delete event
+//
+void xwindow_delete_event_handler(void* client_data)
+{
+    std::cout<<"delete window event"<<std::endl;
+    dsl_pipeline_stop(L"pipeline");
+    dsl_main_loop_quit();
+}
+    
+// 
+// Function to be called on End-of-Stream (EOS) event
+// 
+void eos_event_listener(void* client_data)
+{
+    std::cout<<"Pipeline EOS event"<<std::endl;
+    dsl_pipeline_stop(L"pipeline");
+    dsl_main_loop_quit();
+}    
+
+// 
+// Function to be called on every change of Pipeline state
+// 
+void state_change_listener(uint old_state, uint new_state, void* client_data)
+{
+    std::cout<<"previous state = " << dsl_state_value_to_string(old_state) 
+        << ", new state = " << dsl_state_value_to_string(new_state) << std::endl;
+}
+
+int main(int argc, char** argv)
+{
+    DslReturnType retval = DSL_RESULT_FAILURE;
+
+    // Since we're not using args, we can Let DSL initialize GST on first call    
+    while(true) 
+    {    
+
+        // New File Source using the file path specified above, repeat enabled.
+        retval = dsl_source_file_new(L"file-source", file_path.c_str(), true);
+        if (retval != DSL_RESULT_SUCCESS) break;
+
+        // New Primary GIE using the filespecs defined above, with interval and Id
+        retval = dsl_infer_gie_primary_new(L"primary-gie", 
+            primary_infer_config_file.c_str(), primary_model_engine_file.c_str(), 0);
+        if (retval != DSL_RESULT_SUCCESS) break;
+
+        // New IOU Tracker, setting operational width and height of input frame
+        retval = dsl_tracker_new(L"iou-tracker", 
+            iou_tracker_config_file.c_str(), 480, 272);
+        if (retval != DSL_RESULT_SUCCESS) break;
+
+        // New OSD with text, clock and bbox display all enabled. 
+        retval = dsl_osd_new(L"on-screen-display", true, true, true, false);
+        if (retval != DSL_RESULT_SUCCESS) break;
+
+        // New 3D Window Sink with 0 x/y offsets, and same dimensions as Camera Source
+        // EGL Sink runs on both platforms. 3D Sink is Jetson only.
+        if (dsl_info_gpu_type_get(0) == DSL_GPU_TYPE_INTEGRATED)
+        {
+            retval = dsl_sink_window_3d_new(L"window-sink", 0, 0, 
+                WINDOW_WIDTH, WINDOW_HEIGHT);            
+        }
+        else
+        {
+            retval = dsl_sink_window_egl_new(L"window-sink", 0, 0, 
+                WINDOW_WIDTH, WINDOW_HEIGHT);            
+        }
+        if (retval != DSL_RESULT_SUCCESS) break;
+
+        // Add the XWindow event handler functions defined above
+        retval = dsl_sink_window_key_event_handler_add(L"window-sink", 
+            xwindow_key_event_handler, NULL);
+        if (retval != DSL_RESULT_SUCCESS) break;
+
+        retval = dsl_sink_window_delete_event_handler_add(L"window-sink", 
+            xwindow_delete_event_handler, NULL);
+        if (retval != DSL_RESULT_SUCCESS) break;
+    
+        // New V4L2 Sink 
+        retval = dsl_sink_v4l2_new(L"v4l2-sink", DEVICE_LOCATION.c_str());
+        if (retval != DSL_RESULT_SUCCESS) break;
+
+        // Create a list of Pipeline Components to add to the new Pipeline.
+        const wchar_t* components[] = {L"file-source",  L"primary-gie", 
+            L"iou-tracker", L"on-screen-display", L"window-sink", L"v4l2-sink", NULL};
+        
+        // Add all the components to our pipeline
+        retval = dsl_pipeline_new_component_add_many(L"pipeline", components);
+        if (retval != DSL_RESULT_SUCCESS) break;
+            
+        // Add the EOS listener function defined above
+        retval = dsl_pipeline_eos_listener_add(L"pipeline", eos_event_listener, NULL);
+        if (retval != DSL_RESULT_SUCCESS) break;
+
+        // Play the pipeline
+        retval = dsl_pipeline_play(L"pipeline");
+        if (retval != DSL_RESULT_SUCCESS) break;
+        
+        // Start and join the main-loop
+        dsl_main_loop_run();
+        break;
+
+    }
+    
+    // Print out the final result
+    std::cout << dsl_return_value_to_string(retval) << std::endl;
+
+    dsl_delete_all();
+
+    std::cout<<"Goodbye!"<<std::endl;  
+    return 0;
+}
+            

examples/cpp/1v4l2_pgie_iou_tracker_osd_window.cpp

@@ -24,7 +24,7 @@ THE SOFTWARE.
 
 /* ##############################################################################
 #
-# The simple example demonstrates how to create a set of Pipeline components, 
+# This example demonstrates how to create a set of Pipeline components, 
 # specifically:
 #   - V4L2 Source - Web Camera
 #   - Primary GST Inference Engine (PGIE)
@@ -62,9 +62,6 @@ int brightness=0;
 int contrast=0;
 int hue=0;
 
-std::wstring uri_h265(
-    L"/opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h265.mp4");
-
 // Config and model-engine files 
 std::wstring primary_infer_config_file(
     L"/opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-preprocess-test/config_infer.txt");

examples/python/1file_pgie_iou_tracker_osd_window_rtsp_h265.py

@@ -143,15 +143,15 @@ def main(args):
             break
 
         # New RTSP Server Sink 
-        retVal = dsl_sink_rtsp_server_new('rtsp-sink', 
+        retval = dsl_sink_rtsp_server_new('rtsp-sink', 
             host = "0.0.0.0",       # 0.0.0.0 = "this host, this network."
             udp_port = 5400,        # UDP port 5400 uses the Datagram Protocol.             
             rtsp_port = 8554,       # 
             codec = DSL_CODEC_H265, # High Efficiency Video Coding (HEVC)
             bitrate = 0,            # Set to 0 to use plugin default (4000000)
             interval = 0)           # 0 = encode everyframe           
-        if retVal != DSL_RETURN_SUCCESS:
-            print(dsl_return_value_to_string(retVal)) 
+        if retval != DSL_RETURN_SUCCESS:
+            break
 
         # Add all the components to our pipeline
         retval = dsl_pipeline_new_component_add_many('pipeline',