There are twelve sample codes in this folder.
Example ex_application.c demonstrates how to write a GSTREAMER applications.
Run ex_app binary in terminal or cmd, you will see the moving ball windows displayed.
There are seven main steps to form the application of ex_application.c:
• Step 1: Initialize GStreamer
• Step 2: Create the elements
• Step 3: Create the empty pipeline and then build it
• Step 4: Modify the source's properties
• Step 5: Start playing the pipeline
• Step 6: Message handling
• Step 7: Free resources
Here, we will take ex_application.c for explanation.
In this step, it is required and must to initialize the GStreamer by init for the application.
/* Initialize GStreamer */
gst_init (&argc, &argv);
Create the elements by factory.
/* Create the elements */
source = gst_element_factory_make ("videotestsrc", "source");
sink = gst_element_factory_make ("autovideoconvert", "sink");
Generate a pipe line in order to put all element in it and then playing it later.
/* Create the empty pipeline */
pipeline = gst_pipeline_new ("test-pipeline");
/* Build the pipeline */
gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL);
This step is optional if you are going to set some properties to each element. In ex_application.c, we required to set properties, pattern, for switch the pattern images we are going to use:
/* Modify the source's properties */
g_object_set (source, "pattern", 18, NULL);
Playing the pipeline we created:
/* Start playing */
ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
This step makes sure the message be thrown by the pipeline we created through the GstBus. This message handling could customize based on different event thrown.
/* Wait until error or EOS */
bus = gst_element_get_bus (pipeline);
msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
/* Parse message */
if (msg != NULL)
{
GError *err;
gchar *debug_info;
switch (GST_MESSAGE_TYPE (msg))
{
case GST_MESSAGE_ERROR:
gst_message_parse_error (msg, &err, &debug_info);
g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
g_clear_error (&err);
g_free (debug_info);
break;
case GST_MESSAGE_EOS:
g_print ("End-Of-Stream reached.\n");
break;
default:
/* We should not reach here because we only asked for ERRORs and EOS */
g_printerr ("Unexpected message received.\n");
break;
}
gst_message_unref (msg);
}
Finally, release all used GStreamer instances:
/* Free resources */
gst_object_unref (bus);
gst_element_set_state (pipeline, GST_STATE_NULL);
gst_object_unref (pipeline);
The example codes ex_appsrc_appsink.cpp and ex_appsrc_appsink_advanced.cpp were detailed described in "Integrating the GStreamer Plugin" in the Programming Guide. More details could be referenced in the document, Edge Vision Analytics SDK Programming Guide in Chapter 6.
This sample shows you how to retrieve the adlink metadata from the application. This sample is alike ex_appsrc_appsink_advanced.cpp. First, include the gstadmeta.h:
#include "gstadmeta.h"
Second, Add the subfunction for retrieving the GstAdBatchMeta pointer from GstBuffer:
GstAdBatchMeta* gst_buffer_get_ad_batch_meta(GstBuffer* buffer)
{
gpointer state = NULL;
GstMeta* meta;
const GstMetaInfo* info = GST_AD_BATCH_META_INFO;
while ((meta = gst_buffer_iterate_meta (buffer, &state)))
{
if (meta->info->api == info->api)
{
GstAdMeta *admeta = (GstAdMeta *) meta;
if (admeta->type == AdBatchMeta)
return (GstAdBatchMeta*)meta;
}
}
return NULL;
}
Third, in the callback function, new_sample, of appsink has an extra code block which is going to get the detection metadata:
GstAdBatchMeta *meta = gst_buffer_get_ad_batch_meta(buffer);
if(meta != NULL)
{
AdBatch &batch = meta->batch;
VideoFrameData frame_info = batch.frames[0];
int detectionResultNumber = frame_info.detection_results.size();
cout << "detection result number = " << detectionResultNumber << endl;
for(int i = 0 ; i < detectionResultNumber ; ++i)
{
cout << "========== metadata in application ==========\n";
cout << "Class = " << frame_info.detection_results[i].obj_id << endl;
cout << "Label = " << frame_info.detection_results[i].obj_label << endl;
cout << "Prob = " << frame_info.detection_results[i].prob << endl;
cout << "(x1, y1, x2, y2) = ("
<< frame_info.detection_results[i].x1 << ", "
<< frame_info.detection_results[i].y1 << ", "
<< frame_info.detection_results[i].x2 << ", "
<< frame_info.detection_results[i].y2 << ")" << endl;
cout << "=============================================\n";
}
}
The metadata structure could be find in Edge Vision Analytics SDK Programming Guide : How to Use ADLINK Metadata in Chapter 5. Based on the structure, AdBatch can get the frame in vector and the inference data are stored in each frame based on the inference type: classification, detection, segmentation. Here, the sample use detection to illustrate get the metadata and print out obj_id, obj_label, prob and the box coordination to the terminal or cmd.
This sample illustrate how to get the image data and metadata inside the plugin. More details could be referenced in the document, Edge Vision Analytics SDK Programming Guide in Chapter 2.
This is a gstreamer plugin sample that wrote in python. Used to interpret classification inference result into human readable format and save in gstreamer metadata. Then you can using inference information in stream flow.
$ gst-launch-1.0 filesrc ! decodebin ! videoconvert ! advino model=YOUR_MODEL ! classifier_sample label=YOUR_LABEL class-num=1000 ! admetadrawer ! videoconvert ! ximagesink
This is a gstreamer plugin sample that wrote in python. It will generate five dummy boxs and randomly move in every frame. And boxs information will save in gstreamer metadata. Then you can using inference information in stream flow. This plugin was used for generate fake box to debug detection in other plugin.
$ gst-launch-1.0 filesrc ! decodebin ! videoconvert ! advino model=YOUR_MODEL ! dummy_box_sample ! admetadrawer ! videoconvert ! ximagesink
This is yolo interpret python sample, it will draw yolo's detection box and class.
User can inspect plugin to list configurable inference attribute like output blob
size, input width and height or anchor box.
Plugin design defaul to support YoloV3, however user can set other mask and anchor.
There is one YoloV3-tiny pipeline example at following.
Please reference to below link.
https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html
$ gst-launch-1.0 filesrc ! decodebin ! videoconvert ! advino model=YOUR_YOLO_MODEL ! adsample_yolo_py blob-size=26,52,13 threshold=0.8 input-width=416 input-height=416 label-file=YOUR_LABEL ! videoconvert ! ximagesink
$ gst-launch-1.0 filesrc ! decodebin ! videoconvert ! advino model=YOUR_YOLO_MODEL ! adsample_yolo_py threshold=0.4 input-width=416 input-height=416 label-file=YOUR_LABEL mask='(0,1,2),(3,4,5)' anchor='(10, 14),(23, 27),(37, 58),(81,82),(135,169),(344,319)' blob-size="26,13" ! videoconvert ! ximagesink
This is segmentation python translator sample, it will parse the segmentation result into adlink metadata.
User can inspect plugin to list configurable inference attribute like output blob size, input width and height or label id and label name.
filesrc ! decodebin ! videoconvert ! advino device=CPU model=YOUR_SEGMENTATION_MODEL ! segment_sample ! admetadrawer ! videoconvert ! ximagesink
Demonstrate how to write a plugin which can handle metadata and image buffer.
The main task of this sample is :
In this sample, the detail comment on the section with !!!ADLINK!!!' is included (but not exclusive) to guide user step-by-step from :
There is one pipeline example as following which will read detection box result from a YOLO model then draw result on image stream :
gst-launch-1.0 filesrc location=YOUR_VIDEO_FILE ! decodebin ! videoconvert ! adrt model=YOUR_YOLO_MODEL ! adsample_yolo_py input-width=416 input-height=416 label-file=YOUR_LABEL blob-size="52,26,13" ! adlink_plugin_sample ! videoconvert ! ximagesink
More details could be referenced in the document, Edge Vision Analytics SDK Programming Guide in Chapter 6.