¶ What can you learn from this sample code?
Through this sample code, you can learn:
- Create a command pipeline ended with appsink and capture the frame data through OpenCV Wrapper:VideoCapture when using a webcam device.
- Write frame data back to command pipeline started with appsrc through OpenCV Wrapper:VideoWriter.
- Mixure usage of the GSTREAMER feature element in pipeline and the library in application.
¶ Essential knowledge
- Ever execute the GSTREAMER pipeline command in terminal(Linux) or cmd(Windows).
- Knowledge of two gstreamer elements: appsrc and appsink.
- OpenCV build processes with gstreamer configuration enabled.
- Basic OpenCV VideoCapture and VideoWriter operation in C/C++.
¶ About this sample code
This sample shows how to generate a simple pipeline using OpenCV Wrapper: VideoCapture retrieve the video data in application. After retrieve the video data, use OpenCV Wrapper: VideoWriter to push the video data back to another pipeline. This sample illustrates the appsink with VideoCapture and appsrc with VideoWriter. OpenCV provide this Wrapper for gstreamer user to quick establish the pipeline, but this build feature is disabled when build the OpenCV in default. If user is going to use OpenCV wrapper for gstreamer, this build feature is required enabled in configuring before the
OpenCV library is build.
¶ Description
To grab the frame from the command pipeline with appsink, the OpenCV VideoCapture constructor requires two signatures, the pipeline string and the API preference (enum cv::CAP_GSTREAMER). Take linux as eaxmple, similar to windows, to grab the frame from the v4l2src element of the pipeline, provide the pipeline definition to VideoCapture as in the following example:
VideoCapture cap("v4l2src ! video/x-raw, format=BGR, width=640, height=480, framerate=30/1 ! appsink", CAP_GSTREAMER);
The frame can then be captured via VideoCapture:
Mat frame;
while(true)
{
cap.read(frame);
// do your process ...
}
To send the frame to the pipeline with appsrc, the function provided by the VideoWriter requires the pipeline string, the API preference, and other parameters like fps, frame size, and color flag. The signatures are required for the pipeline caps filter settings. To send the frame data to the pipeline with appsrc, the pipeline definition must be provided to VideoWriter, as in the following example.
cv::VideoWriter writer.open("appsrc ! videoconvert ! video/x-raw, width=640, height=480, framerate=30/1 ! clockoverlay ! videoconvert ! d3dvideosink sync=false", CAP_GSTREAMER, 0, 30, cv::Size(640, 480), true);
After retrieving the frame data, we can do some logics or algorithms before sending them to another command pipeline through VideoWriter. Here we asume we need to resize the frame:
cv::resize(frame,frame,Size(640,480)); // do some image processings here ...
When finish processing in application, we can provide the command pipeline in VideoWriter. The target pipeline settings include, frame size (640x480), frame rate (30), and color format that fits the OpenCV default BGR color format. In this pipeline, clockoverlay element is used to put the local time on the frame to denotes the processing in the pipeline not in application. The pipeline then shows the frame in the xwindows via the ximagesink element. Feed the frame into the pipeline by writing it directly, as in the following example:
writer.write(resize_frame);
¶ Run this sample code
Go to the folder of the binary and run binary in terminal or cmd:
$ ./generate-a-basic-pipeline-in-opencv
and you will see the two display windows:
- Display the retrieved frame from command pipeline after doing resize in aplication:
- Display the clock overlay result in the pipeline:![generate-a-basic-pipeline-in-opencv-1.png]
See the second window exists the time overlayed on the frame comes from the pipeline element, clockoverlay.