EVA SDK IDE User's Manual

Manual Rev.: 1.3

Revision Date: September xx, 2023

Part Number: 50M-00011-1030

Preface

Copyright

Copyright © 2023 ADLINK Technology, Inc. This document contains proprietary information protected by copyright. All rights are reserved. No part of this manual may be reproduced by any mechanical, electronic, or other means in any form without prior written permission of the manufacturer.

Disclaimer

The information in this document is subject to change without prior notice in order to improve reliability, design, and function and does not represent a commitment on the part of the manufacturer. In no event will the manufacturer be liable for direct, indirect, special, incidental, or consequential damages arising out of the use or inability to use the product or documentation, even if advised of the possibility of such damages.

Trademarks

Product names mentioned herein are used for identification purposes only and may be trademarks and/or registered trademarks of their respective companies.

Revision History

1 Introduction

The ADLINK Edge Vision Analytics (EVA) SDK provides an integrated development environment (IDE) for developers wanting to build computer vision and video analytic solutions with deep learning technology. The EVA SDK IDE is based on PyFlow, a general purpose runtime extendable Python Qt visual scripting framework, to expand the EVA SDK. The EVA SDK IDE allows users to quickly establish GStreamer pipelines and easily modify their properties.

This document introduces the EVA SDK IDE interface including how to generate GStreamer pipelines and how to get ROI information from ADLINK GStreamer elements. This document does not include information on PyFlowBase features. For more details about the PyFlowBase, refer to https://pyflow.readthedocs.io/en/latest/index.html.

Refer to the EVA SDK Installation Guide for information on installing any additional required software.

1.1 Use EVA SDK IDE

On Windows, after installing the EVA SDK and required software, execute the EVA SDK IDE using one of the following methods:

Method 1: Run the desktop shortcut.

From the Windows Desktop, double-click the EVA SDK IDE icon.

The shortcut will automatically set the environment variables and execute the EVA SDK IDE.

Method 2: Manually run the environment variable settings, and then execute the EVA SDK IDE.

Open a command prompt in Windows and run the following command to clear the GStreamer cache and set the environment variables.

Note: The environment variables are removed when the command prompt is closed.

After setting the environment variables, execute the EVA SDK IDE.

On Linux (including x86 and AArch64), after installing the EVA SDK and required software, use the following command to execute the EVA SDK IDE.

Note: The environment variables are removed when the command prompt is closed.

1.2 Use EVA SDK IDE with GigE Vision Camera

If you would like to use GigE vision vamera and euresyssrc plugin, follow these method to set up the environment and execute EVA SDK IDE.

 On Linux (Ubuntu 20.04), after installing the EVA SDK and required software, use the following command to execute the EVA SDK IDE.

Note: The environment variables are removed when the command prompt is closed.

For more details about ADLINK EVA, refer to https://eva-support.adlinktech.com.

2 EVA SDK IDE User Interface

This chapter introduces the EVA SDK IDE user interface (UI) and common UI operations.

2.1 Introduction

This section is a detailed description of the UI.

Figure 1: EVA SDK IDE User Interface (UI)

Table 1: EVA SDK IDE User Interface (UI) Items

Item	Description
Canvas	This area is used to edit elements and pipelines. Note: EVA SDK IDE only supports a single canvas to display the graph.
Element (Node)	The element is the node from PyFlowGstreamer. Only support the configurable GStreamer element list.
Pipeline	Use this area to link the elements as a pipeline.
GStreamer Control Buttons	After creating a pipeline, click “Play”, “Pause” or “Stop” to control it. Play: plays a GStreamer pipeline Pause: pauses a GStreamer pipeline. Stop: stops a GStreamer pipeline.
Play Control Buttons	After clicking “Play” or “Pause”, click “Backward” to slow down or “Forward” to speed up the video. After clicking “Pause”, click “Frame-by-Frame” to play next frame. Note:These buttons only support video files, do not support live streaming. Backward: click will reduce the speed rate.(1/2x) speed rate range : 16x~0.015625x Forward: click will increase play speed. (2x) speed rate range : 16x~0.015625x Frame-by-Frame: click to play next frame in pause state.
Properties	Select an element to show its editable properties. For more details, refer to Edit Properties of Elements
ROI Viewer	Shows the width, height, format of the video stream, the execution time, the inference information (AI information), and the translator information. For more details, refer to Reviewing the ROI information.
Output	Display the log of the EVA IDE.
Alignment Buttons	Select the nodes and click the alignment button to align them. Aligns the selected nodes to the left node. Aligns the selected nodes to the right node. Aligns the selected nodes to the top node. Aligns the selected nodes to the bottom node.
Canvas Screenshot	Takes a screenshot of the canvas and saves the image to file. The default image file format is PNG. Click the down arrow to change the image format.

2.2 UI Operations

2.2.1 Move the Canvas

Method 1: Hold down <Alt> and the left mouse button to drag the canvas.

Method 2: Hold down the middle mouse button to drag the canvas.

2.2.2 Zoom the Canvas

Method 1: Scroll the mouse wheel to zoom the canvas.

Method 2: Press  <Ctrl> + <=> to zoom in the canvas; press <Ctrl> + <-> to zoom out the canvas.

2.2.3 Hotkeys

Quickly zoom in and display a node: Click the node (element) on the canvas and press <F>.

Quickly display all nodes: Press <H>.

3 Create an Empty Canvas

Method 1: Execute the EVA SDK IDE to start with an empty canvas.

Method 2: From the File menu, select New file, to create an empty canvas.

Note: 

New file will clean all elements and pipelines in the canvas.

4 Creating GStreamer Pipelines

This chapter describes how to create and run GStreamer Pipelines in the EVA SDK IDE.

4.1 Adding an Element

4.1.1 Right-click Method

Use the following steps to add an element using the right-click method.

1. Right click on the canvas.
   
2. Select PyFlowGstreamer and choose the element, or enter element name in enter node name to search the specified element.
     
3. Drag the specified element to the canvas.
   

4.1.2 NodeBox Method

Use the following steps to add an element using the NodeBox method.

1. Tools -> PyFlowBase -> NodeBox
     
2. Select PyFlowGstreamer and choose the element, or enter element name in enter node name to search the specified element.
     
3. Drag the specified element to the canvas.
   

4.1.3 Pin Hint Method

Use the following steps to add an element using the Pin Hint method.

1. Left-click a source pin of an element (such as src pad or sink pad) and drag it to an empty space.
   
    
2. The NodeBox will display on the screen. Enter an element name in enter node name to search the specified element.
   
    

Note: Pin Hint can filter out incompatible pins between two nodes (elements). However, the src pin or sink pin of some elements has a special format that causes another element’s pin to not be found via Pin Hint, even when the two elements should be connectable. To fix this issue, use the NodeBox method to search the element and drag it to the canvas and link the elements.

For example, glupload element's src pin cannot find glcolorconvert element via Pin Hint.

4.2 Deleting Elements

To delete an element, select it and then press <Delete>.

4.3 Editing Element Properties

Note: Properties must be modified when the pipeline state is stopped.

1. Select an element to modify its properties. For example, select videotestsrc to display its properties. 
2. Click pattern, and select ball.

4.4 Linking/Unlinking Elements

4.4.1 Linking Elements

To connect elements, left-click the source’s element GstPin and drag it to the target’s GstPin. GstPin includes src and sink. In the following example, The left figure is a sink GstPin connected to a src GstPin. The right figure is a src GstPin connected to a sink GstPin.

The following figure is linked.

Note: If using pin hints, the connection will link automatically.

If the sink GstPin or src GstPin supports multiple streams, such as a tee element that supports multiple output streams, multiple elements can be connected.

4.4.2 Unlinking Elements

Method 1: Click the connection near sink or src GstPad to unlink the elements.

Method 2: Right-click on the link line, and then click “Disconnect” to unlink the elements.

4.5 Pipeline Controls

Note: All operations must be performed when the pipeline is stopped, otherwise unpredictable results can occur.

4.5.1 Play a Pipeline

After creating a complete pipeline in the IDE canvas, click the play button () to play the pipeline.

4.5.2 Pause a Pipeline

After creating a complete pipeline in the IDE canvas, click the pause button () to pause the pipeline.

4.5.3 Stop a Pipeline

After creating and playing a complete pipeline in the IDE canvas, click the stop button () to stop the pipeline.

4.6 Collapsing/Expanding Element Properties

Click  to collapse an element, and  to expand it.

4.7 Play Speed Controls

4.71. Slow down 

After creating a complete pipeline in the IDE canvas and clicking the play button or the pause button, click the backward button () to slow down.

4.7.2 Speed up

After creating a complete pipeline in the IDE canvas and clicking the play button or the pause button, click the forward button (  ) to speed up.

4.7.3 Frame-by-Frame to play

After creating a complete pipeline in the IDE canvas and clicking the pause button, click the frame-by-frame button () to speed up.

4.8 Use Cases

This section includes examples on how to use the EVA SDK IDE.

4.8.1 Use Case 1

This example describes how to play a test video via OpenGL.

Use the following gst-launch command.

First, add a videontestscr element, a videoconvert element, and a glimagesink element to the canvas in the EVA SDK IDE. The videotestsrc element must connect to the videoconvert element, and the videoconvert element must connect to the glimagesink element as shown in the figure below.

Click the play button to show the video in an OpenGL renderer.

Click the stop button to stop the pipeline.

4.8.2 Use Case 2

This example describes how to play a test video with a ball pattern via OpenGL.

Following from the example in Use Case 1, in the stop state, select the videotestsrc element and modify its pattern property to ball.

Click the play button to show the ball video in an OpenGL renderer.

Click the stop button to stop the pipeline.

4.9 Gst Commands

To generate Gst commands from pipelines in the canvas, click Play GStreamer pipeline. When the pipeline’s state changes to playing, it will send a play event to trigger gst-command generation. The gst-lauch-1.0 commands will be generated in C:\ADLINK\eva\IDE\command.txt on Windows and /home/<user>/adlink/eva/IDE/command.txt on Linux.

If there is more than one pipeline in the canvas, all of the commands will be included in the file.

In the following example, there are two pipelines in the canvas.

After playing, the gst commands are created in command.txt.

d3dvideosink element only supports Windows. Please use another element such as xvimagesink for testing.

Notes:

1. The command file will be erased each time GStreamer is played. Be sure to back up command.txt if you want to save the Gst commands.

2.  The Gst commands will not generate immediately after clicking the playing pipeline because it is waiting for the play event. For example, a pipeline is playing but a plugin delay occurs that causes an event delay, resulting in the command being generated later when the play event is sent.

5 ROI Information

5.1 Reviewing the ROI Information

The ROI Viewer tool will show the ROI information that includes the general information, the inference information, and the translator information.

The general information includes width, height, format of the video, and the execution time (ms); the inference information (if any) in the AI element, such as adrt, advino, adonnx, it includes the ROI data and blobs; the translator information (if any) in the translator element, such  adtrans_xxx, it includes the translatoer category and data.

After creating a complete pipeline, click ViewImage,  or , on the specified element to capture its ROI information. If there is no video source, the screen will show noImage.

: Close state.

: Open state.

Click the play button to start the pipeline; the image (if any) will display. Then click the specified element; the ROI Viewer will show the ROI information.

After playing and showing the ROI information, you can click the pause button to pause the streams, or click the play button to keep playing them.

Supported image formats:

• BGR
• BGRA
• BGRX
• RGB
• RGBA
• RGBX
• GRAY
• NV12

Note: The pipeline performance will be affected when ViewImage is enabled, such as frame drops and video delays. Enabling ViewImage will increase the latency of the entire pipeline, with performance impact depending on video size and the number of enabled ViewImages. If the latency is too long, the frame timestamp will not match the synchronization plugin, such as ximagesink, resulting in dropped frames.

The ROI Viewer is displayed on the right side of the EVA IDE by default. If you accidentally close the ROI Viewer, you can open it from the menu bar.

Tools -> PyFlowGstreamer -> ROI Viewer

5.2 Drawing the customized ROI points  

If you would like to select the specified region of interest, you can use the ROI Creator with adroi_addroi element to generate it.

Open the ROI creator from the menu bar: 

Tools -> PyFlowGstreamer -> ROI Creator

Show images and get current settings:

1. Create a GStreamer pipeline include adroi_addroi element
   Note: About adroi_addroi element, refer to the EVA SDK User’s Manual.
2. Open ViewImage
3. Play the GStreamer pipeline
4. Pause the GStreamer pipeline(Optional)
5. Click adroi_addroi node
6. ROI Creator will show the current frame and ROI points value in ROI table

Add a new ROI setting:

• Rectangle ROI: (x1, y1), (x2, y2), the range is 0~1.
• Reset: clean the text content
• New ROI: Add the ROI points to ROI table.

There are 2 methods to add a new ROI setting

Method 1: 

1. Enter value in (x1,y1),(x2,y2)
2. Click New ROI to add a ROI setting to ROI table, and the frame will show the new rectangle.

Method 2:

1. Click the frame of the ROI Creator
2. Click Add to (x1,y1) or Add to (x2,y2)
3. Add New ROI to add a ROI setting to ROI table, and the frame will show the new rectangle.

ROI Table:

            When click adroi_addroi node, ROI table will show the current ROI points.

            In the table, you can click “delete” to select the specified points or click Select All to select all items in the ROI Table. When click Remove Selected to remove the selected points setting.

Submit:

            Click Submit to update the “points” property in the adroi_addroi node.

6 Reviewing GStreamer Pipeline Graphs

EVA SDK IDE supports GStreamer pipeline graphs for debugging. For more details, refer to https://gstreamer.freedesktop.org/documentation/tutorials/basic/debugging-tools.html?gi-language=c#getting-pipeline-graphs

Note: This function only supports Linux x86 and Linux ARM platforms.

After creating a pipeline, open the pipeline graphviz windows from the menu bar:

Tools -> PyFlowGstreamer -> Pipeline graphviz

Pipeline graphviz will display at the bottom left of the EVA SDK IDE.

You can click the  button to split the windows and resize them to view the pipeline graph.

If you modify the pipeline on the canvas, you can click Refresh to reload the new graph.

7 Save/Load a GStreamer Pipeline

7.1 Save a GStreamer Pipeline

Select File -> Save or File -> Save as a pygraph file.

7.2 Load a GStreamer Pipeline

Select File -> Load to load a pygraph file.

Note:

Loading a pygraph file will erase the current elements and pipeline in the canvas. It is recommended that you save the graph before loading another file.

If a pygraph file contains unsupported elements, the elements will not be loaded to the canvas. For example, Windows does not support ximagesink. If a pygraph file includes ximagesink, the canvas will not include it, so you will need to modify the pipeline after loading it.

8 Whitelist Plugins and Elements

EVA SDK IDE only supports the specified GStreamer elements and plugins. This chapter introduces how to expand elements and plugins (C/C++ based) in the whitelist.

After first running EVA SDK IDE, a whitelist will be generated in C:\ADLINK\eva\IDE\config\element_list.txt (Windows), or /home/<user>/adlink/eva/IDE/config/element_list.txt (Linux).

The default content of the element_list file is empty.

Use the following rules to enable an element or plugin in EVA SDK IDE:

• [Element_name]
• [Plugin_name]
• +[Element_name]
• +[Plugin_name]

Use the following rule to disable an element or plugin in EVA SDK IDE:

• -[Element_name]

For example, add the autovideosink element and disable the rtpdec element:

1. Open element_list.txt.
2. Enter the following in element_list.txt.

   +autovideosink -rtpdec

3. Restart EVA SDK IDE. 

Note:

1. Only C/C++-based plugins and elements are supported. Enabling python plugins will result in unpredictable behavior.
2. If other third-party plugins or GStreamer plugins excluded by the default whitelist have any issues, contact the developers directly.

9 Wizard Tool

The EVA SDK IDE provide a simple guide tool for beginners to familiar with GStreamer and use EVA. The users only follow the step that the wizard tool will help the users to generate a GStreamer pipeline.

Note: 

1. The wizard tool’s configuration file is generated in C:\ADLINK\eva\IDE\config\wizard.json (Windows), or /home/<user>/adlink/eva/IDE/config/wizard.json (Linux) when the wizard tool is executed for the first time.
2. If the wizard tool’s configuration file already exists, the wizard tool will use the existing configuration file.
3. If you need to restore to default, remove the wizard.json and restart the wizard tool. 

10 Packaging GStreamer Pipeline

The EVA SDK IDE provides 2 tools for packaging the GStreamer pipeline.

• Package the GStreamer pipeline to a zip file for the deployment
• Package the GStreamer pipeline as reference code for the application integration

10.1 Package pipeline to a zip file

The EVA SDK IDE provide packaging the GStreamer pipeline to a zip file for deploying to multiple machines with the same hardware configuration.

Create a GStreamer pipeline and play the GStreamer pipeline to confirm the pipeline can work normal. And then click “Create pipeline package”, the EVA SDK IDE will package the pipeline to zip file.

Package -> Create pipeline package

The zip file will be located at C:\ADLINK\eva\IDE\zip on Windows and /home/<user>/adlink/eva/IDE/zip on Linux by default.

When the packaging procedure is done, the message box will pop up to inform the file path.

The zip file includes python plugins, the necessary files for the specified GStreamer pipeline (such as model, label, and video file), and a python script for running the GStreamer pipeline (run.py).

Open a terminal, set EVA environment variable and run main.py, you can see the demo result.

On Windows, 

On Linux, run the following commands

Note: Modify main.py for your application scenario.

You can refer to eva.html and main.html to modify main.py for your application screnario.

Archive:

Package -> Convert video-sink to appsink -> Archive

Archive the specified directory to zip file. The default path is C:\ADLINK\eva\IDE\workspace on Windows and /home/<user>/adlink/eva/IDE/workspace on Linux.

When the archiving procedure is done, the message box will pop up to inform the file path.

If you choose incorrect directory, the EVA SDK IDE will pop up Error message.

11 Use Limitations

11.1 EVA SDK IDE Limitations

• The EVA SDK IDE does not support custom python plugins/elements.
• The EVA SDK IDE will not retain the dock tools added by users.
• The EVA SDK IDE will not restrict users from mismatched translators to connect the inference models. Confirm that the inference model elements will connect to the matched translator. Otherwise, the incorrect translator may try to access memory without permission, causing the EVA SDK IDE to crash. For example, a Yolo inference model must use the adtrans_yolo element.
• All operations must be performed when the pipeline is stopped, otherwise unpredictable conditions will occur. To fix this issue, restart the EVA SDK IDE.
• The pipeline performance will be affected when ViewImage is enabled, such as frame drops and video delays.
• If Pin Hint cannot find a specified element because of its special format, use the NodeBox method to search for the element and drag it to the canvas to link the elements.
• There are graphics and debugging functions in the EVA SDK IDE that cause performance loss. If you would like to pursue optimization, use the command tool (gst-launch).

11.2 PyFlow Known Issues

• Abnormal mouse operations may make PyFlow crash. To fix this issue, restart the EVA SDK IDE.
• When an element’s pin has a special capability, it can cause another element’s pin to not be found via Pin Hint when the two elements should be connectable.

11.3 GStreamer Pipeline Limitations

• There cannot be any unlinked source elements such as filesrc in the canvas.
• There cannot be any unlinked sink elements such as autovideosink in the canvas.
• There cannot be any unlinked bin elements such as decodebin in the canvas.

11.4 glimagesink Element Limitations

• The glimagesink element only supports one object in the pipeline on Windows.
• Only one glimagesink node can be added to the canvas on Windows.
• When adding a GStreamer element to the canvas, a new gstreamer object will be created automatically. If an existing glimagesink element is deleted from the canvas and a new one is added, there will be multiple glimagesink objects.

9.5 Inference Engine Element Limitations

• The inference node (such as advino, adrt and adonnx) will cache the model in the memory after pressing the "Play" button to avoid the model loading every time (around 5-30 seconds depending on the model size). Remove the unused inference node to free the memory space when editing the pipeline in EVA IDE.

Safety Instructions

Read and follow all instructions marked on the product and in the documentation before you operate your system. Retain all safety and operating instructions for future use.

• Please read these safety instructions carefully.
• Please keep this User‘s Manual for later reference.
• Read the specifications section of this manual for detailed information on the operating environment of this equipment.
• When installing/mounting or uninstalling/removing equipment, turn off the power and unplug any power cords/cables.
• To avoid electrical shock and/or damage to equipment:
  • Keep equipment away from water or liquid sources.
  • Keep equipment away from high heat or high humidity.
  • Keep equipment properly ventilated (do not block or cover ventilation openings).
  • Make sure to use recommended voltage and power source settings.
  • Always install and operate equipment near an easily accessible electrical socket-outlet.
  • Secure the power cord (do not place any object on/over the power cord).
  • Only install/attach and operate equipment on stable surfaces and/or recommended mountings.
  • If the equipment will not be used for long periods of time, turn off and unplug the equipment from its power source.
• Never attempt to fix the equipment. Equipment should only be serviced by qualified personnel.

Please visit the Contact page at www.adlinktech.com for information on how to contact the ADLINK regional office nearest you.