AIM-Linux EdgeAI
Contents
Applications
Face Recognition
FaceView
Advantech FaceView application powered by CyberLink's FaceMe®, an industry-leading facial recognition engine, Advantech’s FaceView application provides precise and scalable real-time facial recognition for various AIoT applications in the retail, hospitality, and public safety fields.
Supported Platforms
- nVidia TX2: EPC-R7000
Installation
0. Before we start for FaceView, please make sure SDK components are installed well on your TX2 device.
1. Get the program file by contact with Advantech, e.g. FaceView_1.0.2.0522_aarch64.run
2. Put the self-extractable file into your device and execute it. Note: Make sure the network is connected!
$ chmod +x FaceView_1.0.2.0522_aarch64.run $ ./FaceView_1.0.2.0522_aarch64.run Verifying archive integrity... 100% All good. Uncompressing FaceView 100% installing FaceView application Press the ENTER key to continue. ************************************************************************ Installing Package... ************************************************************************ ... Preparing to unpack FaceView_1.0.2.0522_aarch64.deb ... Unpacking faceview (1.0.2.0522) ... Setting up faceview (1.0.2.0522) ... ************************************************************************ Done! ************************************************************************
3. Once it's done, you can find the FaceView application in /usr/local/FaceView/ folder.
4. If you want to uninstall FaceView application, use the command below.
$ sudo dpkg --remove FaceView
Setup Camera
We support two kinds of camera on EPC-R7000. One is USB webcam and the other is IP camera.
[USB Webcam]
It's easy to setup webcam by connecting USB with TX2 device.
The two camera products below are verified, but it's OK to use other products which support 1920x1080 or 1280x720 resolution.
- Microsoft LifeCam HD-3000
- Logitech BRIO V-U0040
[IP Camera]
EPC-R7000 supports PoE function, so you can run FaceView with IP camera for more flexible camera configurations.
The IP camera below is verified. We will take it as an example to set up.
- Vivotek_IB9360H
To configure IP camera for FaceView, we have to route RTSP streaming to video device.
1. Connect IP camera with TX2 device via Ethernet cable.
2. Assign corresponding IP address for the PoE port, e.g. 169.254.111.1/16.
3. Install V4L2Loopback utility & drivers.
$ sudo apt-get install v4l2loopback-utils
4. Route RTSP streaming to video device.
$ sudo modprobe v4l2loopback $ export RTSP_PATH="rtsp://viewer:inventec2017@169.254.6.42:5554/live.sdp" $ gst-launch-1.0 rtspsrc location="$RTSP_PATH" latency=300 ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! tee ! v4l2sink device=/dev/video0
Run Application
To execute FaceView application, navigate to /usr/local/FaceView/ folder and double-click FaceView icon, or run the application by command mode.
$ cd /usr/local/FaceView/ $ ./FaceView
Then, you will be asked to input license key. Note: Please make sure your network connection is OK to activate the license successfully.
If the key is valid, you are able to see camera preview screen from the application.
For details of operations, please refer to the FaceView user guide.
Set Up Build Environment
To develop QT application with FaceMe SDK, you have to install the packages listed below.
1. Install FaceMe SDK, e.g.FaceMe_SDK_Ubuntu18_ARM64_3.18.0.run
$ chmod +x FaceMe_SDK_Ubuntu18_ARM64_3.18.0.run $ ./FaceMe_SDK_Ubuntu18_ARM64_3.18.0.run ... What path do you want to install? (Press ENTER to use default)[/home/advrisc] Which detection models do you want to install? ML, and (Press ENTER to use default, DNN DNN-X)[DNN DNN-X] Which extraction models do you want to install? (Press ENTER to use default, H1 H2 H3 VH UH UH3)[H1 H3 VH UH3] Do you want to install Demo System? (Press ENTER to use default, yes(y)/no(n))[yes] Do you want to install Sample code? (Press ENTER to use default, yes(y)/no(n))[yes] Install SDK Path: /home/advrisc/FaceMeSDK Install Detection Model: DNN DNN-X Install Extraction Model: H1 H3 VH UH3 Install GPU support: no Install DemoSystem: yes Install Sample Code: yes Are you sure? (Press ENTER to use default, yes(y)/no(n))[yes]
2. Install pre-built Qt 5.14.2 binaries
$ tar zxvf Qt-5.14.2-Ubuntu18.04-ARM64.tar.gz $ cd Qt-5.14.2-U18.04-ARM64 $ sudo cp -a * / $ sudo vim ~/.profile PATH="/usr/local/Qt-5.14.2/bin:$PATH" $ source ~/.profile
3. Install QT creator
$ sudo apt-get install qtcreator
4. Install pre-built OpenCV 3.4.2 with CUDA & Jpeg1.5.3 support
$ tar zxvf OpenCV-3.4.2-aarch64-U18.04-JPEG1.5.3.tar.gz $ cd OpenCV-3.4.2-aarch64 $ sudo cp -a * /usr/local/
Traffic Analysis
IVS
IVS service is a web application which provides intelligent video analysis for traffic issues, such as vehicle detection, license recognition, car parking & traffic counting.
Supported Platforms
- nVidia TX2: EPC-R7000
Run Application
You can connect the host PC with EPC-R7000 with microUSB cable (RNDIS) or Ethernet. Then, open browser with specific URL to access IVS service.
- RNDIS: 192.168.55.1:5000/admin
- Ethernet: <your device IP address>:5000/admin
After login with correct account/password, you can configure the monitor regions as your requirements.
For more details, please refer to the IVS user guide document.
[Traffic]
[License Recognition]
[People Counting]
Inference Engines
Platform
nVidia TX2
NXP i.MX series
The i.MX 8M Plus family focuses on neural processing unit (NPU) and vision system, advance multimedia, andindustrial automation with high reliability.
- The Neural Processing Unit (NPU) operating at up to 2.3 TOPS
- Keyword detect, noise reduction, beamforming
- Speech recognition (i.e. Deep Speech 2)
- Image recognition (i.e. ResNet-50)
eIQ - A Python Framework for eIQ on i.MX Processors
PyeIQ is written on top of eIQ™ ML Software Development Environment and provides a set of Python classes
allowing the user to run Machine Learning applications in a simplified and efficiently way without spending time on
cross-compilations, deployments or reading extensive guides.
Installation
- Method 1: Use pip3 tool to install the package located at PyPI repository:
$ pip3 install eiq Collecting eiq Downloading [[https://files.pythonhosted.org/packages/10/54/7a78fca1ce02586a91c88ced1c70acb16ca095779e5c6c8bdd379cd43077/eiq-2.1.0.tar.gz https://files.pythonhosted.org/packages/10/54/7a78fca1ce02586a91c88ced1c70acb16ca095779e5c6c8bdd379cd43077/eiq-2.1.0.tar.gz]] Installing collected packages: eiq Running setup.py install for eiq ... done Successfully installed eiq-2.1.0
- Method 2: Get the latest tarball Download files and copy it to the board:
$ pip3 install <tarball>
Other eiq versions:
How to Run Samples
- Start the manager tool:
$ pyeiq
- The above command returns the PyeIQ manager tool options:
| Manager Tool Command | Description | Example |
| pyeiq --list-apps | List the available applications. | |
| pyeiq --list-demos | List the available demos. | |
| pyeiq --run <app_name/demo_name> | Run the application or demo. | # pyeiq --run object_detection_tflite |
| pyeiq --info <app_name/demo_name> | Application or demo short description and usage. | |
| pyeiq --clear-cache | Clear cached media generated by demos. | # pyeiq --info object_detection_dnn |
- Common Demos Parameters
| Argument | Description | Example of usage |
| --download -d | Choose from which server the models are going to download. Options: drive, github, wget. If none is specified, the application search automatically for the best server. |
/opt/eiq/demos# eiq_demo.py --download drive /opt/eiq/demos# eiq_demo.py -d github |
| --help -h | Shows all available arguments for a certain demo and a brief explanation of its usage. |
/opt/eiq/demos# eiq_demo.py --help /opt/eiq/demos# eiq_demo.py -h |
| --image -i | Use an image of your choice within the demo. |
/opt/eiq/demos# eiq_demo.py --image /home/root/image.jpg /opt/eiq/demos# eiq_demo.py -i /home/root/image.jpg |
| --labels -l | Use a labels file of your choice within the demo. Labels and models must be compatible for proper outputs. |
/opt/eiq/demos# eiq_demo.py --labels /home/root/labels.txt /opt/eiq/demos# eiq_demo.py -l /home/root/labels.txt |
| --model -m | Use a model file of your choice within the demo. Labels and models must be compatible for proper outputs. |
/opt/eiq/demos# eiq_demo.py --model /home/root/model.tflite /opt/eiq/demos# eiq_demo.py -m /home/root/model.tflite |
| --res -r | Choose the resolution of your video capture device. Options: full_hd (1920x1080), hd (1280x720), vga (640x480). If none is specified, it uses hd as default. If your video device doesn’t support the chosen resolution, it automatically selects the best one available. |
/opt/eiq/demos# eiq_demo.py --res full_hd /opt/eiq/demos# eiq_demo.py -r vga |
| --video_fwk -f | Choose which video framework is used to display the video. Options: opencv, v4l2, gstreamer (need improvements). If none is specified, it uses v4l2 as default. |
/opt/eiq/demos# eiq_demo.py --video_fwk opencv /opt/eiq/demos# eiq_demo.py -f v4l2 |
| --video_src -v | It makes the demo run inference on a video instead of an image. You can simply use the parameter “True” for it to run, specify your video capture device or even a video file. Options: True, /dev/video, path_to_your_video_file. |
/opt/eiq/demos# eiq_demo.py --video_src /dev/video3 /opt/eiq/demos# eiq_demo.py -v True /opt/eiq/demos# eiq_demo.py -v /home/root/video.mp4 |
Run Applications and Demos
- Applications
| Application Name | Framework | i.MX Board | BSP Release | Inference Core | Status |
| Switch Classification Image | TFLite:2.1.0 | RSB-3720 | 5.4.24_2.1.0 | CPU, GPU, NPU | PASS |
| Switch Detection Video | TFLite:2.1.0 | RSB-3720 | 5.4.24_2.1.0 | CPU, GPU, NPU | PASS |
- Demos
| Demo Name | Framework | i.MX Board | BSP Release | Inference Core | Status |
| Object Classification | TFLite:2.1.0 | RSB-3720 | 5.4.24_2.1.0 | GPU, NPU | PASS |
| Object Detection SSD | TFLite:2.1.0 | RSB-3720 | 5.4.24_2.1.0 | GPU, NPU | PASS |
| Object Detection YOLOv3 | TFLite:2.1.0 | RSB-3720 | 5.4.24_2.1.0 | GPU, NPU | PASS |
| Object Detection DNN | OpenCV:4.2.0 | RSB-3720 | 5.4.24_2.1.0 | CPU | PASS |
| Facial Expression Detection | TFLite:2.1.0 | RSB-3720 | 5.4.24_2.1.0 | GPU, NPU | PASS |
| Fire Classification | TFLite:2.1.0 | RSB-3720 | 5.4.24_2.1.0 | GPU, NPU | PASS |
| Fire Classification | ArmNN:19.08 | RSB-3720 | 5.4.24_2.1.0 | GPU, NPU | PASS |
| Pose Detection | TFLite:2.1.0 | RSB-3720 | 5.4.24_2.1.0 | GPU, NPU | PASS |
| Face/Eyes Detection | OpenCV:4.2.0 | RSB-3720 | 5.4.24_2.1.0 | GPU, NPU | PASS |
Applications Example - Switch Detection Video
This application offers a graphical interface for users to run an object detection demo using either CPU or GPU/NPU to perform inference on a video file.
- Run the Switch Detection Video demo using the following line:
$ pyeiq --run switch_video
- Type on CPU or GPU/NPU in the terminal to switch between cores.
- This runs inference on a default video:
Demos Example - Running Object Detection SSD
Object detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantic objects of a certain class (such as humans, buildings, or cars) in digital images and videos. Well-researched domains of object detection include face detection and pedestrian detection. Object detection has applications in many areas of computer vision, including image retrieval and video surveillance.
- Run the Object Detection Default Image demo using the following line:
$ pyeiq --run object_detection_tflite
* This runs inference on a default image:
- Run the Object Detection Custom Image demo using the following line:
$ pyeiq --run object_detection_tflite --image=/path_to_the_image
- Run the Object Detection Video File using the following line:
$ pyeiq --run object_detection_tflite --video_src=/path_to_the_video
- Run the Object Detection Video Camera or Webcam using the following line:
$ pyeiq --run object_detection_tflite --video_src=/dev/video<index>
References
pyeiq: https://community.nxp.com/t5/Blogs/PyeIQ-3-x-Release-User-Guide/ba-p/1305998