Difference between revisions of "Edge AI SDK/AI Framework/Hailo"

From ESS-WIKI
Jump to: navigation, search
(→‎Edge AI SDK / App ( Object Detection ))
(→‎Hailo-8 Utilization)
 
(14 intermediate revisions by 2 users not shown)
Line 8: Line 8:
 
 
 
 
  
−
= AI Applications =
+
 
  
−
<span style="font-size:larger;">[https://hailo.ai/products/hailo-software/hailo-ai-software-suite/#sw-tappas TAPPAS] is a solution designed to streamline the development and deployment of edge applications demanding high AI performance. This reference application software package empowers users to expedite their time-to-market by minimizing the development workload. TAPPAS encompasses a user-friendly set of fully operational application examples based on GStreamer, featuring pipeline elements and pre-trained AI tasks. These examples leverage advanced Deep Neural Networks, highlighting Hailo's AI processors' top-notch throughput and power efficiency. Furthermore, TAPPAS serves as a demonstration of Hailo's system integration capabilities, showcasing specific use cases on predefined software and hardware platforms. Utilizing TAPPAS simplifies integration with Hailo's runtime software stack and offers a starting point for users to fine-tune their applications. By demonstrating Hailo's system integration scenarios on both predefined software and hardware platforms, it can be used for evaluations, reference code, and demos. This approach effectively accelerates time to market, streamlines integration with Hailo's runtime software stack, and provides customers with a foundation to fine-tune their applications.
+
= Applications =
  
−
Refer to [https://github.com/hailo-ai/tappas github-TAPPAS]
+
<span style="font-size:larger;">[https://hailo.ai/products/hailo-software/hailo-ai-software-suite/#sw-tappas TAPPAS] is a solution designed to streamline the development and deployment of edge applications demanding high AI performance. This reference application software package empowers users to expedite their time-to-market by minimizing the development workload. TAPPAS encompasses a user-friendly set of fully operational application examples based on GStreamer, featuring pipeline elements and pre-trained AI tasks. These examples leverage advanced Deep Neural Networks, highlighting Hailo's AI processors' top-notch throughput and power efficiency. Furthermore, TAPPAS serves as a demonstration of Hailo's system integration capabilities, showcasing specific use cases on predefined software and hardware platforms. Utilizing TAPPAS simplifies integration with Hailo's runtime software stack and offers a starting point for users to fine-tune their applications. By demonstrating Hailo's system integration scenarios on both predefined software and hardware platforms, it can be used for evaluations, reference code, and demos. This approach effectively accelerates time to market, streamlines integration with Hailo's runtime software stack, and provides customers with a foundation to fine-tune their applications.</span>
 +
 
 +
<span style="font-size:larger;">Refer to [https://github.com/hailo-ai/tappas github-TAPPAS]</span>
  
 
&nbsp;
 
&nbsp;
  
−
== Edge AI SDK&nbsp;/ App ( Object Detection ) ==
+
&nbsp;
 +
 
 +
== <span style="font-size:larger;">Edge AI SDK&nbsp;/ Application</span> ==
 +
 
 +
<span style="font-size:larger;">Quick Start / Application / Video or WebCam / VPU</span>
 +
 
 +
<span style="font-size:larger;">[[File:Objdet.png|800x450px|ObjectDetection]]</span>
  
−
Quick Start / Application ( Objection Detection ) / Video or WebCam / VPU
+
<span style="font-size:larger;">&nbsp;</span>
  
−
[[File:Objdet.png|800x450px|ObjectDetection]]
+
{| border="1" cellpadding="1" cellspacing="1" style="width: 500px;"
 +
|-
 +
| style="width: 154px;" | <span style="font-size:larger;">Application</span>
 +
| style="width: 179px;" | <span style="font-size:larger;">Model</span>
 +
|-
 +
| style="width: 154px;" | <span style="font-size:larger;">Object Detection</span>
 +
| style="width: 179px;" | <span style="font-size:larger;">yolov5m_wo_spp_60p.hef</span>
 +
|-
 +
| style="width: 154px;" | <span style="font-size:larger;">Person Detection</span>
 +
| style="width: 179px;" | <span style="font-size:larger;">yolov5s_personface_reid.hef / repvgg_a0_person_reid_2048.hef</span>
 +
|-
 +
| style="width: 154px;" | <span style="font-size:larger;">Face Detection</span>
 +
| style="width: 179px;" | <span style="font-size:larger;">arcface_mobilefacenet_v1.hef / scrfd_10g.hef</span>
 +
|-
 +
| style="width: 154px;" | <span style="font-size:larger;">Pose Estimation</span>
 +
| style="width: 179px;" | <span style="font-size:larger;">centerpose_regnetx_1.6gf_fpn.hef</span>
 +
|}
  
−
&nbsp;
+
= <span style="font-size:larger;">Benchmark</span> =
  
−
= Benchmark =
+
<span style="font-size:larger;"><span style="font-size:larger;">In order to measure FPS, power and latency of the Hailo Model Zoo networks you can use the HailoRT command line interface. For more information please refer to the HailoRT documentation in [https://github.com/hailo-ai/hailo_model_zoo/blob/master/docs/BENCHMARKS.rst link].</span></span>
  
−
<span style="font-size:larger;">In order to measure FPS, power and latency of the Hailo Model Zoo networks you can use the HailoRT command line interface. For more information please refer to the HailoRT documentation in [https://github.com/hailo-ai/hailo_model_zoo/blob/master/docs/BENCHMARKS.rst link].
+
<span style="font-size:larger;"><span style="font-size:larger;">&nbsp;</span></span>
  
−
&nbsp;
+
<span style="font-size:larger;"><span style="font-size:larger;">&nbsp;</span></span>
  
−
&nbsp;
 
  
−
== Hailo-8 Benchmark ==
+
== <span style="font-size:larger;"><span style="font-size:larger;">Hailo-8 Benchmark</span></span> ==
  
−
<span style="font-size:larger;">hailortcli benchmark <Hailo's Model .hef ></span>
+
<span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">hailortcli benchmark <Hailo's Model .hef ></span></span></span>
 
<pre>hailortcli benchmark resnet_v1_18.hef</pre>
 
<pre>hailortcli benchmark resnet_v1_18.hef</pre>
  
−
[[File:Hailortcli benchmark.png|800x450px|benchmark]]
+
<span style="font-size:larger;"><span style="font-size:larger;">[[File:Hailortcli benchmark.png|800x450px|benchmark]]</span></span>
  
−
&nbsp;
+
<span style="font-size:larger;"><span style="font-size:larger;">&nbsp;</span></span>
 +
 
 +
<span style="font-size:larger;"><span style="font-size:larger;">&nbsp;</span></span>
 +
 
 +
== <span style="font-size:larger;"><span style="font-size:larger;">Edge AI SDK&nbsp;/ Benchmark</span></span> ==
  
−
&nbsp;
+
<span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">Evaluate the Hailo-8&nbsp;performance&nbsp;with Edge AI SDK.</span></span></span>
  
−
== Edge AI SDK&nbsp;/ Benchmark ==
+
<span style="font-size:larger;"><span style="font-size:larger;">[[File:Benchmark.png|800x500px|Benchmark]]</span></span>
  
−
<span style="font-size:larger;">Evaluate the Hailo-8&nbsp;performance&nbsp;with Edge AI SDK.</span>
+
<span style="font-size:larger;"><span style="font-size:larger;">&nbsp;</span></span>
  
−
[[File:Benchmark in EdgeAI Suite 1.png|800x500px|Benchmark in EdgeAI Suite 1.png]] &nbsp;
 
  
−
= HailoRT CLI =
+
= <span style="font-size:larger;"><span style="font-size:larger;">HailoRT CLI</span></span> =
  
−
<span style="font-size:larger;">HailoRT CLI - a command line application used to control the Hailo device, run inferences, collect statistics and device events, etc. Use "--help" to exhibit more usages.
+
<span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">HailoRT CLI - a command line application used to control the Hailo device, run inferences, collect statistics and device events, etc. Use "--help" to exhibit more usages.</span></span></span>
 
<pre>hailortcli --help</pre>
 
<pre>hailortcli --help</pre>
−
 
−
Refer to [https://github.com/hailo-ai/hailort/tree/master github-hailort]
 
  
−
&nbsp;
+
<span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">Refer to [https://github.com/hailo-ai/hailort/tree/master github-hailort]</span></span></span>
 +
 
 +
<span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">&nbsp;</span></span></span>
 +
 
 +
<span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">&nbsp;</span></span></span>
  
−
&nbsp;
 
  
−
== Hailo-8 Utilization ==
+
== <span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">Hailo-8 Utilization</span></span></span> ==
 
<pre>hailortcli monitor</pre>
 
<pre>hailortcli monitor</pre>
  
−
[[File:Usage.png|800x400px|Usage.png]]
+
<span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">[[File:EdgeAISDK_Hailo_Usage.jpg|800x400px|Hailo VPU Usage]]</span></span></span>
 +
 
 +
<span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">&nbsp;</span></span></span>
  
 
&nbsp;
 
&nbsp;
  
−
== Hailo-8 Temperature ==
+
== <span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">Hailo-8 Temperature</span></span></span> ==
 
<pre>python ts_monitoring_2023Sep07.py</pre>
 
<pre>python ts_monitoring_2023Sep07.py</pre>
  
−
&nbsp; [[File:Thermal in hailortcli.png|800x150px|Thermal in hailortcli.png]]
+
<span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">&nbsp; [[File:Thermal in hailortcli.png|800x150px|Thermal in hailortcli.png]]</span></span></span>
−
 
 
−
== Edge AI SDK&nbsp;/&nbsp;Monitoring ==
 
  
−
[[File:Usage and thermal.png|800x450px|Usage and thermal.png]]
+
== <span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">Edge AI SDK&nbsp;/&nbsp;Monitoring</span></span></span> ==
  
−
[[Category:Editor]]
+
<span style="font-size:larger;"><span style="font-size:larger;"><span style="font-size:larger;">[[File:Thermal.png|800x450px|thermal]]</span></span></span>

Latest revision as of 04:11, 14 December 2023

Hailo AI Suite

Hailo AI Software Suite offers breakthrough AI accelerators and Vision processors uniquely designed to accelerate embedded deep learning applications on edge devices.

Hailo devices are accompanied by a comprehensive AI SDK that enables the compilation of deep learning models and the implementation of AI applications in production environments. The model build environment seamlessly integrates with common ML frameworks to allow smooth and easy integration in existing development ecosystems. The runtime environment enables integration and deployment in host processors, such as x86 and ARM based products, when utilizing Hailo-8, and in Hailo-15 vision processor.

 

 

Applications

TAPPAS is a solution designed to streamline the development and deployment of edge applications demanding high AI performance. This reference application software package empowers users to expedite their time-to-market by minimizing the development workload. TAPPAS encompasses a user-friendly set of fully operational application examples based on GStreamer, featuring pipeline elements and pre-trained AI tasks. These examples leverage advanced Deep Neural Networks, highlighting Hailo's AI processors' top-notch throughput and power efficiency. Furthermore, TAPPAS serves as a demonstration of Hailo's system integration capabilities, showcasing specific use cases on predefined software and hardware platforms. Utilizing TAPPAS simplifies integration with Hailo's runtime software stack and offers a starting point for users to fine-tune their applications. By demonstrating Hailo's system integration scenarios on both predefined software and hardware platforms, it can be used for evaluations, reference code, and demos. This approach effectively accelerates time to market, streamlines integration with Hailo's runtime software stack, and provides customers with a foundation to fine-tune their applications.

Refer to github-TAPPAS

 

 

Edge AI SDK / Application

Quick Start / Application / Video or WebCam / VPU

ObjectDetection

 

Application Model
Object Detection yolov5m_wo_spp_60p.hef
Person Detection yolov5s_personface_reid.hef / repvgg_a0_person_reid_2048.hef
Face Detection arcface_mobilefacenet_v1.hef / scrfd_10g.hef
Pose Estimation centerpose_regnetx_1.6gf_fpn.hef

Benchmark

In order to measure FPS, power and latency of the Hailo Model Zoo networks you can use the HailoRT command line interface. For more information please refer to the HailoRT documentation in link.

 

 


Hailo-8 Benchmark

hailortcli benchmark <Hailo's Model .hef > 

hailortcli benchmark resnet_v1_18.hef

benchmark

 

 

Edge AI SDK / Benchmark

Evaluate the Hailo-8 performance with Edge AI SDK.

Benchmark

 


HailoRT CLI

HailoRT CLI - a command line application used to control the Hailo device, run inferences, collect statistics and device events, etc. Use "--help" to exhibit more usages. 

hailortcli --help

Refer to github-hailort

 

 


Hailo-8 Utilization

hailortcli monitor

Hailo VPU Usage

 

 

Hailo-8 Temperature

python ts_monitoring_2023Sep07.py

  Thermal in hailortcli.png

Edge AI SDK / Monitoring

thermal

Retrieved from "https://ess-wiki.advantech.com.tw/wiki/index.php?title=Edge_AI_SDK/AI_Framework/Hailo&oldid=32722"