2020 Getting Started wih Computer Vision for Vili Embedded Systems: Par 3 Using OpenCV and File Transfers Besllebox GETTING STARTED WITH COMPUTER VISION FOR VITIS EMBEDDED SYSTEMS: PART 3 USING OPENCV AND FILE TRANSFERS (@ AI(HTTPS://BEETLEBOX ORG/TAG/AI/), ARTIFICIAL INTELLIGENCE (HTTPS:// BEETLEBOX ORG/TAG/ARTIFICIAL- INTELLIGENCE /), COMPUTERVISION (HTTPS://BEETLEBOX ORG/TAG/COMPUTERVISION/), EMBEDDED (HTTPS://BEETLEBOX.ORG/TAG/EMBEDDED/), EMBEDDEDVISION (HTTPS:// BEETLEBOX ORG/TAG/EMBEDDEDVISION/), FPGA (HTTPS:// BEETLEBOX ORG/TAG/FPGA/), OPENCV (HTTPS.//BEETLEBOX ORG/TAG/OPENCV/), VITIS (HTTPS:// BEETLEBOX ORG/TAG/VITIS/), XILINX (HTTPS.//BEETLEBOX.ORG/TAG/XILINX/) 2) 28 FEBRUARY 2020 [= POST A COMMENT (HTTPS://BEETLEBOX.ORG/GETTING-STARTED-WITH-COMPUTER-VISION-FOR-VITIS- EMBEDDED-SYSTEMS-PART-3-USING-OPENCV-AND-FILE-TRANSFERS/#RESPOND) Welcome to part 3 of the tutorial. Over at Beetlebox we are excited for the release of Vitis, which is the unification of all of Xilinx's previous software into a single package, We have been working hard on ‘computer vision using this platform and thought that we could provide some help to others wanting to get started on Xilinx’s development boards. This tutorial will be a multi-part series covering the basics of getting started with computer vision and Vitis and will be covering: + Getting XRT and Linux up and running (https://beetlebox.org/getting-started-with-computer-vi for-vitis-embedded-systems/) hitps fbeetebox.orgigeting-starteditr-computer-ision-fr-vils-embedded:-systems-part3-using-opency-and-e-ransfers! ans2or0038 Geting Started wih Computer Vision for Vis Embedded Systems: Pat 3 Using OpenCV and File Transers ~ Bectlebox + How to build "Hello World” and understanding Vitis Development Flow (hitps://beetlebox.org/getting-started-with-computer-vision-for-vilis-embedded-systems-parl-2/) + Using OpenCV on the embedded system (Current Page) + Using Vitis Vision Library + Accelerating Computer Vision using XRT and Kernels We hope these tutorials will be useful for anyone Looking to get into computer vision on FPGAS. Part 3: Using OpenCV on the Embedded System OpencCv OpenCV is one of the most popular Computer Vision libraries in the world and forms the backbone of many projects, including on FPGAs. This tutorial we will be focused on running OpenCV on our ARM core. giving us a solid foundation for accelerating our computer vision in later tutorials. OpenCVv comes pre-installed on our Petalinux system, but we still need to do a bit of fiddling with our compiler to be able to use it, so in this tutorial we will be setting it up and running through a simple example of making an image greyscale, Pre-installed version of OpenCV For 2019.2, the pre-installed verison of OpenCV is 3.4.3 comes with following enabled: + pythong + eigen + ipeg + png + tif + val + libval + streamer + samples + tbb + gphotoz It even comes with the experimental versions of OpenCV. Most notably missing from this installation version is ffmpeg which may be needed to read certain video files. We won't worry about this yet. however, and instead just focus on images. hitps foeetebox.orgigeting-startedith-computer-ision-for-vils-embedded:-eystems-part--using-opency-and-fle-ransfers! ansTransfering files in Software Emulation and in Hardware on Xilinx FPGAs Using computer vision requires us to test on images and videos, hence knowing how to transfer files properly is critical. This tutorial will also cover transferring images on and off our Zynq, Instructions Pre-requisites: + Follow the previous tutorial guide to have a basic working system (https://beetlebox.org/getting- started-with-computer-vision-for-vitis-embedded-systems/) Our particular setup: + Operating System: Ubuntu 18.04.02 + Development board tested: ZCU104 Getting the OpencCvV libraries in place + Begin from the system project we created last tutorial; + File->New->Application Project + Inthe ‘Create a New Application Project’ window: + Project Name: opency + Click ‘Next’ + Inthe ‘Platform’ window: + Click the platform that we created in the previous tutorial + Click ‘Next’ + Inthe ‘Domain’ window: + Click ‘Next! + Inthe Templates’ window: + Click ‘Vector Addition + Click ‘Finish’ + Inthe Explorer window: + Under ‘opency’ right click the ‘src’ folder + Delete ‘krnl_vadd.epp' and ‘vadd.h’ + Goto ‘opencv.prj' and remove ‘krnl_vadd! from the Hardware Functions + Rename ‘vadd.cpp’ to 'main.cpp’ + In’main.cpp' replace the code with the following hitps foeetebox.orgigeting-startedith-computer-ision-for-vils-embedded:-eystems-part--using-opency-and-fle-ransfers! ans2o2orai8 Getting Started wih Computer Vision for Vis Embedded Systems: Par 3 Using OpenCV and File Transfers — Besllebox HA simple greyscale colour convert in OpencV include #include #include include include #include Hinclude include #include using namespace std; using namespace cv; int main() « Mat src_images Mat grey_image; src_image=imread("test_image. jpg"); if (Isre_image.data) { cout << "Could not open image” << endl; return 0; evtColor(src_image, grey_image, CV_8GR2GRAY) ; imwrite(“grey.Jpe", grey_image); cout << “Created grey image” << endl; return 0; + Make sure to begin with that we are in software emulation mode + When we save the file, we will notice a lot of red under our includes, so we need to configure our setting to solve that + By default the OpenCV that Vitis links is the one that is contained in Vivadb. This is problematic for us because it isn't the exact same installation as we have generated with Petalinux, meaning we will get version clashing + The way we can fix this is by going into the installation directory of Vivado, which be default is, /tools/Xilinx/Vivado/2019.2/include + Remove the ‘opencv’ and ‘opencv2’ folders (We can just cut the folders up a level in case we ever need to restore it + We now need to tell the Vitis where the correct folders are In the Explorer window: hitps:foeotobox orgigeting-startd-with-computer-visior-fr-vils-ombedded:-systems-part-3.using-opency-and ans2or0038 Geting Stared wih Computer Vision for Vis Embedded Systems: Pat 3 Using OpenCV and File Transers ~ Bectlebox + Right click: ‘openev'-> 'C/C+* Build Settings’ + The window ‘Properties for opencv’ should appear + Under 'GCC Host Compiler’ click ‘includes’ + Under the ‘Include paths’ window, add a new line: /xsa_build/zcu1e4_base/src/aarch64-xilinx-linux/usr/include + Under the ‘GCC Host Linker’ click ‘Libraries’ + Under ‘Libraries (-U' add the following libraries: + opency_core + opency_calib3d + opencv_imgcodecs + opency_imgproc + opency_highgui + opency_featuresad + opency_xfeaturesad + Under ‘Library search path (-L), add a new line: /xsa_puild/zcu1e4_base/src/aarch64-xilinx-Linux/usr/1ib Emulating our program in software + To test our system we first need an image, called “test_image,pg” + We need to upload this ir lage to our QEMU so we can run openey on an actual image + Copy and paste ‘test imagejpg' into the ‘src’ folder In the ‘Explorer’ window, click ‘Emulation-SW’ "_vimage'->'emulation’->'sd_card.manifest’ In ‘sd_card.manifest’ add the following line: hitps fbeetebox.orgigeting-starteditr-computer-ision-fr-vils-embedded:-systems-part3-using-opency-and-e-ransfers! ens2o2orai8 Getting Started wih Computer Vision for Vis Embedded Systems: Par 3 Using OpenCV and File Transfers— Besllebox /opencv/src/test_image. jpg + Run the simulation, by right clicking ‘opencv' in the ‘Explorer’ window and clicking ‘Run As'->'2 Launch on Emulator (Single Application Debug)’ + Wait for the program to finish at which point the console should display: Created grey image + Inthe ‘Emulation Console’ window type Is + ‘greyjpg’ should be listed in the results. We now need to extract that data by setting our link between our host and VM + Click Xilinx’->'XCST Console’ + This should open a XCST Console window which we can use to transfer files. Open the link to the VM by typing in the console: connect -url tep:localhost:144 Is + Ourhost ctory should be listed we can now transfer files using two commands: tfile copy -from-host /path/host /path/vm tfile copy -to-host /path/vm /path/host + Let's transfer our grey image off our QEMU: tfile copy -to-host /mnt/grey.jpg /grey. Jpg + Looking at our image directory we should have a grey image hitps fbeetebox.orgigeting-starteditr-computer-ision-fr-vils-embedded:-systems-part3-using-opency-and-e-ransfers! ans2o2orai8 Getting Started wih Computer Vision for Vili Embedded Systems: Par 3 Using OpenCV and File Transfers Besllebox + Asa final note, it would have been possible for us to transfer the image on to the system using XCST and then execute the program, but transfering the file on boot saves a little bit of time Running our program on the Zyngq itself + Since there is no kernels to emulate, we will skip the hardware emulation part and run our program straight on hardware + Swap configurations by going to the ‘opencvppj' file and changing ‘Active Build Configuration’ to Hardware + The C/C+ build settings are not saved between different build configurations, so we need to repeat the process of getting the OpenCv libraries in place + Once all the libraries are back in place, clean the project by right clicking on ‘opencv' and clicking ‘Clean Project’ + Build the project + The project build will be quick here because there are no kernels meaning Vitis does not need to perform any synthesis. The issue is that Vitis will only output ‘opencv.exe' in our ‘sd_card’ folder. Luckily, we can just re-use the SD card image from our previous tutorial and transfer our exe file on + Boot and connect the Zynq platform as in previous tutorials + To transfer our files we will be using scp, + Connect an ethernet cable from the Zynq board to the host + We need to configure our Zynq's address by running the following commands on the Zynq: ifconfig etha inet 192.168.1.10 ifconfig ethé netmask 255.255.255.0 + On the host, the connection needs to be configured using the Internet Protocol Version 4 (TCP/IPva) and setting the properties tor IP address: 192.168.1.11 Subnet mask: 255.255.255.0 + How to do this will vary depending on the operating system, but once it is set. we should be able to. ping our board from the host: ping 192.168.1.1@ + The board should respond, Now we can copy over the files using the following commands on the host: scp /test_image. jpg [email protected]: /mnt scp /opencv/Hardware/sd_card/opencv.exe root @192.168.1.10: /nnt + When prompted for the password, use ‘root’ + Onthe Zynq in ‘/mnt’ , we can run ‘ls'to see that ‘test_image.jpg’ and ‘opencvexe’ have been placed on the board, run the executable -/opency.exe hitps fbeetebox.orgigeting-starteditr-computer-ision-fr-vils-embedded:-systems-part3-using-opency-and-e-ransfers! ms2o2orai8 Getting Started wih Computer Vision for Vis Embedded Systems: Par 3 Using OpenCV and File Transfers Besllebox + Using 'ls' again the new image ‘grey,pg' should have appeared. We can transfer that data out with the scp command on the host: scp [email protected]: imnt/grey.jpg + Opening up jpeg we get our Nower: Now we have a basic framework in place for getting image data into our FPGA, processing it and then getting it back out. Using this we now have the basis for accelerating our computer vision. Next time we will look into what Vitis Vision Libraries and how we can use this great resource for accelerating ‘Computer Vision. Previous: Building a “Hello World” (https://beetlebox.org/getting-started-with-computer-vision-for-vitis~ embedded-systems-part-2/) Next: Vitis Vision Library If you have enjoyed this tutorial but are in current need of talent to build advanced Computer Vision systems on FPGAs, consider joining our ClickCV Early Access programme (https://beetlebox.org/beetlebox-home/clickev-early-access/). ClickCV Early Access provides bespoke service and support for developing advanced Computer Vision systems on FPGAs. We use our own proprietary Computer Vision Library, ClickCV, (https://beetlebox.org/clickcv/) to provide our clients the cutting edge in low latency, high definition processing. Contact us today to find out how we could build your next-generation system (https://beetlebox.org/beetlebox-home/clickcv-early-access/). ‘About the Author: Andrew Swirski is the founder and managing director of Beetlebox, a Computer Vision Acceleration specialist. The company develops the Computer Vision Acceleration library called ClickCV. which is designed to fully utilise the hardware adaptability and performance of FPGA chips, Beetlebox is currently running an Early Access programme (https://beetlebox.org/beetlebox-home/clickcv-early- hitps fbeetebox.orgigeting-starteditr-computer-ision-fr-vils-embedded:-systems-part3-using-opency-and-e-ransfers! ans2or0038 Geting Stared wih Computer Vision for Vis Embedded Systems: Pat 3 Using OpenCV and File Transfers ~ Becllabox access/), where the company provides bespoke service and support to develop client's Computer Vision systems on FPGAs, Before Beetlebox, Andrew Swirski previously worked at Intel (formerly Altera) in FPGA encoding and decoding, He completed a Masters’ in Electrical and Electronic Engineering from Imperial College London in 2017, (htt p// pint eres t.co m/p ine reat e/b utto n/? url= htt ps:// bee tleb ox.0 ral/g etti ng- star ted- wit h co mp hitps fbeetebox.orgigeting-starteditr-computer-ision-fr-vils-embedded:-systems-part3-using-opency-and-e-ransfers! ons2o2orai8 Getting Started wih Computer Vision for Vili Embedded Systems: Par 3 Using OpenCV and File Transfers Besllebox uter visi G+ on- v (htt for- (htt. p:// vitis ps:// plu - twit sgo em ter. ogl bed co eco ded m/s ms - har har syst e e? em url= url= s+ htt. htt part ps:// psi -3- bee bee usi tleb tleb ng- Ox.0 0xX.0 ope rg/g tg/g nev etti etti - ng- ng- and star f star - ted- (htt ted- file- wit p:// wit tran h- ww he. sfer co wfa co s/& mp ceb mp me uter ook. uter dia - co - m/s visi psilf tons2o2orai8 on- for- vitis em bed ded syst em ope nev and file- tran sfer s/t ext har erp hp? u=h ttps fo eetl ebo xor g/g etti ng- star ted- wit h- co mp uter visi on- for- vitis em bed ded Getting Started wih Computer Vision for Vis Embedded Systems: Par 3 Using OpenCV and File Transfers Besllebox on- for- vitis em bed ded syst em ope nev and file- tran sfer 3/8 itle =Ge ttin bee tleb ox.0 ro/ wp- con tent Jupl oad 8/20 20/ 02/ Get ting St arte dh ead ph oto 3 pg& des crip tion =Ge ttin ns2o2araie owi th% 20¢ om put er% 20v isio n%2 Ofor %20 Viti $%2 OE edd ed %20 Syst em s%3 A% 20P art %20 3%2 ous ing %20 Op hitps foeetebox.orgigeting-starteditr-computer-ision-fr-vils-embedded:-systems-part3-using-opency-and-le-ransfers! ng- ope nev and file- tran sfer s/&i ma ges =htt. ps:// bee tleb ox.0 ro wp- con tent fupl oad 8/20 20/ 02/ Gating Started wih Computer Vision for Vis Embedded Systems: Par 3 Using OpenCV and File Transfers— Besllebox owi th% 20¢ om put er% 20v isio n%2 Ofor %20 Viti s%2 OE edd ed %20 Syst em s%3 A% 20P art %20 3%2 ous ing %20 Op owi th% 20 om put er% 20v isio n%2 Ofor %20 Viti s%2 OE mb edd ed %20 Syst em s%3 A% 20P art %20 3%2 ous ing %20 Op rans2o2orai8 Getting Startad wih Computer Vision for Vis Embedded Systems: Par 3 Using OpenCV and File Transfers Besllebox enc Get enc enc V% ting V% V% 20a _St 20a 20a nd arte nd nd %20 d_h %20 %20 File ead File File %20 _ph %20 %20 Tra oto Tra Tra nsf _3j nsf nsf ers) pg) ers) ers) Leave a Reply Your email address will not be published, Required fields are marked * YOUR NAME * YOUR E-MAIL* Your WEasITE Your Comment Save my name, email, and website in this browser for the next time I comment. SEND COMMENT ers! rans2o2orai8 Getting Started wih Computer Vision for Vili Embedded Systems: Par 3 Using OpenCV and File Transfers — Besllebox This site uses Akismet to reduce spam. Learn how your comment data is processed (https://akismet.com/privacy/). SEARCH, Q RECENT POSTS Getting Started with Computer Vision for Vitis Embedded Systems: Part 3 Using OpenCV and File Transfers (https://beetlebox.org/getting-started-with-computer-vision-for-vitis-embedded- systems-part-3-using-opencv-and-file-transfers/) Getting started with Computer Vision for Vitis embedded systems: Part 2 (https://beetlebox.org/getting-started-with-computer-vision-for-vitis-embedded-systems-part- 2) Getting started with Computer Vision for Vitis embedded systems: Part 1 (https://bectlebox.org/getting-started-with-computer-vision-for-vitis-embedded-systems/) Welcoming Dr Bouganis (https://beetlebox.org welcoming-dr-bouganis/) Beetlebox founder Andrew Swirski will be part of a Innovation Academy panel (https://beetlebox.org/beetlebox-founder-andrew-swirski-will-be-part-of-a-innovation-academy- panel/) RECENT COMMENTS SahaDrf on Getting started with Computer Vision for Vitis embedded systems: Part 1 (https://beetlebox.org/getting-started-with-computer-vision-for-vitis-embedded- systems/#comment-189) Bartokon on Getting started with Computer Vision for Vitis embedded systems: Part 2 (https://bectlebox.org/getting-started-with-computer-vision-for-vitis-embedded-systems-part- 2/#comment-185) Andres Manjarres on Getting started with Computer Vision for Vitis embedded systems: Part 1 (https://beetlebox.org/getting-started-with-computer-vision-for-vitis-embedded- systems/#comment-182) ion for Vitis jon-for-vitis- Bobb Piatek (http:/Avww/ishcamp.com) on Getting started with Computer embedded systems: Part 2 (https://beetlebox.org/getting-started-with-computer: embedded-systems-part-2/##comment-180) hitps fbeetebox.orgigeting-starteditr-computer-ision-fr-vils-embedded:-systems-part3-using-opency-and-e-ransfers! sans2o2orai8 Getting Started wih Computer Vision for Vis Embedded Systems: Par 3 Using OpenCV and File Transfers Besllebox ARCHIVES "> February 2020 (https://beetlebox.org/2020/02/) > January 2020 (https://beetlebox.org/2020/01/) > December 2019 (https://beetlebox.org/2019/12/) "> October 2018 (https://beetlebox.org/2018/10/) > May 2018 (https://beetlebox.org/2018/05/) CATEGORIES > Non classé (https://beetlebox.org/category/non-classe/) META Log in (https://beetlebox.org/wp-login.php) Entries RSS. (Really, Simple, Syndicatign) (https://beetlebox.org/feed/) ‘Comments RSS (Really Simple Syndication) (https://beetlebox.org/comments/feed/) WordPress.org (https://wordpress.org/) hitps fbeetebox.orgigeting-starteditr-computer-ision-fr-vils-embedded:-systems-part3-using-opency-and-e-ransfers! sss