ZYBO HDMI hardware FAST corner detection
Introduction
The aim of this project is to perform the FAST corner vision algorithm on the ZYBO board with GNU Linux, OpenCV version 2.4.5 and the HLS video library to take advantage of the hardware acceleration. The video is grabbed from the HDMI input and the results are displayed after computation on the VGA port of the board.
Short description
Originally coming from Digilient's "HDMI in" SDSoC platform, some modification were made in order to use GNU Linux, OpenCV and to have enough ressources in the programmable logic. You can download the prebuilt projet and the SDSoC platform in the dowload section.
HDMI input - VGA output
The "capture and display" architecture stores and reads the stream into the frame buffers. When running, the executable let you choose to display either the video input (first frame buffer) or the second one that can be filled by the OpenCV function or the HW accelerator. The third one is not used at the moment.
Below is a simplified view of the Vivado base project (without accelerator) for capturing and displaying the video.
The input pixels are acquired from the HDMI with the IP "DVI in to RGB" then formated in a AXI4-Stream Video protocol adding the Start Of Frame/End Of Line informations. Finally, the Video DMA takes care of writing/reading the stream in the frame buffer which is stored in the RAM (DDR3). You can notice that there is no input "Video Timings Controller" which was used to get the resolution and the timings according to the video input, it's a choice I made to get more ressources in the programmable logic, so now the resolution is simply hardcoded, you have to choose whether if you'd like 1280x720 @ 60FPS or 1920x1080 @ 60FPS in the C code of the project.
Each pixel is coded on 16 bits allowing not to exceed the maximum memory bandwidth (1050 Mbps) of the ZYBO even with the further addition of the HW accelerator.
Accessing physical space from userspace
Configuring the IPs registers can be quite easily made with the mmap() system call. A problem appeared when I needed to access the frame buffer from both userspace and Video DMA. To solve it I wrote a small kernel module ("myVDMA") which allocates the required amount of memory for the 3 frame buffers and uses sysfs to get their physical addresses from the user space. The sources are located in the platform's src folder.
To be continued (soon)...
Download
You can either download the platform that comes with this project as example or download the built system for SD card.
Warning : you may need an external power supply!
Conclusion
This platform is a starting point to play along with both the OpenCV and HLS video library (or your own code) on the ZYBO. There are still many improvements to do such as detecting when a new frame is acquired and manage the usage of the frame buffers to have a correct frame synchronization.