02. Oktober 2019, 09:00 Uhr | Tobias Schlichtmeier
The stereo camera and the embedded system of the drone.
To develop image processing software for embedded systems, the developer must laboriously translate program code from one language to another. Mostly from C++ to FPGA code. To speed up the process, an EU consortium has now created a development platform.
Image processing components are increasingly being used in development, for example for drones, driver assistance systems or mobile X-ray devices. Embedded systems based on FPGA usually serve as the platform for the components. VHDL is usually used as the programming language for the FPGAs. However, image processing applications are usually available in C or C++, which is why the developer has to laboriously translate the program code. In addition, the hardware has to be adapted, which is not always possible. Software developers sometimes start from zero with image processing applications.
In order to avoid the situation, a consortium of six countries has created a development platform as part of the »Tulipp Project«. The platform consists of design guidelines, a configurable hardware platform, a real-time operating system and a programming tool chain.
In addition, a starter kit helps to develop such applications much faster and easier. Ports from C++ programs to FPGA, on which a developer has often worked for several months, can be done with the starter kit within a few weeks.
In concrete terms this means for the developer: Starting from the software programmed in C++, he considers which code components are to be distributed to which hardware components and which program steps can be optimized. The design guidelines developed help in this. Afterwards the starter kit is used. It contains the configurable hardware to which the required sensors and output devices can be connected, the multiprocessor-capable real-time operating system and the programming tool chain. The included applications allow the C++ program to be optimized so that it can be ported to the FPGA as quickly and easily as possible.
The Fraunhofer Institute for Optronics, Systems Engineering and Image Evaluation IOSB in Karlsruhe is one of the project partners in the development of the Tulipp platform. In order to test the tools during code transformation, three concrete application cases were used. One research team dealt with the procedure for obstacle detection of a drone with stereo cameras, another with pedestrian detection in autonomous vehicles, and a third with live image enhancement in X-rays.
The results were satisfactory for the research team. It was proven that the development kit not only accelerates the programming of the code, but also that the image processing algorithm runs much faster on the embedded system than on a C++ basis. The procedures, which originally took several seconds on a high-performance PC to analyze a single image, run on the drone in real time. Thus, about 30 images per second are analyzed. Pedestrian detection has also been increased by a factor of one hundred in terms of speed: instead of one image per seven seconds, the system produces 14 images per second. Radiation during X-ray examinations was reduced to a quarter due to upstream filters and better image evaluation. At the same time, the energy consumption of all three applications was reduced many times over.
This enables developers to use the development platform to achieve faster programming, faster algorithms and reduced energy consumption.
The embedded system, which evaluates the real-time images and recognizes obstacles, sits in the white box of the drone.