What autonomous cars perceive of their environment and how they interpret the information they receive are important topics in the further development of these vehicles. Computer scientists have now taken an important step towards improved perception of complex urban environments.
The answer of robotics researcher Prof. Dr. Abhinav Valada from Freiburg's Albert Ludwig University and his team from the Robot Learning Lab to the task of improving the environment perception of autonomous vehicles and thus contributing to more safety is amodal panoptic segmentation with the help of AI approaches.
Until now, autonomous vehicles have been able to perceive their surroundings by means of panoptic segmentation. This means that so far they can only predict which pixels of an image belong to which visible regions of an object such as a person or a car, as the Freiburg researchers explain. In contrast, the vehicles are not able to predict the entire shape of objects as well, especially if they are partially obscured by other objects.
Humans, on the other hand, can imagine the complete physical structure of objects, even if they are partially obscured. This ability is still lacking in the current algorithms that enable robots and self-driving vehicles to perceive their surroundings.
This is expected to change with perception using amodal äpanoptic segmentation, which enables a holistic understanding of the environment - similar to humans. According to the Freiburg researchers, amodal in this case means abstracting from partial occlusion of objects - instead of seeing them as fragments, they should be seen in their entirety. This makes a new quality of visual environment detection possible, which means an enormous advance for the road safety of autonomously driving cars.
New AI algorithms for this task could enable robots - and autonomous vehicles - to mimic the visual experience that humans have by perceiving the complete physical structure of objects, explains Prof. Valada. This would be an elementary prerequisite for the further development of autonomous driving, as the sensors of the cars of the future must be able to clearly recognise and also classify their surroundings in road traffic: This is another car, that is a cyclist, that is a pedestrian, etc....
Different sensors will be used for this purpose, as it is not possible to rely on one type of sensor for safety reasons. In addition, different sensors are able to compensate for any deficits of the other sensors. In this way, a lot of data comes together, which is evaluated with the help of artificial intelligence. In this way, the vehicles are trained to identify other road users and surrounding objects ever better.
Artificial intelligence will probably make traffic safer, Prof. Alfred Höß, head of the research project KI-ASIC (AI processor architectures for radar modules in autonomous vehicles) at the Ostbayerische Technische Hochschule (OTH) Amberg-Weiden, is certain. In an interview with Bayerischer Rundfunk, he referred to the Ethics Committee, which had already dealt with the topic. The tenor, according to Prof. Höß, was: with systems that function reliably, more accidents are avoided than are produced on the other side by faulty behaviour of the systems.
The KI-ASIC project serves to research a new type of processor architecture, so-called neuromorphic processors, which should make it possible to use AI methods specifically for pattern recognition and analysis in autonomous driving. With these processors, a significantly improved processing of radar data could already take place in the sensor.
Accordingly, KI-ASIC is intended to transfer neuromorphic electronics from basic academic research to automotive applications in order to offer solutions for the central challenges of autonomous driving. In addition to the OTH Amberg-Weiden, Infineon Technologies as the joint coordinator, BMW, the Technical University of Munich and the Technical University of Dresden are involved in the project.
The combination of artificial intelligence (AI) methods with innovative electronic components thus creates the technological basis for autonomous vehicles to react appropriately to all driving situations.