Augmented reality AI supports assembly

With the help of augmented reality glasses, the digital assistant understands what the wearer sees.

Artificial intelligence (AI) makes it possible for machines to recognize objects. This requires large amounts of high-quality image data with which the algorithms are trained manually. The start-up company Kimoknow, established at the Karlsruhe Institute of Technology (KIT), has developed a technology to automate this training. The first Use Case: A digital assembly assistant for contactless cooperation between humans and machines.

The start-up uses image data that is created for all objects in computer-aided development processes (CAD) and production data management (PDM). Among other things, they provide information about the material, geometry and position of the respective object. The object recognition system trained in this way can be used in many different ways, including in augmented reality glasses. They detect relevant objects in the user's field of vision in real time and also provide the necessary contextual information about the object in question. The virtual assistant guides users through the assembly process, visualizes the assembly instructions step by step and thus shows in which order which part is processed with which tools and assembly materials. It repeats individual steps when errors occur and documents the process. The fitter has both hands free and communicates with the system via eye contact, hand signals or voice commands.