When the Machine Learns to Understand Human Beings

Who doesn't know the Billy bookcase? More than 20 million units of the shelving system have been sold in Germany. Almost as many people have used the corresponding assembly instructions - because anyone buying furniture from the well-know Swedish retailer has to assemble it themselves. However, this can occasionally lead to confusion, as the drawings on paper are not always self explanatory. Many small and medium-sized enterprises (SME's) face similar challenges and often rely on manual production for economic reasons. An intelligent solution that can be implemented quickly and pragmatically - and is affordable - can help: a digital assembly assistant offered by the startup Kimoknow. Using artificial intelligence, the assistant recognizes the object being processed within the user's field of vision and, for example, alerts the user if something has been assembled incorrectly or indicates which steps need to be carried out next. The application was developed by three students at the Center of Artificial Intelligence Talents (CAIT) at KIT's Institute of Information Management in Engineering (IMI) and offers a wide range of potential applications, particulalry in industry. 

"Our system is designed to support skilled workers during assembly and to simplify and automate tasks such as documentation and quality assurance. This allows people to focus more closely on the actual assembly process. Until now,it was not economically viable to teach a program to recognize individual components in manufacturing. Training a system requires several hundred images of an assembly component, which is very time-consuming and only worthwhile for a very limited number of applications, "says Lukas Kriete, a mechanical engineering student and one of the founders. However, by combining 3D models of components with artifical intelligence, the training of object recognition systems can be automated. Together with his fellow students Aaron Boll and Roman Wiegand, Lukas Kriete founded Kimoknow. The name combines the terms AI, assembly, and knowledge. All three founders hold bachelor's degrees in mechancial engineering and completed a large part of their studies at the IMI. 

A demonstrator of the system can be found at specialized assembly workstations provided by ELABO GmbH at the IMI. Such workstations are normally used by assembly workers to manually manufacture individual products, such as complex electric motors or circuit boards. ELABO GmbH has made its specialized workstations available to CAIT for student experiments. "The assembly worker at the workstation wears AR glasses consisting of a camera and a display. At that moment, the assistant understands what the worker is seeing. If, for example, they look at a lamp, the program recognizes that it is a lamp, which can be operated using gestures or simple voice commands. This creates a direct connection between the person and the device. Based on the input from the camera, the system knows what the person is interacting with and observes what they are looking at. Our software therefore enables the context-dependent integration of digital assistants into everyday work processes - something that was previously dificutl to achieve, "explains Aaron Boll. 

Until now, voice control has been one of the few ways of communicating with a machine. However, this approach offers limited possibilities and often functions as a one-way channel. The founders of Kimoknow have now created a system in which information can flow in both directions. As a result, machines and people can communicate with each other in real time. One particularly important advantage of the system is that the Kimoknow algorithms use data that already exists. "Almost every manufactured object has a 3D CAD model containing all the relevant information about the object. We use this data to train the AI, which learns much like a human being does. A person learns in a particular environment and can then apply that knowledge in a different setting, "explains Aaron Boll. 

Kimoknow is supported by Professor Jivka Ovtcharova, founders of CAIT and head ot the IMI. The institute equips students from a range of disciplines, including mechanical engineering, industrial engineering, computer science, and electrical engineering, with skills in digital technologies. "New values and business models for and through AI are becoming a prerequisite for the competitiveness of individuals, companies, and locations alike. AI also affects the way we interact with one another, both in our professional lives and in everyday life. CAIT has the vision of rapidly and pragmatically transforming AI solutions into practical skills and competencies for everyday and professional applications, "says the professor, who is a memer of KIT's Department of Mechanical Engineering. 

Ovtcharova workes in the automotive industry for many years before taking up her professorship and establising new areas of focus at KIT. At the IMI, studnets are not only taught how to acquire knowledge but also how to develop practical skills. "We develop software and prototypes here - everything that is visible and tangible. This is why the IMI is a magnet for industry. Companies are only partially interested in theory; they are interested in research solutions that can be adapted quickly, "says Ovtcharova. 

Industry representatives can see demonstrations of the practical work carried out at thei IMI in the CAIT  laboratories. Together with the IMI, company represenatives develop realistic scenarios and practical solutions for a wide range of requirement in the "Industry 4.0 Collaboration Lab", a test an qualification laboratory. The success of this approach is demonstrated by several spin-offs that have emerged from the IMI. The successfull KIT spin-off Inreal Technologies, which specialzies in virtual reality appplications, is just one example. 

For the founders of Kimoknow, the decision to establish a spin-off was an easy one. Through the IMI and Professor Ovtcharova, they quickly established contacts with industry and gained insights into where their ideas could be applied. "We encourage and support students in founding startups. If you have a good idea, you have to get started right away. Learning and applying knowledge go hand in hand here. Digitalization allows you to teach yourself many things and provides excellent opportunities to acquire knowledge independently. This can accelerate the learning process. Here in Germany, too much time is still spent in the "embryonic phase." Preparing students for professional practice unfortunately still takes too long. With our projects and startups at the IMI, we are determined to change that, " explains Jivka Ovtcharova. 

Contact: jivka ovtcharova ∂does-not-exist.kit edu and michael.grethler@kit.edu