For many skilled workers with a sight impairment, daily tasks can be full of obstacles: Reading a device label, deciphering a handwritten note, or finding their way in unknown spaces is time-consuming or not possible without help. Conventional aids often hit their limits beyond the computer workstation. This is where the AssistiveXR4Work project comes into play: “We’re developing a system that automatically recognizes or magnifies visual information, displays it with high contrast or reads it out loud, depending on the individual vision requirements,” said Professor Rainer Stiefelhagen from KIT’s Center for Digital Accessibility and Assistive Technologies (ACCESS@KIT). “For example, it can automatically recognize and magnify the contents of a whiteboard in a meeting or convert handwritten notes to plain text and further process it without human intervention.” In engineering working environments, on the other hand, the system can help with orientation and highlight relevant components.
The system uses advanced XR glasses with a high-resolution camera image. They capture the surroundings continuously and digitally adapt the display to the individual vision requirements by enhancing contrast, magnifying contents, recognizing texts, and reading them out loud on demand. In addition, an integrated AI system helps with orientation or recognizes objects present in the surroundings. If needed, a human helper can be connected via a live stream.
How Inclusion Helps Securing Skilled Workforce
The project not only helps visually impaired people participate in the labor market, but also addresses the shortage of skilled workforce. Actually, many highly skilled employees lose their jobs or have to shift their field of activity when their vision deteriorates. The goal behind AssistiveXR4Work is to help them pursue their career even then. According to the Institute for Employment Research (IAB), a research institution operated by the Federal Employment Agency, severe disabilities are widespread in Germany, the majority of which are occurring during the person’s professional life. The probability to still be part of the labor force five years after the disability strikes is reduced by roughly 16 percentage points for these persons.
It is planned to construct an operational demonstrator within the four years to come and then test it in practice. The Federal Ministry of Labour and Social Affairs is funding the project with around EUR 1.5 million.