IMAGING IoT SOLUTIONS

Recently, the term “AI” has become commonplace. AI technologies are expected to evolve rapidly and have a significant impact on people’s lives and businesses. Today, AI-based businesses at Konica Minolta are led by the imaging solutions business. This business is underpinned by imaging-IoT technologies, which help turn the vision into reality by integrating information obtained by imaging input devices with various types of sensor data and by achieving advanced recognition and judgment based on AI processing.

The imaging solutions business, which came into being around 2014, is one of the youngest businesses of Konica Minolta, which has a history of 150 years. And yet even though it is a state-of-the-art business, it also inherits the core technologies of optics and imaging refined since the company’s founding. It is the result of engineers’ ceaseless quest to improve technologies amid turbulent times.

Standard-bearers in the early days of the digital era

The digitization of our products is considered to have started in 1989 when several engineers, who were invited from both in-house and outside the company, started to develop LSI chips *. At that time, companies were just starting to introduce personal computers and the Internet. As digital technologies rapidly evolved, they began to have a significant impact on society.

All of our products, including cameras, copiers, and X-ray equipment, were analog. These products handled huge amounts of imaging information, and so LSI chips were essential for digitization. The engineers went around the company explaining the need for digitization, and impressively, managed to develop LSI chips for products in almost all the company’s businesses in less than seven years.

Regarding cameras, it was generally believed, even among the general public, that the quality of digital cameras would never surpass that of analog cameras. But an in-house color management expert was sure it could be done, and showed the results of their experiments. This changed the situation. Image processing engineers and system engineers joined the development team to develop in-house an image signal processor (ISP), which was indispensable for image processing.

As the digitization of products accelerated, various types of LSI chips were developed in line with the evolution of copiers to MFPs, which were equipped with printer and scanner functions, and the competition with competitors in terms of specifications became even fiercer.

*LSI (large-scale integration) chip: A single chip that integrates many electronic parts and circuits

Attaining imaging-IoT by mobilizing human assets

In the 2000s, the IoT (Internet of Things, a technology that enables devices to exchange information, enable automation and improve efficiency by connecting to the Internet) started to attract public attention. The term “IoT” appeared for the first time in our three-year Medium-term Business Plan from 2014 and we started to work on building the business, aiming to transform from the conventional selling of products to service providers, to offering solutions by using data derived from imaging technologies.

The engineers who were in charge of optics, image processing, and systematization in the digital camera business returned to the spotlight. We had withdrawn from the camera business, which had been our foundation, in 2006, and the engineers had been assigned to other businesses. Many of them had worked on tasks not related to imaging, so they were keen to play an active role in their specialty fields. Soon after they joined the project, they started to develop specific solutions.

Our strengths lie in the development and manufacture of various input devices in addition to image processing technologies refined in the camera business. We were able to gather data using sensors and imaging systems and make improvements to obtain the data we needed. This was only possible because we were a manufacturer.

We sought to detect behavior by using imaging technologies. Deep learning was essential to build the algorithms needed for digitizing and analyzing people’s movements. In the field of image recognition, AI evolved rapidly from around 2010 and started to attract much public attention. We created a deep learning development environment in 2015 when we introduced a U.S.-made supercomputer for AI development. At that time, only two private companies and one research institute had this computer in Japan, and we were the first to get this system up and running. This also helped attract many excellent AI experts.

In the next phase, which involved putting algorithms in devices, engineers who had developed LSI chips for copiers played a key role. Copiers scan and digitize large amounts of images and perform high-speed processing, so these engineers excelled at technologies for accurately running huge programs with limited hardware resources.

We possessed various input devices and technologies for handling images, as well as advanced technologies for building and implementing algorithms by using deep learning. We have integrated these strengths to create imaging-based IoT, namely, our imaging-IoT solutions.

Technologies refined through a difficult mission

Our imaging-IoT solutions originated from a monitoring system for nursing care facilities for elderly persons using human behavior recognition technologies. In this system, a behavior analysis sensor equipped with a camera is installed on the ceiling of a room to capture images of the entire room and analyze the behavior of residents. When behavior such as getting up, getting out of bed, slipping, and falling is recognized, nursing care staff receive a notification and video on their smartphones. This allows the staff to go to the room quickly only when needed, eliminating the needs for excess patrols. This increased efficiency creates extra time and helps enhance the quality of nursing care.

However, it was difficult to identify the posture of people based on a video captured by a wide-angle lens from the ceiling; accurate behavior detection was not easy. Various motions were mistaken for the behavior of residents, such as a moving electric fan and shadows cast when opening or closing curtains. In fact, during the initial phase of development, false positives occurred frequently, with alarms issued even though residents had not slipped. However, since slipping and falling can result in serious injuries, false negatives caused by lowering sensor sensitivity had to be avoided. It was necessary to meet the severe conditions of minimizing both false positives and false negatives.

A breakthrough in solving the problem was made by introducing deep learning. To increase the accuracy of image recognition, the project team conducted trial and error with the facilities’ cooperation and continued to train the AI. The more the AI is trained, the larger the algorithm becomes. Advanced technology to implement the algorithm in small devices demonstrated its potential. “This experience reinforced our imaging-IoT technology,” said the engineers. The accomplishment of this difficult mission clearly demonstrated the potential of imaging-IoT and drove further development.

Many other solutions using imaging-IoT technology were developed. For example, we built a system to identify individuals at commercial facilities based on the overall image of individuals, including their clothing and personal belongings, without facial recognition, analyze the behavior of visitors, and assist optimization of shop layouts and product displays depending on the flow of people. We also created a system for visualizing hydrocarbon gases, such as methane, based on advanced optical and image processing technologies using infrared rays, enabling users to intuitively grasp the location and amount of gas leakage. This visualization system was released in the North American market in 2023.

Providing imaging-IoT solutions as the norm

As the various imaging-IoT-related technologies required for development gradually became available, the next question was how to offer solutions to customers. We concluded that services should be provided via the cloud, instead of door-to-door sales, to make the imaging-IoT business feasible. Delivering applications via the cloud and charging fees were expected to increase the efficiency and enable global deployment. This led to the idea of building a platform as the foundation for sharing information and value and offering services.

The concept came to fruition as FORXAI, an imaging-IoT platform, which we started to offer in April 2020. FORXAI aims to create a community with not only customers but also partner companies and to build an ecosystem to enhance quality based on feedback to solutions offered. This marked the beginning of taking on new challenges to transform from our conventional approach of selling products into a service provider.

To make services quickly available through FORXAI, it was decided to openly offer an IoT platform and our proprietary edge devices and imaging AI technologies. FORXAI partners who have strengths in respective technologies and channels have already gathered and created opportunities to develop entirely new solutions. FORXAI has also enabled our existing businesses to work with partners who have engineering capabilities in such fields as the cloud, AI, and embedding, and to realize various solutions.

Through these initiatives, it has become the norm to offer imaging-IoT technologies to customers even in our existing businesses. For example, in the Healthcare Business, we assist diagnosis by incorporating unique image processing technologies in dynamic X-ray image analysis workstations , thus helping to quantify and evaluate the movements of organs and joints. In the Professional Print Business, image calibration, which was previously performed by experienced experts at printing companies, has been made possible by AI and image recognition technologies.

The target set in 2014 of increasing the number of imaging-IoT experts, including AI engineers and data scientists, to 1,000 by 2023 has been attained. These experts have been assigned to various businesses. It has been a decade since we launched the development of imaging-IoT solutions, and various aspects of our operations have benefited.

For 150 years, we have met customers’ needs to turn their visions into reality and helped people attain their purpose in life. Although imaging-IoT is still relatively new in our history, we will continue to work on solving social issues by developing unique technologies that meet emerging needs for visualization.