What is Mixed Reality? How to use it in the manufacturing industry

About Mixed Reality

MR ( Mixed Reality ) is a technology that projects things in the VR (virtual reality) world into the real world using technologies such as CG. It is truly a world where the virtual and the real come together.

What is the difference between VR and AR?

Concepts related to MR include “VR” and “AR.” How are “VR”, “AR” and “MR” different?

First of all, VR means “virtual reality.” VR refers to the technology that allows an artificial virtual space created from digital data to be seen and felt like the real world through VR goggles or a VR headset. In the field of VR, entertainment headsets such as “oculus Go” and “PlayStation VR” have already been commercialized for the general public. The virtual world that you experience with VR has depth and breadth similar to reality, and you can also experience the experience of moving through VR yourself. However, the world expanded by VR is artificial and completely separated from the real world. Of course, the VR world and the real world do not interfere with each other.

AR is ” augmented reality ,” and it is a technology that projects and extends digital information into the real world. It takes the form of using real space as a foundation and placing digital information on top of it. A famous service using AR technology is the hugely popular smartphone game “Pokemon Go.” In this game, real-life locations are set as Pokéstops, and digital information such as items and monsters is added to them and displayed on the smartphone screen. In addition to smartphones, goggles and HMDs (head-mounted displays) such as Microsoft HoloLens are on sale as devices and viewers used in AR.

MR is a world that takes VR and AR one step further. It is a technology that creates a new world by combining objects that exist in the real world with a virtual world created by digital information through HMDs and other devices.

A major feature of MR is that it allows direct access to the digital virtual world from the real world. Until now, when accessing digital information, we had to “consult” a computer using a keyboard or touch panel, and the answer from the digital side was indirectly communicated with the digital world through the display of a PC or smartphone. In contrast, in a world made possible by MR, instead of looking into the digital world through a keyboard or display, we imagine interacting directly with the images and images created by the digital world by touching it.

MR technology, in which images become tangible in the real world, can be said to be a revolutionary technology, especially in the field of manufacturing such as CAD. For this reason, research on the use of MR in simulations at research and design sites using digital information, as well as in the medical, manufacturing, and architectural fields, is rapidly progressing.

How will the world change in the future with the advancement of MR?

It is expected that MR technology will further expand in the future and be used in a variety of situations. For example, it may be possible to use MR technology to project the image of a person who is far away and have a conversation with them. Instead of being on a monitor, you will be able to communicate with each other in the real world, such as sitting face to face at a table at home.

The “real world” realized by MR can also be used in business training. By projecting the world created with digital data into the real world and seamlessly performing two-way exchanges between the digital and real worlds, we are able to provide training that is more realistic. In particular, in the case of dangerous work, how to train inexperienced workers to become proficient has been a major challenge, but with MR technology, it is possible to repeatedly and safely conduct highly accurate training.

In this way, it is predicted that by 2025, MR will permeate society and gradually change the way people work, learn, and communicate.

Microsoft’s “Mixed Reality”

The MR service provided by Microsoft is “Windows Mixed Reality”. You can experience the MR world with the “Mixed Reality Portal” app, which is standard on Windows 10. In addition, MRT (MixedReality ToolKit), a data collection for developing applications that run on Windows Mixed Reality, has been released. Microsoft Azure also offers Azure Mixed Reality.

Microsoft’s HMD “Microsoft HoloLens” is also a device that realizes this mixed reality. With HoloLens, you can project digital images into real space and see both. Of course, you can not only project images, but also manipulate what appears as images by touching them directly with your hands.

How can mixed reality be used in manufacturing?

An increasing number of companies are also trying to introduce MR technology in the manufacturing industry. A major strength of MR technology is the ability to access data in real time and experience it with real feeling. The first area that comes up as a field where this directly leads to improved operational efficiency is “product simulation.”

Access data in real time

Especially in the manufacturing industry, improving operational efficiency is an important point that directly leads to improved productivity. In order to improve business efficiency, it is necessary to thoroughly eliminate waste and prevent problems from occurring.

To prevent trouble, production lines and other areas have been monitored using display monitors, but the drawback of monitoring two-dimensional images through a display is that it is easy to miss certain details. On the other hand, if we use MR technology to monitor the “reality” by projecting it close to the administrator, we can expect the same effect as if the administrator were monitoring the target directly.

Using MR technology, it will be possible to monitor work being carried out in remote locations in real time “right in front of your eyes,” and it will also become possible to perform double and triple checks with high precision. Masu. If there is no need to assign a manager to each site, it will be possible to centrally manage the work of factories scattered around the country in one place, which will lead to significant streamlining.

Product simulation

In the manufacturing industry, simulations are repeated over and over again until a single product is completed. However, two-dimensional simulations, in which data is input into a computer and analyzed and displayed on a monitor display, inevitably lack some sense of reality.

To compensate for this problem, prototypes are created as part of simulation in the manufacturing industry, but using MR technology, it is also possible to perform sophisticated simulations that are almost the same as creating prototypes. The use of MR is particularly advanced in simulations in the architectural field. Technology has emerged that allows you to project full-size 3D design drawings into a real space and check the appearance from the top, bottom, left, and right. Because it allows observation and verification from any angle in advance, it is attracting attention as a technology that is expected to grow further and be widely used in the future.

Remote business support

At manufacturing sites, situations often arise where problems require us to contact designers and engineers. If you have a technician nearby, such as in the same department or company, you may be able to come directly to the site and see the situation firsthand. However, if the technician is located in a remote location, you will have to contact them by phone, email, or video call. With these methods, you may not be able to convey what you want to convey to the other person, or you may not be able to fully understand what the other person is saying. Especially when it comes to mechanical troubles, many situations may occur that are difficult to explain with words alone.

MR technology also shows its power in situations like this. You can see the engineer right in front of you, have him or her directly check the problem area, and then receive a detailed explanation while looking at the actual item. MR is a hands-free system that does not require a monitor or keyboard, so you can work while receiving guidance. Through the intervention of MR technology, it is possible to create an environment where employees can work together smoothly.

Employee training and skill improvement

Training and upskilling employees is an essential element in improving productivity and operational efficiency. However, in the manufacturing industry, it is necessary to allocate materials to practice, which incurs material costs. Additionally, the risk of injury to inexperienced employees during hazardous work cannot be ignored.

Even in such cases, by using MR technology to mix virtual data with reality, training can be carried out with a sense of reality. It is also possible to perform training and check procedures by displaying a digital image of the machine in front of your eyes. This method is useful not only for training inexperienced employees, but also for training for employees who have acquired some level of technical skill to further hone their skills.

summary

By utilizing MR technology, things that were previously considered impossible are becoming possible in the manufacturing industry. MR technology has the potential to help improve operational efficiency in all fields, including product simulation and improving employee skills. Let’s actively utilize MR, which is currently attracting a lot of attention, and improve productivity.