It’s safe to say that reality technology has caused quite a stir in recent years. Whether you’re unwinding with a video game, shopping for clothes, or taking on some hands-on training, the likes of virtual reality and augmented reality are already beginning to make their collective presence felt.
Reality adoption is rapidly accelerating and alongside it, funding for augmented reality and mixed reality enterprises is increasing exponentially.
(Image: Jeff Bullas)
In fact, investment in AR and MR companies alone have eclipsed that of virtual reality endeavours today – a stark contrast to just five years ago where both AR and MR was largely underappreciated by businesses.
The rise of augmented reality in both the gaming industry and retail, in particular, has led to the technology becoming something of a household name among businesses and consumers alike, but mixed reality hasn’t enjoyed the same levels of recognition across the world. So what is it? And why is it so exciting for investors?
Well, mixed reality essentially takes the best of virtual and augmented reality technology and actively blends both the physical and digital worlds together, allowing users to both interpret and interrogate digital information with ease. They can physically navigate and interact with various swathes of data by simply gazing, gesturing or speaking.
The resulting experience offered by MR is entirely immersive and offers a particularly suitable platform for training purposes as well as conceptualising, planning or managing various operational functions.
Understanding The Role of Mixed Reality
Mixed reality draws on many of the key facets of augmented reality. AR draws on a ‘hybrid’ environment that was a huge success behind the rise of Pokemon Go in the summer of 2016. In hybrid environments, interactive virtual objects can be mapped to physical environments that can seamlessly blend real-life and virtual reality.
Whilst it may seem that both AR and MR are similar in delivering both real life and virtual environments, there are significant differences to be found in the technology that drives both realities. Mixed reality, for instance, is largely headset-based, while AR is generally viewed through a screen, like a smartphone or a tablet display. MR technology also works with a deeper understanding of the geometry of the environment in which it operates – meaning that it’s capable of acting as a canvas for users to build immersive content on – all the while remaining defined by the space in which it’s being used.
The release of Magic Leap has led to a significant increase in MR interest, but the tech has existed long before this. For example, Microsoft’s 2016 Hololens release was capable of championing MR solutions. Today, Microsoft is more dedicated to solving enterprise needs, while Magic Leap is acting more as an entertainment tool.
However, in the near future, talk surrounding mixed reality hardware will fall away as the technology continues to gather momentum. Once this occurs, the emphasis will rightfully be on the quality of the content itself.
Underlining The Differences Between MR and VR
Virtual reality offers an experience that’s completely digitally rendered. Here, a user can enter a virtual world by putting on a VR headset that includes artificial audio and video. As the user moves their head, they are able to look around a 3D space. VR headsets can also come with gloves or handheld sticks that may be used to interact with virtual environments.
Unlike with VR, mixed reality starts with a real-world, physical space and places virtual objects within it. Users can interact with these objects without wearing special gloves or using dedicated objects to facilitate interactivity. To help underline this, virtual reality headsets turn totally black when turned off, whereas MR headsets turn transparent – like a pair of glasses.
Underlining The Differences Between MR and AR
Unlike with VR, mixed reality behaves a lot closer to augmented reality, as we’ve already touched upon. AR and MR both rely on real-world environments, and AR ‘augments’ reality by overlaying virtual objects on top of it – such as Snapchat filters for instance. However, users can’t interact directly with overlaid objects. For instance, augmented reality has the power to display virtual 3D boxes onto physical tables, whereas MR has the power to enable users to pick up and open said box.
Augmented reality technology is perfectly capable of being operated via the use of a camera and a screen, but the more immersive nature of MR means that users must interact directly with the 3D environment via the use of a head-mounted display.
There are plenty of applications for mixed reality already entering industries, and the technology has already been implemented into police training methods, remote guidance for construction workers and virtual testing for engineers. While it’s still very much an emerging technology, MR can also be combined with the ever-expanding world of robotics – potentially paving the way for surgeons to perform surgeries remotely with finer and steadier tools.
Locating The Mixed Reality Spectrum
Mixed reality blends physical and digital worlds together in a way that other existing reality technologies are unable to replicate. This helps to create links between human, computer and environmental interactions. The reality is based heavily on advancements in computer vision, graphical processing power, display technology, and input systems.
The term ‘mixed reality’ itself was only introduced into our lexicon in 1994 via a paper by Paul Milgram and Fumio Kishino entitled ‘A Taxonomy of Mixed Reality Visual Displays’. The paper looked into the concept of the virtuality continuum and the categorization of taxonomy applied to displays. In the decades that followed, the application of mixed reality has expanded exponentially beyond the confines of displays and today includes factors like environmental input, spacial sound and both locations and positioning in both real and virtual spaces.
(Image: Microsoft)
In their technical webpage that defined the structure of mixed reality, Microsoft explains that “as MR actively blends both physical and digital worlds, the two realities define the polar ends of the virtuality continuum.”
This combines the array of realities in the mixed reality spectrum. As the chart above shows, on the left, we have the physical reality that humans exist in, and on the right, we have the corresponding digital world.
Mixed Perceptions
Over the past few decades, an exploration into the relationship between human and computer input has gathered momentum, paving the way for a discipline known as ‘human-computer interaction’ or HCI. Human input happens through various separate means, such as keyboards, mice, touchpads, ink and Kinect skeletal tracking.
Technological advancements in these sensors and processing are paving the way for a new concentration in computer input from environments. The levels of communication between computers and environments effectively manifest itself as environmental understanding or perception – and this is why the API names in Windows that show environmental information are known as perception APIs.
Environmental input can capture things like a person’s location in the world (head tracking), surfaces and boundaries (spatial mapping), ambient lighting, environmental sound, object recognition and location.
(Image: Microsoft)
As the graphic above illustrates, the combination of computer processing, human input and environmental input paves the way for effective mixed reality experiences. Movement through the physical world can help to create movement in the digital world. Whereas boundaries in the physical world may directly influence the experience of users inside digital applications in terms of gameplay. Without any environmental input, it’s impossible for physical and digital realities to blend together to create mixed reality.
How is Mixed Reality Technology Being Used?
As mixed reality is still in its formative stages, businesses are still waiting to discover the full potential of MR. However, use cases are emerging all the time across a range of industries and the early results will be encouraging for enterprises:
Mixed Reality in Manufacturing
MR is evolving to become significantly useful in architectural scenarios within manufacturing, such as the positioning of machinery in a factory. Users are able to hold smartphones or tablet screens towards an area of a factory and see digital renderings of equipment and move it around a space.
Speaking to ZDNet, Tuong Nguyen, a senior principal analyst for immersive technologies at Gartner said: “Imagine me moving a digital object on my screen and saying ‘Oh, is this going to fit here? I could put another piece of machinery next to it. Because the system understands the world, it will say ‘that’s not going to fit there.’ The other object will bump into the wall or another barrier and show me that it doesn’t fit.”
The technology also helps when it comes to technical training and instruction. For instance, workers can use a headset to receive interactive instructions from a superior on how to operate machinery within factory environments, all while communicating remotely with supervisors and undergoing training programs in preparation for undertaking business operations.
Mixed reality offers the opportunity to shorten repair cycle times and offer more training solutions for businesses looking to develop the skills of employees without the presence of staff or complex machinery. This can help to reskill and upskill workers faster and offer them troubleshooting scenarios with relative ease.
Mixed Reality in Healthcare
In the rich and diverse field of healthcare, mixed reality is actively helping students to practice surgeries while gaining better visuals of operating room data during procedures as a means of keeping surgeons focused on their task, without looking away from the screen.
MR has been identified as a key tool in delivering enterprise solutions – especially when it comes to the increasingly popular field of remote collaboration. For example, Microsoft recently unveiled their Holoportation development – allowing individuals with mixed reality devices to see and interact remotely with coworkers as fully-rendered 3D holograms.
Despite there being clear evidence of MR being utilised across an array of industries, it’s worth remembering that the technology is still very much in its infancy. As mixed reality continues to develop, we’ll see new applications across industries – and even some left-field use cases along the way.
The Arrival of Transformative Technology
The arrival of smartphones has been a game-changer for our daily lives – and it’s all down to mobile handsets providing users with considerably more convenient access to something that was already available – the internet.
Mixed reality is a similarly game-changing development, but rather than adding convenience to a technology that already exists simply by making access pocket-sized, but by completely changing the way we interact with the real-world environments around us.
The chance to shake off the shackles from our physical boundaries could ultimately have huge ramifications for how we see the world.
For instance, consider the impact that MR will have on travel. If businesses can create digitally rendered face-to-face meetings and conferences through mixed reality devices, there’s no need for businesses to spend money on travel and employees on commuting. This could ultimately save time and money, and help to ensure that workers are more alert without the need to continually be in transit.
For example, an MR company Mesmerise, have created a virtual reality conference option for the digital Morningstar Investment Conference. Attendees will be immersed in a VR experience, taking part in live sessions and sponsor booths.
Likewise with education. If students have access to a perpetual artificial intelligence assistant that’s capable of understanding our goals and providing visual help via the use of immersive digital cues.
Digital overlays can help us to gain and retain knowledge in real-time, helping users to learn new skills differently when compared to our current, more analogue education system. In turn, mixed reality can help to democratize intelligence.
Enterprising Applications
In 2018, a survey conducted by Toshiba found that as much as 82% of enterprises predict that mixed reality solutions alongside AR smart glasses will be used in their business in the next three years. MR offers a significant opportunity for business decision-makers when it comes to solving problems. Notably, being capable of digitally visualizing what your company is trying to do is a very effective asset to have.
For instance, take the case of French automotive manufacturer, Renault. In setting up a collaboration with their technology partner, Immersion, the company adopted the use of a HoloLens at its facilities to help improve quality control processes within its engine assembly processes.
Bertrand Felix, an engineer at Renault Trucks who is leading the project, said: “In practice, quality control operators will wear Microsoft HoloLens smartglasses in which all the digitalized engine parts will be integrated. Via the glasses and Mixed Reality interface, operators will see decision-making instructions that will guide them through the most complex control operations. At the moment, operators working on control points are still using paper instructions.”
An Increasing Marketplace
While HoloLens has become a name that’s synonymous with mixed reality, it’s worth noting that the market for MR eyewear is steadily expanding. In 2014, Google Glass was released and immediately struggled to make an impact on consumer markets because of privacy concerns. However, in an industrial setting, the technology has been much more of a revelation – leading to the popular release of Google Glass Enterprise Edition. Google’s hands-free device for hands-on workers has been on-hand to help an array of industries from automobile assembly to doctors, to teachers.
With digital annotations, immersive training videos and thorough quality assurance checks easily capable of being taken out, Google’s Glass models can pave the way for seamless collaboration among employees, who can see fellow users via a live stream and collaborate in real-time as well as offer troubleshooting.
It’s worth remembering that MR headsets are still in their formative stages in terms of development, with plenty of exciting devices patented for releases in the coming years – most notably with Apple looking to wade in with a sensor-based head-mounted display. Alongside the aforementioned Google Glass models, here’s a look at some of the other major players in the mixed reality headset marketplace:
Microsoft HoloLens
Microsoft’s HoloLens 2 headset features a Qualcomm Snapdragon 850 compute engine alongside a custom AI holographic coprocessor. The HoloLens 2 is predominantly a headset that’s aimed more at business-based adoption, rather than reaching consumers as a priority.
The headset has a flip-up visor, a huge vertical field of view, and articulated hand-eye tracking, and is available for around $3,500 at the time of writing.
Magic Leap 1
Entering the market back in 2018, Magic Leap may not be the most advanced headset available to buy today, but it still acts as a powerful spatial computing system in the form of connected goggles with a Lightpack, or attached processing unit that can be controlled using a handheld touchpad.
It may be a little bit older than the HoloLens 2, but it still packs a punch for a slightly cheaper price at around $2,295. The smart glasses are even available with prescription lenses fitted.
RealWear HMT-1
The RealWear HMT-1 won the award for Best Headworn Device at the 10th annual Augmented World Expo, and it’s easy to see why with how the headset excels for enterprise workers. The HMT-1 is capable of summoning heavy data such as user manuals and usage data for a range of engineering and warehouse-based tasks.
Despite being an award-winning headset, the RealWear HMT-1 isn’t packed with superfluous thrills, and due to its serious and streamlined nature, it’s available to buy for a relatively cheap $1,500 for its class.
Oculus Quest
You know new technology has a bright future if Facebook is desperate to get involved, and with the social giant’s Oculus Quest headset, it’s clear that they’ve created a headset that may be ideal for enterprise training – as shown by Walmart. The technology is particularly useful at a business level due to the arrival of Oculus for Business software, a suite that combines device setup and management tools, enterprise-level service, and greater levels of user experience that’s customizable based on the business use case.
Another positive aspect of the Oculus Quest comes from its pricing. The non-tethered nature of the headset means that the device is comfortably the cheapest listed, weighing in at just $399 as a starting price.
The Bright Future of Mixed Reality
Deloitte understands that over 150 companies across a range of industries – including 52 of the Fortune 500 – are testing or have deployed solutions within reality technology. With over 10% of the Fortune 500 looking at cutting edge technologies to improve their businesses, it’s perhaps unsurprising that another Forrester Research report has found that almost 14.4 million employees will be wearing smart glasses by 2025 in the US.
(Image: Tractica)
As the chart above illustrates, the distribution of smart glasses will transcend just enterprise usage. Mixed reality has the potential to greatly enhance our daily lives and change how we see and interact with the world. By comprehensively blending both physical and digital realities, it’s possible to build new levels of engagement and immersiveness in all of our processes. In terms of entertainment alone, the potential for MR technology is virtually limitless.
William Griggs, CEO of VirtualRealityRental.co summarized the potential of mixed reality while speaking to ZDNet. Describing the role that the burgeoning technology could play, Griggs said: “Mixed reality is the latest advancement in the virtual and augmented reality world, and you can expect it to have a lasting impact on the way we do business and solve enterprise problems.”
“We’ve already seen our real and artificial worlds come together to create popular and revenue-boosting phenomena like social media filters and popular mobile device games — but this is just the beginning for what’s possible with mixed reality,” Griggs explained.
While it’s clear that mixed reality will make big impacts in the world of entertainment and gaming – leading to more watershed moments for video games like with augmented reality’s hugely popular Pokemon Go release. However, there can be tangible excitement for how the arrival of MR will impact the world of business and enterprises.
In a world that’s being forced to evolve even quicker due to the constraints caused by COVID-19, many business leaders may be bemoaning the fact that mixed reality devices are still in their formative stages.
As more employees are forced to work from home, the opportunity to carry out immersive meetings with prospective clients, and train apprentices in a more hands-on way despite global lockdowns have shown that MR really can play an active role in our transition from traditional business models to something resembling the ‘new normal’.
Fundamentally, as mixed reality continues to develop, and more devices are released onto the market, businesses are likely to begin adopting the technology more widely. As it’s become clear that there’s a need for MR to help the development of industries in the future, we may see mixed reality solutions begin to evolve faster as adoption rates rise. However, it’s fair to say that the initial uptake may be slow as decision-makers wait to see the full potential of reality technology.
While MR may be more of a technology for the future, the arrival of MR entertainment may happen sooner rather than later. With Pokemon Go showing consumers and businesses alike the potential for reality technology gaming, the future of mixed reality may be arriving, in some form, sooner than we might think.
