Microsoft revealed the HoloLens 2 mixed reality headset at MWC 2019. The company made it clear that the device is not yet intended for consumer use, but will be one day. That doesn't mean the future of Mixed Reality isn't interesting though. 

A new patent has revealed Microsoft's future plans for Mixed Reality include integrating your smartphone into the experience. In this patent application, the company describes a system that would allow users to interact with their smartphones while in the Mixed Reality world. 

The patent makes it clear that Microsoft is acknowledging the drawbacks of controllers for such experiences. It notes that users need to learn how the controllers work, where the buttons are and how using those controllers translates into movement in the virtual space:

"Hand-held controllers can present various drawbacks when used to provide interaction in a virtual reality experience...such controllers are often designed specifically for virtual reality experiences and thus tend to be unfamiliar to users as they are becoming acquainted with virtual reality technology... "

The proposed solution is to allow people to use a device they're already familiar with i.e. their smartphones:

"... the touch-sensitive input device may enable the reception of touch input, which may be an intuitive and familiar input paradigm to a user (e.g., as a result of repeated, daily use of a smart-phone). Touch and/or other types of input may extend the range of inputs that can be supplied beyond translation and rotation enabled by the hand-held controllers described above."

"The input device itself may also provide a familiar paradigm of interaction, as in some examples the input device may be, as referenced above, a smart/phone or other mobile device of the user. Moreover, the virtual reality experience may complement and/or extend input device functionality. As described in further detail below, the virtual reality experience may render a representation of the input device that visually indicates the device and also provides a user interface for interacting with the device and virtual reality experience. In some examples, the user interface may provide additional information and points of interaction beyond a user interface rendered by the input device itself. "

This is potentially interesting in a variety of ways. Firstly, the use of a smartphone within Mixed Reality is clearly intended to enhance the user experience by making it immediately more accessible. Using a smartphone as a controller will allow users to pick up the control scheme much more quickly too. 

Secondly, the patent notes that bringing the smartphone into the mix may also enhance the experience further by complementing and extending how the system works. An additional screen, apps on the phone and another way to engage with the virtual surroundings bring a potential for an even more immersive experience. 

The accompanying images also show the company plans on allowing users to see their phones while wearing the headset too. 

Of course there's no official word on if and when this will happen, but it shows there's plenty of potential for an interesting future for Mixed Reality.