We’ve spoken to the top minds in augmented reality throughout AR Week on Pocket-lint and, when we asked them what was holding this exciting technology back from the big time, we noticed a certain pattern to their answers.
So, if you’ve always wondered why you keep seeing the same kinds of AR apps and why there’s still such a buzz about it, then read on for the short version of just what the hold up is all about.
You might have noticed when you’re trying to use GPS on your mobile phone that there seems to be quite a margin of error. Sadly, that’s a serious issue for mobile AR while out and about, seeing as the information that your smartphone app is trying to generate on your display is dependent on knowing exactly where you are in the big, wide world. The standard GPS system is accurate only to within 9m and that’s just not good enough.
One way around the issue is by installing surveyor-grade GPS into all phones which is accurate instead to within a matter of centimetres and infinitely more doable for a workable mobile solution. Unfortunately, that costs in excess of £10,000. So, we can forget about that method for the next 10 years or so at least. The other solution is not to rely on what we think is supposed to be in front of us, but actually recognising it instead.
In other words, instead of your phone consulting maps, the camera could recognise objects and scenes, which it can check against a database, to know exactly what it is you’ve got it pointed at. That’s the research area of computer vision and this kind of solution requires having a very large database or huge model of the environment to reference - that or some very clever thing that nobody’s thought of yet to allow location like this to work in real time.
Of course, the other thing you can do is keep augmented reality to specific tasks in known, pre-modelled environments which is, in fact, the way it began life. So, whether that’s table top gaming or AR for maintenance and repair, do take a look at the works of Blair MacIntyre and Steve Feiner for a better idea of what you can achieve without having to step outside.
Most of the research into AR is about getting the real and virtual to line up properly. Even if the GPS location was spot on, the application still needs to know exactly which way you’re facing, which angle you’re looking at and what orientation your head is in at the time; and it needs to do all of that in an instant to keep the computer generated overlay in precise sync with the real world. Not an easy task. What makes this harder is that, first, the sensors in your smartphone - the gyros, compass and accelerometer - are not as accurate as they need to be, surprise, surprise - but, what’s more, again, your device has no computer vision of the real world and the camera no real perception of what it sees.
Of course, upgrading the sensors in smartphones is one thing to do and hopefully this will happen as mobile AR devices improve but, naturally, that’s going to come at a cost. Another way around this is what the likes of Qualcomm and Nokia are up to in the AR research centres that they’ve set up all over the world. Instead, they can improve the way that their chips interact with the hardware already available, and that looks to take tracking a certain distance on its own.
More interesting still is the area of object and physical environment recognition to solve and aid the issue of tracking. Computer webcams used in Flash animation desktop AR involve software which can recognise markers to produce the computer generated graphic overlay and, one step further, they can even detect textures too, rather than just a black and white square code. Both of these are very specific and pre-programmed however. For AR on the move to work, it needs to be able to read the situation in real time and we’ll be taking a look at that later in AR Week when we speak to tracking and AR expert Georg Klein. Suffice to say, it’s a very tricky area.
Ignoring the issue of the sensors in your smartphone not being good enough for the moment, there’s still something fundamentally wrong about the model of mobile augmented reality. Right now, it requires the user to pull something expensive out of their pocket and wave it around in front of their face. Your arm gets tired and you look a little silly doing it as well. The camera also isn’t the best for it either. It’s designed to fit people into your pictures and not give you an accurate 1:1 scale representation of the way you see.
We’ve already looked in detail in AR Week on the kinds of devices that will take augmented reality forward but, essentially, it’s at its best with an always on, full frame, lightweight, comfortable and socially acceptable looking solution. The two best choices for these are spectacle-like AR glasses or even AR contact lenses if you really want to head into the future. That said, mobile phones are coming on leaps and bounds - the number crunching and graphics processing is good enough these days - and there’s plenty of development we can do with them. There needs to be a bit more money poured into the head-mounted display research field but, that aside, it’s all moving in the right direction here.
Strangely enough, this is actually the least of the worries of the AR experts that we spoke to, but it is something that needs to get sorted nonetheless. Much of mobile-based augmented reality at the moment involves an app on your phone which calls up to the Internet for information based on the GPS signals of where you are so that it can display some nice graphics on the screen for you. Now, that’s all very well if you’re on Wi-Fi or smack in the middle of a city burning with 3G, but coverage around the entire country isn’t good enough. What’s more, some of the really heavy apps out there might need to pull down information of a slightly larger scale.
Fortunately, there are a lot of companies with a vested interest in expanding the mobile broadband network and doing it fast. The arrival of 4G devices is another push in the back of the networks and, as they begin to roll out these new technologies, a more blanket coverage of 3G comes too.
Of course, the other way around the problem is to have all the information you need for your AR experience already on your device thanks to a largish app download. That’s less doable for out and about tourism but, for augmented reality in smaller, controlled, known environments, that’s all you need. As it happens, the original applications for AR - maintenance and repair, etc - were all about those kinds of set up. So, although a better level of connectivity would be great, it’s only a barrier for the current flush of AR apps - Layar, Wikitude, Junaio and the like.
As augmented reality guru Steve Feiner put it when we spoke to him, the really interesting problem is software. It’s all very well having all the other bits and pieces in place but if we have no idea how to present a usable, non-intrusive, intuitive interface to the consumer, then nobody is going to bother with AR at all.
Fortunately, there’s lots of research groups out there looking at this very problem. Naturally, Feiner and his department at Columbia University have such projects going on which involve the ideas of proximity and filtering. The basic idea is that the stimuli nearest the user are the ones of most importance, so it’s these that should appear the largest or most obvious on an AR UI, with all others either smaller or absent altogether.
Looking at the problem from another angle, Christian Sandor and his team at the Magic Visions Lab at University of South Australia are looking at not only how they can display the augmentations, but also how one could adpat the image of the real environment to make it more useful as well. So, if you want to take a look at buildings melting and X-ray vision, then make sure to read our interview with the them.
Augmented reality is only about 20 years old, so it’s not a huge surprise that there are so many challenges to overcome before it really takes off. These five are probably the main categories of concern but it’s really meant as just a taster of what AR is all about and where we are with it at the moment. If you want to know more then there’s plenty of in-depth interviews and ideas to drill down to into in AR Week on Pocket-lint. Do take a look.
For more information on what Qualcomm is doing with Augmented Reality please click: http://www.qualcomm.co.uk/products/augmented-reality
And for more on AR Week head over to our AR Week homepage on Pocket-lint