To experience AUGMENTED REALITY, you look through a device screen or put on a headset and a virtual image is laid over the room you're in. You can see what's around you, but part of it is blocked out by whatever video projection is playing on your headset.
The Basic Setup
▶ A camera and screen equipped with computer vision, a technology that identifies objects and surfaces. Adding depth and motion sensors lets a device map the room around you and track your motion through it. Your app can then overlay anything from a first-person-shooter zombie attack to the steps to replace a fan belt.
▶ For now it's pretty simple: catching Pokémon (Pokémon GO), mapping constellations (Sky Map), inking a tattoo (InkHunter), turning you into a half-dog (Snapchat).
AR can't scan a room and identify every object. But you can teach its computer vision to identify individual objects, like a motorcycle, when prompted, says Mike Campbell, executive vice president of the ThingWorx AR platform. "There's not enough computing power to analyze everything it sees."
▶ Hands-on skill training, interior design, wearable computing.
AR can lead a factory worker on a tutorial, but right now the technology won't change your life unless you own a factory, says Amber Case, a fellow at Harvard's Berkman Klein Center for Internet & Society. A Microsoft HoloLens can overlay hidden parts such as a tucked-away air filter and demonstrate its removal. Similar programs are in development for phones and tablets and could soon offer life-changing relief for tasks like Ikea furniture assembly.
▶ Motion sickness sets in when your perceived motion—what you see—doesn't match what your inner ear feels. That's not the case with augmented reality, says Robert Scoble, coauthor of The Fourth Transformation: How Augmented Reality & Artificial Intelligence Will Change Everything. You're still looking out on the real world and the same horizon.
▶ AR on mobile devices really is mobile. Unlike high-end VR, which can't leave a room, AR can enhance a city tour or museum. Last winter, the Detroit Institute of Arts lent visitors Android phones to view the skeleton inside a 2,000-year-old sarcophagus and to see the original colors on a now-biege Assyrian sculpture.
However, AR is difficult to wear on your face. Everybody thinks we'll be walking around with the next Google Glass but social constraints prevent that, says Case, adding, "Sunshine makes headset AR difficult to see, voice and hand controls are still unreliable."
How Apple Will Own It
▶ AR will explode in the next year. Today, relatively few devices offer a rich AR experience, leading to a lack of demand for new AR apps—phones with Google's Tango AR number less than a million. Expect that to change after Apple's June release of iOS 11 ARKit for developers, says Scoble. ARKit is a bundled suite of AR tools that can reach a quarter billion Apple devices. Additionally, this fall's new iPhone adds 3D sensors and room mapping that can play hologram-like counter-op games or virtually measure and then furnish a room without draining battery.
To experience VIRTUAL REALITY, you put on a headset and your entire environment is replaced by whatever the headset is projecting. You can't see the room you're in. Suddenly, whether you look up or down or side to side, you're on a roller coaster, or at the beach, or at a Paul McCartney concert.
The Basic Setup
▶ A headset with screens that, for each eye display, offsets right and left images that trick your brain into seeing depth—same as a 3D movie. Motion sensors allow your view to change as you look around. Additional hand controls with motion sensors let you grab, hold, or throw objects in the VR world. And tethered sets (attached to a PC and power source) can use wall-mounted sensors to map and track your movements around a room.
▶ Relatively simple games and programs: crashing cubicles (Job Simulator), painting in 3D (Tilt Brush), dodging bullets (Superhot VR).
Interactive VR is not like our reality. It's safely on the Minecraft side of the uncanny valley, with video games on the most powerful devices often looking like a crisp version of N64 graphics because of the graphic processing power required for your tracking and movements.
▶ Medical treatment, automotive prototyping, astronaut training. At the University of Washington Harborview Burn Center, burn victims use VR headsets during procedures so painful, morphine isn't effective. Researchers there believe VR reduces pain to manageable levels by immersing the brain in another world and leaving fewer mental resources to process pain. Newer studies have found positive results in wound care, chronic pain, and the dentist's chair.
▶ VR on a phone-based headset is largely immobile.Tethered VR, which can move around a small room, has the power of graphics cards that alone cost as much as a phone. Lacking that power, mobile VR is limited to processing the turning of your head.
▶ VR can make you feel sick. But the faster a headset refreshes its images, the less likely you are to feel motion sickness. More powerful tethered units, like the HTC Vive and the Oculus Rift, run at least 90 frames per second. Mobile VR, with less processing power, tops out at 60 fps and usually operates at 30.
How Apple Will Own It
▶ VR, like AR, is waiting for an affordable device that offers an immersive experience. There's no hero device on the horizon, says Scoble, but it'll probably be a phone—albeit significantly more powerful than today's options. "Tethered devices require $2,000 gaming PCs. Not everyone needs those, but they do need phones." Again like AR, he adds, our best bet is the next iPhone release.
This story appears in the October 2017 issue.