Medtech

I Tried Apple’s Vision Pro Headset and It’s Incredibly Impressive

The eye tracking and hand tracking are next level.

Published

11 months ago

June 8, 2023

This article was originally published by TheStreet

Apple made history this week in a unique way—the technology giant didn’t just unveil a new product, but its first entry into an entirely new product category since the Apple Watch in 2014.

It’s not another iconic phone or a functional smartwatch, but a headset that the company calls the “most advanced personal electronics device ever.” Apple’s Vision Pro is a spatial computer that packs a bunch of hardware, including both the company’s M2 Chip and a new R1 chip, 12 cameras, and a multitude of sensors. It’s also running an entirely new platform: Vision OS.

The result is a headset that looks like a high-end pair of ski goggles that’s built from premium materials like aluminum, stainless steel, and glass. It also has an external battery (a silver brick about the size of an iPhone) with a cord that mechanically attaches to the Vision Pro on the rear right side. It provides about two hours of use, but you can also plug it in via USB-C for a longer runtime.

The Vision Pro is a mixed reality headset, so the interface it provides lives on top of the real world. You can watch a movie being projected on a 100-inch screen, work in many windows set up on the expansive canvas that is your space, and even interact with 3D objects.

When does Vision Pro release and how much will it cost? Apple’s Vision Pro will launch in early 2024 with a starting price of $3,499. That price point stretches well beyond other headset like the Meta Quest, but let’s be clear–Apple’s (AAPL) – Get Free Report entry into this category should far outpace the others and deliver an ultra-premium experience that feels like the future.

At the end of WWDC–Apple’s annual Worldwide Developers Conference–on June 5, 2023 I had the chance to spend about 40 minutes with the Vision Pro. The result is a device that is by all accounts remarkable, but also familiar and intuitive. Ahead, I’m highlighting the results of some of my very early hands-on testing.

The Arena Media Brands, LLC and respective content providers to this website may receive compensation for some links to products and services on this website.

Meeting Vision Pro

My experience began with a head scan from an iPhone that is nearly identical to setting up Face ID. I also had it scan my ears which is similar to setting up personalized spatial audio for AirPods Pro, a process which required me turning my head to the left and the right as the iPhone’s camera took measurements and examined the shape. This is because Vision Pro needs to fit everyone’s own unique head, and you want to ensure there is a proper light seal for a tight, comfortable, and immersive experience.

Given that I wear glasses, I had to meet with a vision scientist who scanned them, got the prescription, and then matched me with a pair of Zeiss lenses that magnetically attach to the inside of Vision Pro. Yes, Apple’s headset cannot be worn with glasses, so this could result in additional costs at launch.

Once that was set, I headed into a private room with a few Apple representatives and sat down center on a sofa. In front of me on the table was the talk of the town–the new Vision Pro, which appears as a premium pair of ski goggles to the uninformed.

The front is a black glass with a screen inside that has aluminum edges for cooling and packs a boatload of technology. From standard HD to 3D cameras and a multitude of sensors, it’s really a technical feat.

It attaches around the back of the head through a main mesh strap that is breathable and soft. Apple will sell different styles of bands when this launches. The unit I was using, which is not final production, was a gray cloth material. You can adjust the tightness via a dial on the rear left side. A top strap is optional, though Apple recommends it for help with adjusting the weight.

View the original article to see embedded media.

A Whole New Way of Controlling

Unlike the (SNEJF) PlayStation VR 2, which requires controllers for navigating the interface and using apps–and even the (META) – Get Free Report Meta Quest 2 which supports hand tracking as well as controllers, Apple is entirely relying on elements of your body to be the controllers. It might sound strange, but it’s one of the most impressive features of this technology.

Around your eyes in Vision Pro are infrared lights, which allow for extremely sensitive eye tracking. After you put the Vision Pro on for the first time, you’ll follow around moving dots in a circle twice which let it learn your eyes and basically map them. This is critically important and Apple’s implementation here far outpaces the competition.

Your eyes are the proverbial cursor. Vision Pro presents you with a floating home screen in your space, displayed on top of the background that is still a crystal clear view of your surroundings that you would see even without the headset on, and you just glance at what you want to open–akin to using a mouse to hover over an app or a finger approaching a screen to tap. It’s augmented reality and virtual reality mixed together in one.

So in my case, I had the Vision Pro home screen which is a floating set of three rows that is reminiscent of the Apple Watch app viewer, floating about a coffee table with a person to the left, one to the right and artwork on the wall behind it. Talk about wow.

I then could look at Photos, and see a subtle spotlight, similar to hovering over an app before selecting it on a Mac. To select it, well that’s where the new clicker comes in and, surprise, it’s your hand. I just tapped my index finger with my thumb on my right hand and Photos opened. From there I was greeted with the app window that resembled macOS, iPadOS, or iOS. I could see some spotlighted shots and to enlarge one, I just looked at it and tapped again. It’s much more accurate over the Meta Quest 2’s hand tracking with better accuracy.

To swipe, I held those fingers together and gestured to the left or right. It was that simple and intuitive, fitting the old Apple mantra of “it just works.” How it works is the story of all those patents and countless hours spent in secret labs at Apple’s campus.

The real lightbulb is viewing a panoramic image shot on an iPhone in full screen, as it stretches beyond my direct field of vision. I could turn my head to the left or the right, and even zoom in further. When in full screen it will immerse you into that image with a mode that Apple calls an “environment.”

It may sound disconcerting, but it doesn’t fully disconnect you. Rather, when someone is speaking to me and I look at them, they come through in a sort of haze that turns clear with a high amount of clarity. It’s similar to a camera focusing, a trick that Vision Pro also does as it sharpens whatever you’re focused on at and subtly unfocuses the rest.

Apple’s version of pass through, essentially seeing the space you’re actually in while wearing the headset, is nothing sort of epic. It’s clear, zero latency, and looks akin to the real world. That’s thanks to the large amount of hardware here including the R1 chip, which handles the real time processing of the view from the multitude of cameras.

Of course, the screen you’re viewing is premium and that fits with the astronomically high price of $3,499. Each eye is given the equivalent of a 4K display that looks sharp, lets colors pop, adds depth, and delivers an incredibly high amount of contrast.

Forget the Theater – You’ll Never Watch Movies the Same Way

I got the chance to watch a snippet of Avatar in Vision Pro, and it might be the first time I actually appreciated a 3D film. Parts of the Na’vi people and other creatures could pop out of the window that was on top of the room I was in, or I could shift into a blacked out cinema-like experience to be even more immersed. Still, other folks around me were just a passthrough away.

Apple already has chops in the world of audio and they’re saying they have the most advanced Spatial Audio stack on the Vision Pro. The result are two speakers near your left and right ears that produce the soundtrack. Similar to AirPods with Spatial Audio, the sound matches the elements on the screen. It only added to the immersion here, but it’s not noise canceling.

If I wanted the real world back, without other folks speaking to me, I could turn the Digital Crown—yes the same one as on the Apple Watch or AirPods Max—counterclockwise or clockwise to take away or let more of the passthrough in.

I also had a demo with Dinosaur Encounter that started with a rectangular portal opening across from me and a butterfly flying into the room and near me. I stuck out my finger, and in real time, that digital creature fully rendered, landed on the ground and reacted a bit. Before I knew it, I stood up for the first time in the demo and approached a T-Rex that moved closer as I got closer to it.

Still a First-Gen

Yes, this headset is definitely exciting and novel, but there are a ton of potential use cases for the headset as a whole.

And that’s a big reason as to why Apple is announcing this in June 2023 but not shipping it until early 2024. For a first-generation product, it is nothing short of a modern tech marvel. I had my doubts going in, but the fact that I could control the entire experience with my eyes and hands really was remarkable.

Now, this is only my first impression from a demo that lasted a bit longer than 30 minutes, so I can’t truly say if everyone should run out and grab this. But those 30 minutes felt like the future in a pretty massive way. It still remains to be seen what this product could accomplish exceptionally well or what is the must-have experience. You could use it for productivity with apps opened all around you, watching immersive content, playing an epic game, or even FaceTiming with folks through “Personas.”

At $3,499 it’s definitely expensive, and an absolute luxury in this day and age. But it could be a worthy investment for a number of folks, and our guess is that it will supercharge the AR/VR sector and inspire further innovation. But one thing is clear to me: Apple is ready to run. Now it’s up to the developers to build a killer app or two to really drive this experience home.

The other key thing to remember is that the Vision Pro is a wearable computer and a standalone platform. It will have an App Store at launch and will deeply integrate with other Apple devices and services, but it is not a device that needs to be tethered to another.

Prices are accurate and items in stock at time of publishing.

prescription
device
hardware
devices
apps
augmented reality