February 22, 2024

How the Vision Pro could become Apple’s best gaming device

After trying out the Vision Pro in one of Apple’s 30-minute in-store demos, what struck me about the high-end headset wasn’t the weight, the external battery, or the passthrough cameras, but how beautiful the screens were, and how great they seemed to be for playing games.

Since announcing the Vision Pro without a super clear sense of direction, Apple has focused on watching video, whether it’s spatial video, or movies and TV shows, as one of the best things you can do with a headset on. It’s all over the company’s “Hello Vision Pro” marketing campaign, and it was a clear highlight Other way around‘s review of the device. And as great as watching movies and TV can be in the Vision Pro, playing games can work just as well.

Apple makes the obligatory shoutout to Apple Arcade on the Vision Pro, but I’m talking about the more full-featured games we play on consoles and other VR headsets. Deep, immersive experiences that take more time and aren’t necessarily built around Apple’s highly lucrative in-app purchasing system. The kind of titles, like the remake of Resident Evil 4which are only now coming to iPads and iPhones.

We know the Vision Pro is powerful enough to run these types of games; we know there is a deep range of virtual reality and mixed reality experiences that can be transferred; we know that Apple is taking a more active stance when it comes to gaming. With the right tweaks, the Vision Pro could become a great gaming console in a way that Apple’s other devices haven’t been.

M2 is still powerful

The M2 and R1 work together to make the Vision Pro’s passthrough magic trick work.

Apple

One example where following Apple’s branding of the Vision Pro as a spatial computer makes sense is in the device’s processing power. The headset features both Apple’s M2 chip and a new R1 chip that manages the Vision Pro’s extensive camera system and other sensors. These are the same chips that are in the current iPad Pro, the MacBook Air and even desktop computers like the Mac mini. It’s powerful enough to run games like Death Stranding: Director’s Cutit should be able to do the same when mounted on your face.

The catch, and it will be visible if you can spend time using the Vision Pro, is the high-resolution, high-refresh rate micro-OLED screens. Apple uses foveated rendering to give the M2 chip some wiggle room while overlaying digital objects over a sharp (if somewhat dull) video feed of the world around you. That means the headset detects what you’re looking at and only displays what you’re looking at in full, leaving the edges blurred and out of focus to save power. This is a completely normal technique that headsets like Sony’s PSVR 2 use, but it suggests that the M2 and R1 combination probably wouldn’t be able to do all the computer things it’s supposed to do and make the screen look good.

That poses a potential problem for ports of graphically intensive AAA video games. Some of them may not work as well as on a gaming PC, a dedicated console or even a Mac with the same chip. Not every game will be able to run on the first generation Vision Pro.

VR libraries are enormous

If even a fraction of the Quest 3s library could make it to Vision Pro, that would be a win.

Photo by Raymond Wong

The great thing is that there are plenty of VR and MR games that would work just as well and probably require a lot less of the Vision Pro. Meta has been making a consumer VR push for years, since the release of the Quest 2 in 2020, if not earlier. There are hundreds of games on the store that can be ported, some are funded, others are multiplatform. There’s also SteamVR and its extensive catalog of PC VR games that could make the jump.

This all depends on whether the Vision Pro catches on, and more importantly, whether games can be ported. Most VR games rely on dedicated motion controllers and wouldn’t have an easy transition to hand and eye tracking. Apple currently offers support for third-party gamepads such as the PlayStation DualSense or the Xbox Wireless Controller, but they are actually intended for 2D games.

The best-case scenario is that Apple adds support for good VR controllers, perhaps by blessing a few existing ones or releasing APIs and frameworks that would make it easier for an accessory maker to build one that works with the camera system Vision Pro. It’s not out of the question that a motion controller will be added to Apple’s MFi program. However, until that happens, solutions will be developed to reverse engineer support for existing platforms and accessories.

Apple’s attitude towards gaming is changing

Apple’s M-series chips are powerful enough to run modern games.

Photo by Raymond Wong

In addition to saying no to developers, Apple appears to be taking more of a backseat approach to gaming on the App Store, raking in the 30 percent fees and otherwise finding ways to keep people from avoiding them. Recently, that seems to be changing. The company made gaming a big part of the selling point for the M3 chip in the new MacBook Pros and the A17 Pro in the iPhone 15 Pro. Apple even suggested this when asked. “Gaming was a fundamental part of Apple’s silicon design,” Doug Brooks, a member of the Mac product marketing team, told Other way around late last year.

The company has also opted to relax restrictions around game streaming apps, allowing a service like Nvidia GeForce Now or Xbox Cloud Gaming to exist on the App Store and sell subscriptions, provided they follow age rating rules from Apple follow. That wasn’t a decision Apple made without threats of further investigation from the European Commission, but it’s still an important change that will hopefully pay off for devices like the Vision Pro. We know the Quest 3 is a competent replacement for an Xbox and a TV with the right internet connection. The displays on Apple’s headset could make that experience even better.

These aren’t the kind of tweaks that happen overnight, and one of the biggest issues, the Vision Pro’s $3,500 price tag, probably won’t go away for a few revisions. There’s certainly time for the Vision Pro to become a better gaming console, and perhaps by then it will be affordable and comfortable to wear too.

Leave a Reply

Your email address will not be published. Required fields are marked *