Dive into visionOS Development (New OS by Apple)

Kickstart your journey in the realm of spatial computing by delving into 46 educational sessions. Gain insights into visionOS, the art of designing for spatial computing, and the usage of developer tools such as Xcode, Reality Composer Pro, and various other utilities from Apple.

Grasp visionOS Concepts

SwiftUI is your most effective tool for creating a user interface that adapts seamlessly across all platforms. Its declarative syntax, coupled with visual design tools, enables rapid code development and testing. Even though UIKit and other technologies are viable options for app development, SwiftUI serves as a crucial technology for generating spatial content for visionOS.

Implement Scenes in Your Interface.

Scenes offer a vital method to structure your app’s UI. They’re available in both SwiftUI and UIKit for your iPadOS and iOS apps. In visionOS, scenes are mandatory to display your interface.

Construct Adaptable Interface Layouts.

Building apps with adaptable layouts makes transitioning your code to new platforms, like visionOS, effortless. Throughout the development phase, ensure you test your layout on a variety of devices and device orientations. Ensure that layout changes are animated effectively and correctly.

Adopt Standard Gestures and Input.

Built-in gesture recognizers simplify cross-platform support, with visionOS mapping inputs to these recognizers. Limit the use of custom input and gesture recognizers, as these may necessitate design alterations in visionOS.

Enable Diverse Interactions with Your Content.

Support command menus to allow task performance via a connected keyboard. Enhance your app’s accessibility to allow interaction via specialized devices. These alternate interactions offer ways for users to interact with your app when touch or mouse events are unavailable.

Assess Hardware and Feature Availability Before Use.

Several frameworks provide checks to verify if the current device supports specific features. For instance, Core Location enables the determination of available location services on a device. To avoid potential errors when transitioning your app to visionOS, always ensure features are supported on the current device prior to use.

Ensure Your App is Inclusive.

Accessible and inclusive apps have the potential to transform lives, and visionOS offers the tools to attain this objective. Enable your app to cater to a global audience by internationalizing and localizing it. Review your app’s accessibility support to simplify navigation and information access for all users. Utilize inclusive language and terms to cultivate an environment of welcome.

Steer Clear of Device-Specific Code and Decisions.

Device-dependent decisions may hinder the proper operation of your app in visionOS. Instead of deciding based on the device idiom, opt for situational requirements and offer logical default behavior. For instance, arrange views using the system-provided size, not the expected device size.

Upgrade Code Calling Deprecated APIs.

Apple occasionally deprecates frameworks and APIs to accommodate modern replacements. Transition to these replacements as early as possible to ensure seamless operation of your code in visionOS.

Delve into RealityKit and 3D Content.

Apps designed for visionOS can leverage depth and an infinite canvas to create moments of immersive experience. Explore how you can utilize RealityKit and other frameworks to get ready for app building in visionOS.

Ponder when to incorporate 3D elements into your interface. Windows in visionOS can be augmented with 3D shapes and meshes, or those items can be displayed in the user’s environment. A shopping app could present a 3D model of an item alongside the product description in its main window, and allow users to drag the item out of the window to visualize it in their living room. Consider how to integrate depth into your app’s windows and when to add volumes or immersive scenes.

Discover RealityKit.

As Apple’s 3D rendering engine for visionOS, RealityKit is now deeply integrated with SwiftUI to assist in building sharp, responsive, and volumetric interfaces. 3D content can be bound to SwiftUI views and states, creating a seamless blend of 2D and 3D visual elements. If your project already includes 3D content, consider using RealityKit to render it.

Explore MaterialX shaders.

MaterialX is a widely supported open standard used by leading companies in the film, visual effects, entertainment, and gaming industries. Learn how to create surface and geometry shaders with MaterialX in Reality Composer Pro.

Familiarize Yourself with USDZ.

Begin learning the tools and techniques needed to generate USDZ assets for your 3D content. Numerous 3D design programs now offer USDZ import and export. Starting later this month, you can use Reality Composer Pro to build USDZ assets for your projects.

Prepare Next-Level Audio and Video.

Begin contemplating modifications required for your app’s audio and video content to support visionOS:

Prime your app’s soundtrack.

Spatial Audio in your existing apps adds a new dimension to your content. This works even better in visionOS, which defaults to Spatial Audio for playback. Use PHASE and AVFAudio frameworks to develop your audio content and position it within your scenes. Start adding 3D support to video assets. Get ready for the introduction of stereoscopic viewing in visionOS by capturing video in 3D and preparing your assets for playback. QuickTime File Format extensions let you include multiple video tracks for stereoscopic playback, but revert to monoscopic playback when needed. Use AVFoundation to adapt playback automatically to the current device. Consider alternatives to haptic feedback. If your current app provides feedback using haptics, be prepared to substitute those haptics with sounds in visionOS. Begin building a library of audio replacements or contemplate alternate types of feedback. Enhance SharePlay Experiences. Sharing and collaboration are central to visionOS. Similar to macOS, any app window can be shared with others during a FaceTime call. With SharePlay, the next generation of shared experiences becomes feasible.

Prepare your apps for SharePlay in visionOS by identifying activities that can be shared now, and implement them using the Group Activities framework. Consider what additional data you might need to share between devices to keep content visually synchronized. For instance, you might want to share a window’s scroll position in addition to its contents. Brainstorm how users might enjoy your app’s activities in a face-to-face setting, and watch the WWDC sessions on designing and building spatial SharePlay experiences.

Prioritize Privacy.

visionOS places substantial emphasis on privacy, assisting users in protecting their personal information. In instances where accessing sensitive information is necessary, apps must obtain authorization from the device wearer.

Whether your app works with identifying information, financial data, location data, contacts, content, purchase history, search history, diagnostics, or other information, ensure you adhere to the fundamental principles of privacy:

Provide a tangible benefit for any data you collect.

When someone agrees to give you their personal information, make sure your app delivers tangible value in return. Don’t collect someone’s personal data just to have it. Put the data to work in the app itself, and don’t use it for advertising. Explain clearly how you use the information you collect. Be transparent with how you use people’s data. When requesting authorization for privacy-sensitive technologies, provide clear usage description strings that help someone understand why you need the data. Explain the data on your app’s page on the App Store. Secure the data you collect. Protect data by encrypting it whenever it’s not in use by your app. If you must store data locally, encrypt it before you write it to disk. Follow the App Store Review Guidelines and all applicable laws. For more information, view App privacy details on the App Store. For additional privacy guidelines, see Accessing private data.

Tune Your App’s Performance.

Performance optimization should be a key part of your development process. Optimizing your app now will make it easier to meet your performance objectives when you transition to visionOS. Regularly collect performance metrics, and use them to identify regressions before they escalate into larger problems.

Be aware of your workload.

Make sure the work your app performs provides a tangible benefit. Optimize algorithms to minimize your app’s use of CPU and GPU resources. Identify the workflows and scenarios to test. Create XCTest cases to measure the performance of your app’s most critical features. Run these tests under varying system loads to see how your app performs. Set performance and power targets for your app. Set aggressive performance targets for your app, and monitor those targets using Instruments. Make sure your app doesn’t waste power on unnecessary or duplicated tasks. Adopt a continuous integration (CI) workflow. Set up your builds to run in Xcode Cloud. Adopt a CI mindset by making sure every commit maintains the quality and stability of your code base. Run performance-related tests as part of your test suite. For more information about performance tuning, see Learning more about performance tuning.

Run Your iPad and iPhone Apps

visionOS includes the same frameworks available in iPadOS and iOS. This means your iPad and iPhone apps will run on Apple Vision Pro, each as a single scalable 2D window with their original look and feel. When the App Store becomes available, compatible apps will automatically become available with no action needed, providing users with more options in visionOS. Later this month, you can use Simulator and TestFlight on Apple Vision Pro to test your app. If your app isn’t well-suited for Apple Vision Pro, you’ll be able to modify its availability in App Store Connect.