iOS 16 allows apps to initiate hands-free real-world actions

Apple has announced that its upcoming iOS 16 update will bring a new level of functionality to the way users interact with their devices. One of the most exciting features is the ability for apps to trigger real-world actions hands-free. This means that users can perform tasks like playing music or turning on an e-bike just by walking into a room or getting on it.

Table of Contents

Hands-Free Actions and Bluetooth LE Accessories

The new functionality leverages Apple’s Nearby Interaction framework, which allows third-party app developers to tap into the U1 or Ultra Wideband (UWB) chip on iPhone 11 and later devices. With iOS 16, apps will be able to start a Nearby Interaction session with a Bluetooth LE accessory that also supports Ultra Wideband in the background.

This update paves the way for a future where the line between apps and the physical world blurs. The new feature is part of a broader update to Apple’s Nearby Interaction framework, which was introduced at WWDC 2020 with iOS 14. This framework allows third-party app developers to tap into the device’s location services and interact with objects in the real world.

Background Sessions and Ultra Wideband

During the WWDC session, Apple explained how apps today can connect to and exchange data with Bluetooth LE accessories even while running in the background. With iOS 16, however, apps will be able to start a Nearby Interaction session with a Bluetooth LE accessory that also supports Ultra Wideband in the background.

Related to this update, Apple has updated the specification for accessory manufacturers to support these new background sessions. This paves the way for a future where devices and accessories can interact seamlessly with each other, even when the app is not actively running.

ARKit Integration and Enhanced Interactions

The new feature also includes integration with ARKit, Apple’s augmented reality developer toolkit. This allows developers to tap into the device’s trajectory as computed from ARKit, so their devices can smartly guide a user to a misplaced item or another object a user may want to interact with.

By leveraging ARKit, developers will gain more consistent distance and directional information than if they were using Nearby Interaction alone. The functionality doesn’t have to be only used for AirTag-like accessories manufactured by third parties, however. Apple demoed another use case where a museum could use Ultra Wideband accessories to guide visitors through its exhibits.

Use Cases and Future Possibilities

The updated functionality lays the groundwork for Apple’s rumored mixed reality smart glasses, where presumably, AR-powered apps would be core to the experience. The new feature is rolling out to beta testers of the iOS 16 software update which will reach the general public later this year.

Some potential use cases for the new feature include:

AirTag-like accessories: Third-party developers can build apps that integrate with AirTag-like accessories, allowing users to locate misplaced items more easily.
Museum exhibits: Museums could use Ultra Wideband accessories to guide visitors through exhibits, providing a more immersive experience.
Smart home automation: Users can control their smart home devices hands-free, using voice commands or gestures.

Developer Adoption and Future Possibilities

The success of the new feature will depend on how developers choose to adopt it. If accessory makers and third-party app developers integrate with the new functionality, we may see a future where the line between apps and the physical world blurs even further.

However, it remains to be seen whether developers will choose to put the technology to use. Apple has provided the necessary tools and framework for developers to build apps that interact with objects in the real world, but ultimately, it’s up to them to create innovative experiences that users want.

Conclusion

The new functionality arriving in iOS 16 is an exciting development in the world of mobile devices. With hands-free actions and enhanced interactions, users will be able to perform tasks more easily and efficiently. The integration with ARKit and Ultra Wideband accessories opens up new possibilities for developers to create innovative experiences that blur the line between apps and the physical world.

As we look to the future, it’s clear that Apple is pushing the boundaries of what’s possible with mobile devices. With this update, users can expect a more seamless and intuitive experience when interacting with their devices.