Google has outlined new developments in bringing its Gemini artificial intelligence (AI) technology to glasses and headsets through the Android XR platform.
The company is aiming to integrate Gemini AI into devices such as smart glasses and headsets, providing users with a hands-free assistant that understands their immediate context and environment. Android XR is described as the first Android platform developed for the Gemini era and has been designed to support a range of XR (extended reality) devices.
According to Shahram Izadi, Vice President and General Manager, XR at Google, the integration of AI and XR is intended to deliver a more natural, accessible digital experience. "What if your AI assistant could see the world from your perspective and offer hands-free help? That's the vision driving our latest advancements in Android XR. It's the first Android platform built in the Gemini era, and it powers an ecosystem of headsets, glasses and everything in between," Izadi stated.
He further explained the immediate benefits to users: "With Gemini on these devices, you can have an assistant that's right there with you, sharing your vantage point, ready to help whenever you start the conversation. This frees up your hands so you can stay present and engaged with the world around you, whether it's real or virtual."
One of the headline features is the ability for Android XR headsets, including expected releases like Samsung's Project Moohan later this year, to understand and act upon what the user is seeing. For example, wearers can ask questions about their environment, receive contextual information, or have actions performed via the device.
The company highlighted that Android XR glasses, equipped with cameras, microphones and speakers, can work in conjunction with a paired smartphone. Users will be able to access apps and receive information on an optional in-lens display. This setup allows for features such as private notifications, navigation, and direct communication, all while enabling users to remain hands-free.
Izadi noted the broader aim of this technology: "Equipped with a camera, microphones and speakers, these glasses work in tandem with your phone, giving you access to your apps without ever having to reach in your pocket. And an optional in-lens display privately provides helpful information right when you need it. Pairing these glasses with Gemini means they see and hear what you do, so they understand your context, remember what's important to you and can help you throughout your day."
Demonstrations included live scenarios such as messaging, scheduling appointments, navigating with turn-by-turn directions, taking photos, and performing live translations. The company also showed how the technology could provide real-time subtitles during conversations, potentially assisting users to overcome language barriers.
On the subject of user comfort and everyday adoption, Izadi acknowledged the balance between functionality and wearability. "We're working with brands and partners to bring this technology to life. We know that glasses can only truly be helpful if you want to wear them all day. Today, we announced that we're partnering with innovative eyewear brands, starting with Gentle Monster and Warby Parker, to create stylish glasses with Android XR. And in the future, we look forward to working with more partners, like Kering Eyewear, to bring even more options to users."
Google is also building on its relationship with Samsung to extend Android XR beyond headsets and into glasses. The collaboration aims to set up both a software and reference hardware platform, giving developers the resources to begin creating products for the XR ecosystem later this year.
Addressing the importance of privacy, Izadi commented, "To help make sure we're building a truly assistive product that respects privacy for you and those around you, we've begun gathering feedback on our prototypes with trusted testers."
The company intends to release further updates on the progress of its Android XR glasses and device availability as development continues.
