Gemini Live already consumed Google Assistant, body and soul. Now, the AI that talks to you should be available for all users starting Tuesday. If you don’t care about a phone-based AI assistant, Microsoft is offering similar capabilities on Windows 11 through Copilot. If you’ve been jonesing to replicate Spike Jonze’s movie Her with your phone or computer, these programs may offer your first—but not likely your last—opportunity to get a little too intimate with your devices.
Gemini Live is accessible on Android through a long press of the power button. Rather than receiving the usual Assistant icon, users will now see the Gemini blue star icon appear. Gemini is already capable of comprehending what’s happening on screen, in case you want to ask it questions about a YouTube video you’re watching. It’s also capable of quick AI image generation in whatever text app you’re using at the time. Google first showed off Gemini Live alongside its Pixel 9 lineup back in August during its Made by Google event.
Previously, Gemini Live was only available for those paying for Gemini Advanced. That service is free the first month but costs $20 each month after. If you hold the power button, you’ll see a popup with an icon in the bottom right corner allowing you to use Gemini Live. You need to accept a one-page tutorial and chose a voice from a selection of male- and female-sounding English dialects before you can start bugging your phone with your inane questions.
Live is now available for all Gemini users in English on the Android app. We can’t wait for you to try it. https://t.co/jev4pnuZJ0
— Google Gemini App (@GeminiApp) September 30, 2024
Through the interface, you’ll be able to talk “naturally” to the phone and not have to worry about any flubs of speech, awkward phrasing, or accents that may have hindered Google Assistant. Gemini Live should have access to a wide variety of tasks on your phone, including interacting between your various apps, like messages and email.
On Tuesday, Microsoft declared that Copilot is getting similar, significant upgrades that will let it comprehend what’s happening on your screen. This capability is called Copilot Vision, and in the latest Windows update it will be aware of text or video you’re seeing on screen, allowing users to ask more questions about that content. Or, like in Microsoft’s promo video, you can have a very awkward conversation about photos of your grandma with an uncaring, unflinching AI chatbot.
“Your grandma has such a timeless style,” Microsoft’s Copilot AI tells Andrew, somehow sounding insincere, even for an AI. “Looks like she’s… uhh… a pretty remarkable person.”
Then there’s Copilot Voice, a similar feature to Gemini Live that’s accessible on your phone through the Copilot app. In its blog, Microsoft showed how you can ask Copilot for advice about buying fancy olive oils. Unfortunately, it doesn’t detail the very nuanced aspects of olive oils, about imports, how different brands might actually be blends of various countries’ exports, about types of olive oils best used for dressing rather than for cooking.
You also need to consider the security implications of both AI features. Gemini Live automatically turns on your phone’s microphone. You need to hold down the Hold or End buttons or tell it to “Stop” in order to quit the automatic recording. Gemini’s processing is saved to your Gemini Apps Activity, and those conversations are saved for 72 hours, according to Google’s privacy page.
Microsoft said Copilot Vision does not save any data to the cloud or your computer, and “the moment you end your session, data is permanently discarded.” It will also work with a limited list of websites to start, and it won’t work on any content contained behind a paywall. Microsoft probably doesn’t want to again take heat for its poor privacy decisions like with Copilot+ Recall or deal with online publishers and some members of Congress upset that Big Tech has scraped their content to develop their AI models.
Both features from Google and Microsoft are free, for now at least. Copilot Voice is rolling out slowly to users starting Tuesday, but Copilot Vision may be restricted to a limited number of Copilot Labs beta users.
