The innovative Gemini Live feature in Google’s Gemini application has received a remarkable enhancement, allowing the AI to provide immediate responses to queries based on what it perceives through your smartphone’s camera and display. This groundbreaking capability is initially available on the Google Pixel 9 and Samsung Galaxy S25.

In the past, users could submit images or screenshots for analysis. However, the introduction of real-time interaction marks a significant shift, giving the impression that the AI can perceive and interpret the environment surrounding you. This functionality was previewed by Google under the Project Astra initiative last year, hinting at its potential scope.

As Samsung describes, the AI behaves like a reliable companion ready to assist, while Google highlights its potential for providing tailored shopping guidance, solving issues with broken items, or tidying disorganized spaces. Interacting with Gemini Live allows you to discuss anything within its visual field.

Currently, this updated feature is accessible as a complimentary upgrade for Pixel 9 and Galaxy S25 users, with other Android devices to receive it in the near future. However, broader access will depend on a Gemini Advanced subscription. As it stands, there’s no comprehensive list of devices scheduled to receive the upgrade, although it likely requires specific processing capabilities. There has been no announcement regarding its availability for iPhone users.

Users are reminded to verify information for accuracy. The presence of an advanced interface does not guarantee improved reliability for Gemini AI. Additionally, a stable internet connection is essential for functionality, as the application relies on online resources.

Locating the feature is straightforward: open the Gemini Live interface by clicking the icon located at the far right of the input field in any Gemini conversation (it resembles a sound wave). This will reveal two new options at the bottom: one for using the camera (depicted as a video camera icon) and another for sharing the phone’s screen (represented by an arrow within a rectangle).

When you exit the Gemini Live interface, your conversation will be saved as a typical text chat, allowing you to reference it later if necessary. During testing on a Google Pixel 9, queries were posed based on known answers to assess the AI’s accuracy.

Evaluating Gemini Live

The initial test involved the camera interface, where a question about the episode of Severance being viewed on a laptop was posed. At first, the AI mistakenly identified the show as You, likely due to confusing the actors, but it swiftly corrected itself by identifying the correct series and actors.

Next, information about a package labeled with the UN3481 classification—specifically, lithium-ion batteries contained in equipment (over-ear headphones)—was requested. The AI accurately acknowledged the involvement of lithium-ion batteries, stating they require “extra care,” yet provided incomplete information. When probed further, it incorrectly asserted that these batteries were stored separately, confused and misidentifying UN3480 instead.

Furthermore, when the camera was aimed at a Fitbit Charge 6, Gemini Live provided instructions for resetting the device, initially mistaking it for the previous Charge 5 model—an understandable error. This illustrates the potential utility of the AI in troubleshooting electronic items when users are uncertain about make and model details.

The screen-sharing feature with Gemini Live is noteworthy. The app minimizes to a compact widget, allowing normal phone functionalities, while enabling the user to pose questions about the visible content. Gemini Live adeptly recognized the applications in use and some in-app content, such as movie posters and band images, and even accurately translated a foreign language social media post.

Regarding a webpage with details on the recent Leicester v Newcastle soccer match, Gemini Live accurately reported the score and identified the goal scorers—information that was already displayed on-screen. However, when questioned about the match date, it incorrectly stated it occurred on May 22, 2023—a date from nearly two years prior for the same teams.

The rapid response time of Gemini Live is commendable, as is its polite and reassuring demeanor. However, concerns persist regarding the accuracy of its outputs. The convenience offered—pointing the camera and simply asking “how do I fix this?”—may lead many users to favor this approach, despite occasional inaccuracies, which remains a concern.

Ultimately, this enhancement is akin to a more instant variant of visual searches. Earlier, one would type queries like “UN3481 label” on Google for similar queries. Nevertheless, while traditional search results are composed of clickable links that allow users to evaluate the reliability of the information, Gemini Live operates as a more contained system, obscuring its reasoning process. Although interactions may seem almost magical due to its interface, the necessity to fact-check responses is less than ideal.