Google brings powerful visual search to AI Mode for Android and iPhone users

Google may be the current king of search, but thanks to competition from artificial intelligence services like ChatGPT and Grok, its continued dominance is becoming less guaranteed. In other words, Google can no longer rest on its laurels with search -- it must either evolve or dissolve.

In an effort to keep up with the times, Google is delivering its AI Mode feature to more users in the U.S. today. This essentially adds new capabilities that combine visual search with the Gemini AI model. Originally limited to Google One AI Premium subscribers, the feature is now available to millions through the Labs program, accessible in the Google app for both Android and iPhone.

AI Mode has historically been used for longer, more complex queries compared to traditional Google searches. According to the company, users are turning to the feature for tasks like comparing products, learning how to do something, or planning trips. The average AI Mode query is roughly twice the length of a standard search. So yes, people are engaging with it in a different way.

The latest update adds advanced visual search by integrating Google Lens directly into AI Mode. Users can now take a photo or upload an image, ask a question about it, and receive a detailed response with helpful links. This isn’t just basic image recognition, folks -- it is designed to understand the context of the image, including the relationships between objects, colors, materials, and layout. Very cool.

Gemini’s multimodal approach lets AI Mode interpret entire scenes. For example, if a user uploads a photo of a bookshelf, AI Mode can identify each book, provide information on them, offer similar recommendations, and link to places where those books can be purchased. The system can also respond to follow-up questions based on the initial image. It seems almost magical.

Behind the scenes, Google uses what it calls a “query fan-out” method. Instead of running a single search, AI Mode generates multiple queries based on the image and its components, aiming to deliver broader and more detailed information than traditional search.

The feature is still in testing, for now, so keep that in mind if you decide to play around with it. Those interested in trying it can access AI Mode through the Google app on Android or iOS.