Google is turning your phone’s camera into a real-world CTRL + F

Google is turning your phone's camera into a real-world CTRL + F

“Scene explorer is a powerful ability in our devices’ ability to understand the world the way we do, to see relevant information overlaid in the context of the world all around us,” Raghavan said. “This is like having a supercharged Control-F [find shortcut] for the world all around you.” Another feature expands Google’s multimodal search, which combines text and images into one search query. Now, by adding “near me” text, you can tailor searches to nearby results. For example, you can combine a photo of some unknown dish with the words “near me” to find a restaurant nearby that’ll serve it, Raghavan said. That feature will arrive later this year for English speakers.

Google I/O will include “scene explorer” technology that will assist you in finding the perfect product on a crowded store shelf. Google launched two new search tools on Wednesday that use pictures from the web or photos you take in a shop, as part of the company’s push to go beyond the words you put into a search box. The scene explorer function, which was revealed at the Google I/O conference, allows you to sweep your phone camera across a shelf of merchandise at a supermarket or pharmacy to recognise the objects in view. Google then overlays product information and reviews on the screen, allowing you to select nut-free snacks or scent-free cosmetics, according to search chief Prabhakar Raghavan. It is an addition to the Google Lens app.

Highlights

  • Search, the first service Google offered and the one that propelled it into today’s multi-product juggernaut. It remains a core part of the company’s mission to make information useful to the world’s population, and search ads are still Google’s single biggest source of revenue. At Google I/O, Google also said it’s adopted a new 10-tone Monk Skin Tone Scale to better improve diversity in its AI training data, search results, and other operations.