Apple is taking its search capabilities to the next level with Apple Visual Intelligence. This new feature, showcased at the recent Apple event, allows users to search using the native iPhone camera. Similar to Google Lens, Apple Visual Intelligence uses third-party search providers like Google, OpenAI’s ChatGPT, and Yelp to deliver search results based on the type of query.
The search experience with Apple Visual Intelligence seems seamless and intuitive. For example, if you come across a bike you want to buy during your walk, simply snap a photo of it and the app will display “Searching with Google…” as it retrieves relevant information. However, it should be noted that the search results shown in the demonstration may have been altered for presentation purposes.
Another example showcased during the event was a local search for a restaurant. By taking a photo of the restaurant, Apple Visual Intelligence pulls up local search results from Apple Maps, which is powered by Yelp and OpenTable. This integration allows users to easily find more details about the restaurant and even make reservations through OpenTable directly within Apple Maps.
Apple seems to be leveraging AI as a tool rather than a foundational element for its devices. By integrating with various search providers, Apple Visual Intelligence provides users with a comprehensive search experience. There is undoubtedly underlying AI and machine learning happening on the Apple device, but the results are sourced from third-party providers like Google and OpenAI.
However, it’s worth noting that early beta reviews from the Washington Post suggest that Apple Visual Intelligence still has some room for improvement. The review mentions issues with hallucinations, marking spam emails as priority, and other problems. As with any new technology, there are bound to be some hiccups along the way.
Overall, Apple Visual Intelligence is an exciting addition to the iPhone camera capabilities. It allows users to effortlessly search for information by simply taking a photo. With its integration with popular search providers, Apple Visual Intelligence provides a seamless and comprehensive search experience. While there may be some areas for improvement, it’s clear that Apple is pushing the boundaries of AI and machine learning to enhance the user experience on their devices.
