Understanding what you are looking at
Visual Intelligence is Apple’s feature for using AI to understand what the iPhone camera or the screen is showing, and connecting that to search or an action. Check a shop’s opening hours from its sign, add a date from a flyer to your calendar, identify a plant or an animal.
Beyond recognizing an image, it reads text, searches, connects to external AI, and drives app actions in one continuous flow. It is best understood as Apple’s application of multimodal capability — handling images and text together.
When it appeared, and what changed
Visual Intelligence is not new in WWDC26. It was first announced on 9 September 2024 with the iPhone 16 series, initially as a way to open the camera from Camera Control and look up objects and places around you.
At WWDC26 it became more integrated as part of Siri AI, extending beyond the camera to understanding what is on screen and turning that into questions and actions.
What it does
- Look up objects, animals, plants, and places seen through the camera
- Read text from posters and flyers and turn it into events or contacts
- Search for a product or an image on screen
- Ask Siri AI about what you are looking at
- Use external services such as web search or ChatGPT where appropriate
Cautions
Image recognition confuses similar products and plants and misreads text. Do not use it directly for decisions involving health, safety, or contracts — check the original information.
When an external service is used, images and questions may be sent outside Apple. Check the confirmation screen before sending and be aware of what is being shared.