Tokencito Game of the day Play

What is computer vision?

Computer vision is the field of AI that teaches a machine to understand images and video: recognize a face, read a license plate or find a dog in your photos. To the machine, a photo is a grid of numbers; the model learns to pull out of it what is there and where.

📷A photo🔢Pixels: numbers🤖The model finds patterns🐶"Dog, bottom left"

Put it to the test

A question from the app. Pick an answer.

A dog-spotting app learned from daytime photos. What about your night photos?

Start playing on the web

What a machine sees in a photo

To a computer, a photo is millions of pixels, and each one is a color number. Today, instead of writing it the rule "a dog has ears like this", you show it thousands of labeled photos and the model learns the patterns. That is machine learning; with neural networks of many layers, it is deep learning.

The typical tasks: classifying (is there a dog?), detecting (where is each dog?), separating each object from the background and reading the text in the image.

You use it every day

You unlock your phone with your face: the phone compares what the camera sees with your saved face. The more serious systems add a depth sensor, so a photo of you is not enough to fool it.

You type "dog" in your gallery and your dog photos show up, even though you never named them: a model looked at them for you, often on the phone itself. And the parking gate that opens by itself is reading the car's license plate.

Why it gets things wrong

Light and angle. In low light, against the sun, from the side or with half the face covered, the numbers in the photo change a lot, and the model may miss what you see right away.

The data. If training leaves some groups of people underrepresented, error rates can be unequal. A 2018 study (Gender Shades) found more errors with darker-skinned women in three commercial systems that classified gender from faces. And sometimes the model learns the wrong shortcut: in a well known experiment, a wolf versus husky classifier was really checking whether there was snow in the background.

Computer vision vs multimodal AI

Computer vision is the field: getting a machine to understand images and video. Multimodal AI combines more than one type of data, like the chat you send a photo to and ask what it sees. That chat uses computer vision inside, but a detector that only takes images is computer vision too, even if it is not multimodal.