Facebook and facial recognition – you’ve been tagged



Around 2 million photographs are uploaded to Facebook each day. As of this week, every new image will be processed by automatic face recognition software, designed to identify the people in the photographs.

It’s not the addition of this new function that has caused the creeping sense of unease, but the covert manner with which it has been activated.

The new feature has been switched on as a default: all of your photos will now be processed and tagged unless you decide to opt out of the service.

What should have been provided as a tool to make lives easier has instead left people feeling their personal data is under the control of a technology they don’t understand.

How worried should we be about the widespread implementation of face recognition technology?

First of all it is important to understand that the technology would not work without the Facebook user’s guidance. As Dr Brian Lovell has already explained to the Australian media this week, face recognition software is especially poor when dealing with images taken in uncontrolled environments such as a bar, club, and music festival locations, which appear to make up the majority of the 90 billion image corpus.

Given the significant degree of error generated by these algorithms (my colleague Dr Kristy Martire told me today that her friend was tagged as an obscure Argentinian statue), the user input is necessary to constrain the number crunching.

Manual tagging is actually the clever part, because humans have an especially impressive ability to recognise faces. Psychologists generally agree this is the most advanced feat of recognition that the human visual system has developed.

To understand just how good we are at recognising familiar faces it’s useful to firstly consider just how difficult the task becomes with unfamiliar faces. So, a little experiment:

In the image array below I have collated ten photographs. The one you see on the bottom right is my current passport image. Five are from my own Facebook account. The remaining four are of a friend that looks similar to me.

I invite you to try and cleave this mixture of images into two separate identities. Depending on how confident you feel, you can post your answer in the comments section. If you do, I will pick the winner later.

Psychological research has repeatedly shown that humans are actually very bad at verifying the identity of unfamiliar people from photographs. This is true when comparing good quality photographs to one another, and also when comparing photo-ID to physically present individuals.

Because the same tasks become trivially easy when we are familiar with the person in the images, a number of visual scientists are currently trying to unravel the process that drives this improvement.

Computers and people are similar in respect to faces – both are unable to reliably tell if two images of unfamiliar people are of the same person or two different people. This unreliability is a real problem in security settings and perhaps why it has been deemed necessary to allow other biometric data (such as iris recognition) to be stored on travel documents.

Civil libertarians beware, however: Facebook provides the ideal platform to develop a procedure for accurate automatic facial recognition.

Firstly, it harnesses humans expertise at recognising familiar faces by giving them a monitoring role.

Secondly, the high degree of image variation available on user profiles is in fact ideal for training face recognition algorithms, allowing them to reach an almost perfect level of accuracy when searching small databases.

The performance of face recognition software has improved dramatically over the last ten years. This improvement leads some to believe that, instead of having to endure all the boring and time-consuming introductions at social gatherings, we might soon be able to shove our camera phone in the faces of strangers and be done with it.

Alas, current data suggest this will not happen soon, if at all.

In the meantime, it’s not the algorithms that we should worry about. It is the snap-happy friend who indiscriminately tags photos of you in your least flattering pose.