Has Facial Recognition Gone Too Far?

I remember the first time I got creeped out by Facebook’s facial recognition capabilities like it was yesterday. I was browsing my Facebook, checking my newsfeed, writing dumb statuses, and then BAM! Facebook accidentally tagged me in a really funny unfortunate photo of my mom.


Sorry mom, I swear it’s just for class

The Deepface project was officially presented in the IEEE Computer Conference in March of last year. As as Verge writer Josh Lowensohn states, “the technology maps out 3D facial features, then makes a flat model that’s filtered by color to characterize specific facial elements.”

Screen Shot 2015-03-17 at 8.21.20 PM

According to Facebook, this system allows the site to recognize user’s faces with 97.5% accuracy. To put that in perspective, that is significantly more advanced than the FBI’s facial recognition system, which only guesses faces with about 85% accuracy. The reason for this is that unlike the FBI, Facebook does not rely on poorly angled photos taken at various bureaucratic interactions or taken through security cameras, but rather on the 250 billion photos that Facebook users have posted of themselves. Most of these pictures are of excellent quality, and are in fact designed precisely for members in our social network to recognize our face immediately.

To illustrate the significance of this power more tangibly, Alessandro Acquisiti provides an experiment he did with this tool (or something close to it) with students at a university (this is taken from this Ted talk):

“We asked students who were walking by to participate in a study, and we took a shot with a webcam, and we asked them to fill out a survey on a laptop. While they were filling out the survey, we uploaded their shot to a cloud-computing cluster, and we started using a facial recognizer to match that shot to a database of some hundreds of thousands of images which we had downloaded from Facebook profiles. By the time the subject reached the last page on the survey, the page had been dynamically updated with the 10 best matching photos which the recognizer had found, and we asked the subjects to indicate whether he or she found themselves in the photo… the computer did, and in fact did so for one out of three subjects.

The power of this system is striking, and the FBI and other government agencies know it.

As of June of 2014, Facebook lost a case against the Manhattan district attorney that allows the Manhattan city government to demand  “nearly complete account data… ranging from pages they had liked to photos and private messages” and also to Facebook’s facial recognition system. If scaled up, this means that national agencies and state and local governments may have the right to demand this information as well. While the case is currently in appeals, it nonetheless suggests that Facebook’s remarkable facial recognition system could be in the hands of the government in just a few years.

Even if it isn’t, this still show’s how contentious and powerful facial recognition is in the digital age. On this point, I think most people, like myself, find this to be quite creepy and chilling. While of course there are very strong political and business incentives for the government and for Facebook to use this information properly, morally, and legally (i.e for security purposes, quick-tagging, etc.), the threat that they may abuse this power against our own will is a serious concern. A Harvard Law Review article entitled “In the Face of Danger: Facial Recognition and Limits of Privacy Law” makes this point clear: “as our networked world has expanded what is possible, it has been slow to catch up and determine what is permissible.” That is to say, the abuse of facial recognition, either by the government, or by Facebook, or by some other third party, may (and maybe must?) occur before any legal changes can be made to prevent it from happening again or altogether.

If these seems as crazy to you as it does to me, then you may be interested in buying these privacy glasses that prevent Deepface from recognizing your facial features. I can’t promise you that you’ll look good, but this is definitely an interesting way to protect your privacy rights, and perhaps to prevent the emergence of Skynet and the upcoming robot apocalypse.


Actually, yeah, I can’t promise you that you’ll look good. But I can promise you that you’ll look really, really, really weird. 


  1. I definitely agree with all your points here (and the great picture of your mom). I’m concerned with the internet becoming “permissible”, as you found in the Harvard Law Review article you quoted. Although I understand the benefits of big data and facial recognition for companies, I fail to see the benefits for us “regular folk”. Yes, it can be easier if Facebook tags me or my friends automatically because it saves me from doing it myself, but it’s not life changing. Until recently, I didn’t even know there was an option to have Facebook not automatically tag you and use facial recognition in that way- but I’m glad there is. I feel like we eventually will have even less say than we have now regarding our privacy and data mining issues.

  2. It’s funny that you posted about the topic of facial recognition. This morning I was tagged in one of my friend’s photos on Facebook (she had graduated from BC last year). The photo was an image of her with other cheerleaders. When I checked my notification, I thought I looked nothing like the girl in the photo. For example, a quite noticeable difference was that I have blonde hair and she was a brunette. I can’t help but wonder if Facebook recognized and tagged me in the photo because it is aware that I am a cheerleader, I have been photographed at a distanced and slightly resembled this girl, or if we have uncanny similar features that for some reason I can’t see. As mentioned in your post, the 97.5% recognition accuracy is such a high percentage. Personally, I would rather go through and tag my friends in photos I post than have Facebook do it for me. As @hannagreenstein was saying, I believe as well that one day we will have less say regarding our privacy and data mining issues. I wonder if Facebook will automatically tag friends in our photos based on the facial recognition system without our consent. Only time will tell.

  3. When I read this post, I immediately thought of the article I read today about Alibaba’s new facial recognition for payments! The new “Smile to Pay” works just like Apple Pay’s finger print scan, except it scans your face.
    Here is the link to the article: http://www.businessinsider.com/alibaba-jack-ma-shows-facial-recognition-for-mobile-payments-2015-3
    While you raise very good points about facial recognition and privacy issues, what about if you think about it from the opposite perspective? For example, could Alibaba’s new invention be safer for making online payments? If their facial recognition is close to 99% accurate, this could be revolutionary! But at the same time, facial recognition doesn’t always work and maybe this is creating a new way for people to hack into accounts.
    I think it is incredible what technology can do now days and it is so interesting to see how people apply these new technologies to create new applications or to try and simplify things. But with all these changes, people also always discover new ways to misuse the technology.

  4. This is such a cool post! I always wondered what technology Facebook leverages to identify who users want to tag in their photos. It’s crazy to hear that Facebook has better recognition technology than the FBI. It would be interesting to know if Facebook and the FBI could possibly collaborate in the future (which they probably already do!). Though I do think the facial recognition in Facebook is extremely creepy, I have never thought of it as an invasion of privacy. After reading your points, however, I now see the privacy issues with it. I think the quote, “as our networked world has expanded what is possible, it has been slow to catch up and determine what is permissible,” that you included is a great way to sum up the lack of legality that is involved with social media. It will be interesting to see how the legal implications with social media will evolve in the next few years.

  5. Interesting! I also wondered how facebook recognizes my friends’ faces to tag them in the picture as well. I think besides all sources of information we give up to social media, the face that identifies us is one of the very intimate and true to who we are. Privacy has always been an debating topic, so I don’t know how the government will regulate this.

  6. I really enjoyed reading your post. I also think that it is very creepy, that Facebook now knows not just my name, age, hometown and what I like, but also what I look like. As I don’t upload dozens of pictures, the new function Facebook provides is not very attractive to me. I don’t mind tagging my friends by myself. Furthermore I totally agree that facial recognition can be a threat in the future and can easily be abused.
    While reading social media news for our twitter activities, I came across another facial recognition app called NameTag. The creators of the app say that it can spot a a face using Google Glass’ camera or an Android phone and within seconds return a match complete with a name, pictures and social media profiles. But this app is already being discussed widely to open a door for stalkers and predators. So hopefully soon gets forbidden on every device.

  7. Really great post. Facial recognition does creep me out a bit, but I don’t think there’s any stopping it. My banking app on my iphone actually uses facial recognition instead of a pin, which is actually quite handy.

  8. Wow, 97.5% accuracy! That is crazy. This weekend I was stunned when I posted a picture of my friend it automatically tagged him. The first time I came into contact with facial recognition was on iPhoto a long time ago, I remember being in awe as I thought the feature was the coolest thing ever. It began to sort my pictures by people that ultimately saved me a lot of time when searching for pictures. Since then my feeling towards it has become increasingly skeptical as the accuracy is increasing. As @profkane predicts, I too do not think it is going away anytime soon! It will be interesting to see the effects and innovation to come as the technology and software continue to develop, I had no idea it was possible to unlock a bank account. I really like your creative approach, nice post.

  9. I really enjoyed reading this post. I didn’t know much about facial recognition and learned a lot from this! However, facial recognition doesn’t scare me (is that bad??). I find it very useful on Facebook because it saves me a ton of time when I upload pictures. Furthermore, I don’t feel that I have much to hide from any government agency looking to gain access to Facebook’s information. If I’m not hiding from anything does it really matter? I think facial recognition will continue to gain popularity and we’ll see it being used in all kinds of new ways and I’m excited to see what companies choose to use it for!

  10. Facial recognition is becoming a scary thing! while I do think the improvements aspect to this technology form is a credit to how incredible technology in general continues to become, I just think that there are so many negatives that could occur if the government gets its hands on this.

  11. imagine how well Snapchat can identify us

%d bloggers like this: