A Faustian Bargain

Over the course of this class, there was probably at least one moment per meeting when it struck me that we must be on the precipice of a new technology trend that would change the way society or the economy functioned in a more dramatic way than ever before.  These moments included learning about how Bitcoin and other crypto-currencies work and Amazon news hours about how Amazon is scheming to take over our lives.

During each of these conversations, the creepy-cool line was consistently brought up and the class was usually split on a consensus.  Half of the class was willing to accept


these new technologies as vehicles for added convenience in our lives and the other half rejected them for taking away a piece of privacy that is too precious to give up.  Whichever side of this debate you landed on, if you looked back at where the creepy-cool line was just ten years ago, there were issues that we all would have been on the creepy side of that, that with the passage of time, we’ve all come to accept as the norm.  For example, I remember the first time that an ad for the shoes I looked at on DSW.com showed up as an ad on my Facebook timeline. Or, years ago we would have found it creepy for an app to ask for access to our location, let alone an app designed specifically for this like Find My Friends.  By now, I have welcomed this onto my phone just for the convenience of knowing if my roommates are home.

However, there is still one topic that I am vehemently on the creepy side of, while knowing full well that in a matter of years I won’t have a choice to avoid.  This has to do with the trust economy.  There is a show called Black Mirror in which each episodeblack mirror.jpg tells a different story about a dystopia based on an aspect of technology pervading our lives.  One episode in particular called “Nosedive” is a disturbing commentary on the trust economy.  It follows a young woman’s life in a world where everyone rates each other based on every interaction, and these ratings operate as a currency that dictate every aspect of life from social circles to the ability to rent a car or live in an apartment complex.  This woman’s obsession with her ratings eventually leads to her downfall and isolation from society.

uber pic.PNG


What makes this episode so cringe-worthy is how close it feels to where we are now as a society.  While there are benefits to these rating systems, like holding each other accountable for our behavior, it is not all good: “Ratings have turned customers into unwitting, and sometimes unwittingly ruthless, middle managers.”  The end of the episode had me questioning whether I should be willing to participate in these systems at all, but at a certain point there is no option of opting out if the world is built around the expectation of having access to these services.  When trustworthiness is a currency, opting out completely is equivalent to being deemed untrustworthy.

What is more frightening than independent rating systems for each company, is a conglomeration of every one of them monitored by the government to dictate our opportunities.  This is essentially what’s happening China right now.  Currently, China does not have credit rating agencies like Equifax, Experian, and TransUnion in the U.S. This makes it difficult for people to get loans or take out credit cards, because companies have no basis on which to trust them.  But no worries, China has found a “solution” to ameliorate this inconvenience. In place of credit rating agencies, the Chinese government plans to create a “Citizen Score” to decide the trustworthiness of citizens ranked against all other citizens.  The Citizens Score takes into account behavior like purchases, ratings from service providers in the sharing economy, and most frighteningly, who you are friends with and even associate with.  Moreover, the government judges each of these attributes as good or bad.  So if you are applying for a loan and need a good Citizen Score, you will not want to associate with people who act in ways that the government does not agree with.  In effect, this could be a way to incentivize people to disassociate from and isolate people who speak out against the government.  Though this may be far off, it is not difficult to imagine ways in which our individual ratings might eventually be used in ways that are severely detrimental to society and even democracy.

One of the biggest lessons I have taken away from ISYS6621 is that innovation can lead to highly positive change – but there will always be some way for that innovation to be manipulated.  It takes iterations and revisions to respond to ways these innovations can be abused.  Though this may sound like a downer conclusion, I think that as such potent technology becomes a more significant part of our lives this lesson is becoming increasingly important.  I really appreciate that this class has helped me look at these changes through a more nuanced lens.  I might still be wowed by the ability to unlock my phone with my face, but will also be inclined to consider the ripple effects of such changes in other aspects.


  1. I completely agree with wanting to stay away from certain programs that provide ratings for one another. But just like the network of Facebook, it becomes so big that it is almost impossible to avoid. I do find the Citizenscore very intriguing. I would love to see how that plays out. If it is run by the government it seems to be a way for the government to control dissension in the country. If it was done here I could easily see it as a way to prevent free speech; if you say something that someone else doesn’t like and they control part of the score ratings, they will inherently rate you as bad in some categories because you don’t think like them.

  2. I love your blog post! I too watch Black Mirror and think your comparison of the similarity of “Nose Dive” and trust economy really interesting. The purpose of the show is definitely to shed light on how technology can affect our lives in a negative way. There is an episode where the technology enables users to block other people out of their life, if a person is blocked, they cannot be heard or interact and appear as a shadow outline. It’s a crazy concept, but definitely brings to light where the creepy cool line lies with technology. I thought #IS6621 did a really great job all semester of bringing up new technologies in an unbiased way that really let the class discussions debate on the pros and cons of the innovation. Great blog!

  3. Great post Marielle. That one episode of Black Mirror is definitely creepy. As you mentioned, the standards of “creepy” have drastically changed over the last few years, and the line is getting ever more blurry. But I hope the community-wide rating system for every interaction and transaction will never be on the “cool” side, at least here in the US. The “Citizen Score” that the Chinese government is planning to implement is downright absurd – especially if it means that choosing not to participate in the program is the equivalent of deeming yourself untrustworthy and makes you ineligible for loans and mortgages.. that’s really abusive. Thanks for bringing the possibility of that program to light – I hope it never materializes.

  4. I thought that Black Mirror episode was interesting. My solution to the ratings problem is that I use the ratings as input to my decision, but I’m the decider. That works in a classroom where the professor is supposed to be the authority (and maybe in a company for similar reasons), but not in government or society.

%d bloggers like this: