Growing up I was always partial to Math over English because there was no subjectivity, the answer you provide is either right or wrong. An essay in class could be one teacher’s A and another teacher’s C and you never know what to expect because it is just a feeling. However, what if we had essay graded artificial intelligence?
There is already a ton of bias in subjectivity related tasks, especially when it is a repetitive task. A study completed 10 years ago indicates that repeated decisions make people tired and they start looking for simple answers. The analysis was done in an Israeli court system, surveying 8 judges who made over 1,000 decisions who ruled on convict’s parole request. A hearing at the beginning of the day had 65% success rate, but dropping to almost 0% at the end of the day. However, if the judge had a snack break, their approval rate for paroles bounced back up to 65%.
This means that car buyers will decide to take standard options, judges might deny requests and let things stand as they are, and graders will just assume a grade and move on. This is just human nature as fatigue sets in or distractions occur. However, there has recently been a large adoption of robo-readers to help ease the burden of humans reading essays all the time. In fact, robo-readers have been used by the GRE. One human reader and a robo-reader, called an e-Rater, will read an essay on the GRE, if score differ substantially, another human reader is brought in to settle the discrepancy. The hope is that this will standardize scores better than leave a graduate application to the whims of whether you were first in the pile or last.
However, as we have mentioned before in class, with the positives there is always a downside and that some of the machine learning algorithms need to be continually audited. While bias is not as much of a notable concern with this. If a person were to learn what factors the algorithm uses and how they weight them, an essay could be written that does not make any sense, but still scores highly on the algorithm, which is why the human needs to be involved as well to keep a check on the machine. This differs from the SAT, which only uses human graders, and if the 2 human scores vary differently they bring in a 3rd.
This dovetails with the theme we have discussed throughout the semester regarding whether technology will actually take over our jobs. However, it appears, at least for now, technology is being built to help humans do their jobs better, instead of completely replacing humans. The synergies brought on by using a robo-reader means that the overall grading system costs less and accomplishes more, with less opportunity for bias. We have seen in other industries like farming or medicine, those roles have not necessarily been replaced, but those who use technology for their roles replace the ones that don’t.
In fact, in education at large there is growing support to use more artificial intelligence and robo-readers at younger ages. A study done by the University of Chicago indicates that machine learning can be used to identify at-risk students when they are much younger, allowing schools to focus extra resources on those kids to reduce the dropout rates overall. Additionally, it will be able to tell high achievers as well, meaning that instead of a one-size-fits-all approach to education, students can have a more adaptive and personalized learning path they can follow that will allow them to reach their full potential. It also can help speed up the pace in a larger class setting as well as artificial intelligence can help with new ways of perceiving information like visualization or simulation that a single teacher might not have found or been able to create on their own. Moreover, we have all been in those classes with the teachers who have tenure and been teaching the same things the same way for decades. However, using artificial intelligence the basis of the content can the stay the same, but can be modernized to be relevant to the newer audiences and have it customized for different learning curves of the classes.
I think about how quickly our phones pick up words we commonly use to try to complete our sentences to effectively make it easier to type. Now, when I type emails on my phone, or work on a group project on Google Docs, Outlook or Google will make suggestions based on other similar emails or papers based on what it reads from above. This is obviously done with all the content Microsoft and Google have at their disposal to be able to predict what I am writing and make suggestions for me to help me complete my work faster. Additionally, as someone who was numbers focused, it helps me ensure I am using proper grammar and might trigger a better way for me to say something than what I was initially thinking, helping me become a better writer as well.
A machine reviewing our work allows us to remove the biggest impediment and error prone component of the process, the human element. Hopefully we can use this to eliminate some of the unconscious bias evaluators, leading to more standardized scores. Not only will this allow for fairer evaluations, but hopefully leads to more personalized learning at a younger ages so we can give attention to those who need it and challenge those who need the challenge. Otherwise, we might never inspire students to think critically enough to find a way to push back when the machines are smarter than us.