My first exposure to Tobii Eye Tracking Technology occurred when I was hanging out with my cousin Buck. He has Cerebral Palsy and as a kid, I was barely able to communicate with him. Having Cerebral Palsy for him meant that he had limited motor function yet his brain was still working at full speed. He’s only 2 years younger than me but because he was unable to speak words, there was a communication barrier which made it far more difficult for me to connect with him than it normally would be for someone his age. My first conversations with him involved a smile for yes and turning his head to the left for no. This made our conversations very simple and yet, there was so much more going through Buck’s head that I couldn’t learn about.
However, as I grew older I got to know Buck through other methods. Buck is able to pretty precisely move his head left and right so I was able to ask him preferential questions about his favorite sports teams (Buck loves the Red Sox and the Boston Bruins). I would hold up each finger and ask him which game was on today that he was excited for. Yet, there still was a barrier to know just what Buck was thinking at a deeper level.
When Buck acquired his first eye Tracking Technology computer, I was finally able to hear Buck speak! When setup in his wheelchair, Buck could gaze at icons on a screen and the computer would be able to track his eye movements to know where he was staring at. When he centers on an icon for long enough, the computer registers this as a click and the word or phrase can be recited by the device. Using this for the first time with Buck was unlike any other conversation we had before. He was able to ask me questions back, or make quips and interject when other people were speaking. The conversation felt much more dynamic and as a result Buck could participate more easily.
Pictured Below is what the Screen for Buck might look like. Customization is high and different word sets can be selected depending on the setting. For example, if he is at the Beach, it might have a bunch of Icons which are labeled Towel, Sunglasses, and Sunscreen.
More recently, Buck has actually gained a new feature with this device integrated with his cell phone. He is now able to make phone calls and send text messages through it. Using Eye Gaze technology, we had a quick phone call as I was telling him about this very article and the voice from his computer allowed him to speak back to me. Through Eye Gaze technology, Buck is able to participate in the texting phenomenon which previously was more difficult for him to participate in.
Tobii Technology has been around since 2001 when the company was first founded but the technology has been improving ever since. Tobii first received Venture Capital funding in 2007 ($14 Million) and have received other rounds of funding including Intel in 2012 for $21 Million. Most recently there has been a partnership with Perceptive Devices announced in August of 2017 to bring facial recognition and smile control to Tobii’s devices. Doing so would allow for people like Buck to merely smile when he wanted to click, rather than having to gaze and stare at an icon which can sometimes require more strain than a simple smile.
Unfortunately, these devices are quite expensive and they’ve been difficult to acquire in the past. However, eWeek wrote about Microsoft trying to expand the use of eye tracking technology to a much wider audience. Announced as Beta on August 3rd of 2017, Microsoft Eye Control might be the key to unlock this treasure trove of eye tracking technology for people with disabilities. Interestingly, this technology actually has to be paired with a Tobii Eye Tracker 4C, a $149 device designed for use on a computer not produced by Tobii. Unlike the eWeek article posted up above, the promotion video from Tobii actually emphasizes the ability for everyday PC users to seamlessly and intelligently navigate windows, dim their screen and access new feature in video games (see below).
Most people have not yet been exposed to eye tracking technology because it hasn’t been high enough quality, it’s too expensive, or there hasn’t been a use case which has been proven to justify the cost of the current technology. However, Tobii has officially done the coolest thing ever and made their product more affordable ($150) and adapted it for video games (thank you). They actually have added some sweet features highlighted here to games like Assassin’s Creed Origins, the newest release in the popular Assassin’s Creed Series. Players can turn the camera angle based upon their gaze, change the brightness of certain areas if a player is staring at the sun for example, or “tag” enemies when a player looks at them. These borderline features actually look quite cool from an immersion standpoint, but with the two analog sticks, the current camera adjustment method from first person shooter games has become second nature for many gamers anyways.
Games have struggled over time with camera adjustment and until the two analog stick controller arose, people constantly were fighting to be able to see where in the game they wanted to. Games on the Nintendo 64, way back in the 1996, had a single analog stick and sometimes utilized the “d-pad” for camera adjustments which when I tried it, was not a great experience (see Mario above). As mentioned above, most of the time this issue is fixed through the use of two analog sticks but “For Honor”, a new medieval sword fighting style game, has opted for a new control scheme which complements Tobii’s Eye Gaze camera control. They actually use the right analog stick for sword movements when locked in combat and as a result, changing the camera angle is impossible. Seeing more creative uses for the second analog stick or giving users a more primary focus on buttons would unlock new ways to interact with video games in the future. I’m thoroughly excited to see if this technology becomes more widely adopted and influences the Video Game Industry.