The latest video game controller isn't made of plastic.  It's your face.

The latest video game controller isn’t made of plastic. It’s your face.


Over decades, input devices in the video game industry have evolved from simple joysticks to sophisticated controllers that provide haptic feedback. But with Enabled Play, a new assistive tool created by self-taught developer Alex Dunn, users are embracing another type of input: facial expressions.

While companies like Microsoft have tried to expand accessibility through adaptive controls and accessories, Dunn’s new device takes those efforts even further, translating users’ head movements, facial expressions, real-time speech and other non-traditional input methods into mouse clicks, keystrokes and thumb movements. The device makes users raise their eyebrows – literally.

“Enabled Play is a device that learns to work with you — not a device you have to learn to work with,” Dunn, who lives in Boston, said via Zoom.

Dunn, 26, created Enabled Play so that everyone—including his younger brother with a disability—can interact with technology in a natural and intuitive way. At the start of the pandemic, the only thing he and his New Hampshire-based brother could do together, while about 70 miles apart, was game.

“And that’s when I started to see some of the challenges that he had and the limitations that gaming had for people with really all kinds of disabilities,” he added.

This geographer helps video game developers avoid teasing countries

At 17, Dunn dropped out of Worcester Polytechnic Institute to become a full-time software engineer. He began researching and developing Enabled Play two and a half years ago, which initially proved challenging, as most speech recognition software lagged behind in response time.

“I built some prototypes with voice commands, and then I started talking to people who were deaf and had a range of disabilities, and I found that voice commands didn’t cut it,” Dunn said.

That’s when he started thinking outside the box.

Already built Suave keysa voice-controlled program for players with disabilities, Dunn created Snap Keys — an extension that turns a user’s Snapchat lens into a controller when playing games like Call of Duty, “Fall Guys” and “Dark Souls.” In 2020, he won two awards for his work on Snap Inc.’s Snap Kit Developer Challenge, a competition among third-party app creators to revamp Snapchat’s developer toolkit.

With Enabled Play, Dunn takes accessibility to the next level. With a wider range of inputs, users can connect the aid – equipped with a robust CPU and 8GB of RAM – to a computer, game console or other device to play games in the way that works best for them.

Dunn also spent time making sure Enabled Play was accessible to people who are deaf, as well as people who want to use non-verbal audio input, like “ooh” or “aah”, to perform an action. Enabled Play’s vocal sound detection model is based on “The song-controlled joystick,” which engineers and linguistics experts at the University of Washington developed in 2006.

“Essentially, it looks to predict the word you’re going to say based on what’s in the profile, rather than trying to guess it could be any word in the dictionary,” Dunn said. “This helps cut through machine learning bias by learning more about how the individual speaks and applying that to their desired commands.”

Dunn’s AI-enabled controls take into account a person’s natural tendencies. If a player wants to set a jump command every time they open their mouth, Enabled Play would identify that person’s individual resting position and set that as the baseline.

In January, Enabled Play was officially launched in six countries – its user base stretches from the US to the UK, Ghana and Austria. For Dunn, one of his primary goals was to fill a void availability and pricing compared to other gaming aids.

“There are things like the Xbox Adaptive Controller. There are things like the HORI Flex [for Nintendo Switch]. There are things like Tobii, which does eye tracking and stuff like that. But it still seemed like it wasn’t enough, he said.

Compared to some devices that are only compatible with one gaming system or computer at a time, Dunn’s AI-enabled controller — priced at $249.99 — supports a combination of inputs and outputs. Speech therapists say that compared to augmentative and alternative communication (AAC) devices, which are medically necessary for some with disabilities, Dunn’s device offers simplicity.

“This is just the beginning,” says Julia Franklin, a speech pathologist at the Community School of Davidson in Davidson, NC. Franklin introduced students to Enabled Play this summer and feels it’s a better alternative to other AAC devices on the market that are often “expensive, bulky and limited” in usability. Many sophisticated AAC system Can reaches from $6,000 to $11,500 for high-tech devices, with low-end eye trackers running into the thousands. A person can also download AAC apps on their mobile devices, which range from $49.99 to $299.99 for the app alone.

“For many people who have physical and cognitive differences, they often exhaust themselves to learn a complex AAC system that has limits,” she said. “The Enabled Play device allows individuals to tap into their strengths and movements that already exist.”

Internet users have applauded Dunn for his work, noting that asking for accessibility should not be equated with asking for an “easy mode” – a misconception often cited by critics of making games more accessible.

“How to make games accessible” wrote a Reddit user about Enabled Play. “Not by dumbing it down, but by creating mechanical solutions that allow users to have the same experience and achieve the same feats as [people without disabilities].” Another user who said they regularly work with young patients with cerebral palsy speculated that Enabled Play would “literally change their lives.”

Accessibility consultants are building a more inclusive video game industry behind the scenes

But the device is not limited to the gaming sphere. It is also used in schools to make computer labs more accessible. With the rise of remote work and online learning environments brought about by the pandemic, Jaipreet Virdi, a historian, author and professor at the University of Delaware, said the device could serve as a model for “inclusive participation” in schools.

“If disabled students can learn and keep up with the expected pace of education through these [assistive] technology, then that way they can graduate with more opportunities than their disabled ancestors ever had,” Virdi said.

In some therapy programs in the United States, specialists use Enabled Play to track facial expressions and gamify therapy sessions. Alissa McFall, a speech-language pathologist and orofacial myologist in Sacramento, said it can be used to analyze how a patient’s muscles are working so healthcare professionals can then use that feedback to develop customized treatment plans.

“The biggest value we’ve seen so far using the Enabled Play device is that it can be programmed to read natural communication gestures and connect each sound or facial expression to a function that is meaningful to an individual,” McFall said.

Since its launch in January, Enabled Play has partnered with a number of organizations in the gaming and assistive technology spheres, including Special Effect, Makers Making Change and – more recently – Microsoft with its Designed for Xbox accessibility partner program. Next Dunn hopes to soon roll out “virtual devices,” which would allow other developers to add Enabled Play’s inputs to their apps. These add-ons allow a person to use facial expressions and voice commands in Microsoft Word and Adobe Photoshop without purchasing a separate device.

As developers look for ways to make the technology more accessible, Dunn hopes to help drive that change and encourage others to think far beyond the typical keyboard and mouse inputs.

“It’s a very personal mission of mine to solve these problems,” he said. “That’s the difference that I’m after, which is to build devices that change the human-computer interaction paradigm to one that’s just more inclusive.”

Amanda Florian is a journalist based between the US and Shanghai. She reports on technology, culture and China’s new media scene.

#latest #video #game #controller #isnt #plastic #face

Leave a Comment

Your email address will not be published.