Big Meaning in the Smallest Movements

Article by Robin Berghaus appeared on BU Today and Bostonia magazine.
Above: Chris Kwan demonstrates his Camera Canvas software to Corey Petitt. Photo by Cydney Scott

Born with spastic quadriplegia and cerebral palsy, 49-year-old Rick Hoyt has never been able to speak or use his hands to write. But that doesn’t mean he can’t communicate.

Hoyt, who is best known for competing in more than 1,000 races, pushed in a wheelchair by his father, Dick Hoyt, is testing assistive technology developed through a decade-long collaboration between Margrit Betke and James Gips, professors of computer science, with help from more than 50 students.

Photo courtesy Team Hoyt

Camera Mouse, a tool for people with medical conditions such as cerebral palsy, spinal muscular atrophy, ALS, multiple sclerosis, and traumatic brain injury, employs a computer webcam to lock onto and track a chosen section of the user’s face—a nostril or the tip of an eyebrow, for example—and then links that person’s head movement to a cursor on the screen. Move right and the cursor goes right. If Hoyt pauses for more than one second, dwelling over a button or link, it clicks to active.

Betke and fellow researchers have adapted Camera Mouse to work with several popular programs, such as Microsoft Word. They’ve also created custom software that allows computer users with disabilities to type emails, edit photographs, create music, and fight space aliens, among other activities.

Now two graduate students in Betke’s lab, Chris Kwan and John Magee, have crafted companion software that piggybacks on Camera Mouse’s underlying technology. While testing Camera Mouse with Hoyt, Magee noticed that Hoyt moved his head more easily in a diagonal motion—a trend Magee sees in other people with motion impairments—making it difficult for Hoyt to propel the mouse pointer in a straight line. So Magee built an adaptive mouse functions software that would translate Hoyt’s actual movement into his intended movement, sending the mouse pointer straight across the screen and helping Hoyt to more easily hit his target commands.

Magee hopes to finish developing adaptive mouse functions and release it free on

John Magee and Chris Kwan. Photo by Cydney Scott

Hoyt has also tested Camera Canvas, software custom-built by Kwan, which received the 2011 Science Day Graduate School of Arts & Sciences Dean’s Award and is already free online. The software allows Camera Mouse users to edit photos and create illustrations, performing tasks that require physically complex mouse moves with traditional software.

Kwan, who began developing the software with classmates Won-Beom Kim and Igor Fedyuk, and further refined it through the University’s Undergraduate Research Opportunities Program (UROP), hopes Camera Canvas will give people with motor impairments an outlet to express their creativity.

Working with co-creator Isaac Paquette, Kwan also developed a tool within Camera Canvas that could have much broader applications. The two developers created a menu whose settings, like button size and screen placement, can be modified based on users’ abilities. Called Menu Controller, it eliminates the need for pull-down menus with small, crowded entries that can be difficult to navigate with Camera Mouse.

Margrit Betke. Photo by Mira Whiting

Menu Controller “is an important development,” Betke says, “because it can be applied to any commercially available software to convert it into a piece of software that can then be used by people with severe motion disabilities.”

Camera Mouse has been downloaded more than 350,000 times worldwide, from as far away as Greece, South Africa, India, Northern Ireland, Pakistan, Australia, and Malaysia. Magee believes this community will continue to grow as more software is built for use with Camera Mouse.

“These projects have the possibility to be used by a small group of people,” he says. “But the change that it can make in their lives, if you do a good job, is huge.”


Credits | Robin Berghaus

Additional Credit
Still photography by Cydney Scott, Mira Whiting and Team Hoyt

Boston University students and faculty build software that empowers users with severe motion impairments to type emails, edit photographs, create music, and fight space aliens, among other activities.