Ground-breaking software allows users to control their PC through their webcam using face and body movements.
Hero image: UCL Computer Science student Alexandros Theofanou. Credit: Intel
The software could revolutionise the way that millions of people use computers by allowing those with mobility issues to easily interact with their PCs without the need to buy adapted computers and use pointer devices.
It has already been endorsed by charities including the International Alliance of ALS/MND Associations, which supports people with Motor Neurone Disease (MND) around the world and is making a positive impact on people’s daily lives.
Former elite gymnast Paris Baker, who won medals for Great Britain at the European and World Championships, was diagnosed with MND at the age of just 26. Now aged 31 and living with the disease, she said after becoming one of the first users of the software: “The potential for MotionInput to change lives is limitless.”
Called ‘UCL MotionInput Version 3’, the software removes the need for a keyboard or mouse, allowing users to interact with their computer by moving their hands, fingers, head, eyes, nose, eyebrows, mouth or full body in front of the webcam. They can also simultaneously give spoken commands and dictate text, for example to fill in a website form, captured by their computer microphone.
Professor Dean Mohamedally (UCL Computer Science), the academic who led the project and tasked the team of 54 students with developing the software, said: “We initially worked on the software in response to the Covid-19 pandemic as a means of supporting NHS frontline staff. We wanted a way to help prevent the spread of viruses and other bugs by enabling medical staff to use communal computers without touching the keyboard or mouse, but the applications are much further reaching than that.”
“It is unique in that it can be used on the majority of Windows PCs and doesn’t require any specific equipment other than the webcam and microphone, which most laptops already have, meaning there are very few barriers to access. People don’t need pointers, or a mouse, to move a cursor. They can simply raise an eyebrow or open their mouth to click and use their eyes to direct the cursor.
“I’m incredibly proud of what the students have achieved with colleagues in industry and the real-life, positive impact this is already having on people with mobility issues and those working in high-risk environments. The software is still maturing, but with a supportive community collaborating with us, we hope to see it flourish with more user needs met, making computing more accessible and enjoyable to millions.”
UCL academics and students have been developing the software for over two years, also in collaboration with Great Ormond Street Children’s Hospital (GOSH) and UCLH Institute for Child Health.
Two Computer Science students, Sinead V Tattan and Carmen Meinson, currently in their final and second years respectively, led the team of 54 students who had volunteered to work on developing the software, under the guidance of Professor Mohamedally and Professor Graham Roberts. Carmen Meinson focused on highly optimising the software architecture whilst Sinead Tattan led on development.
Sinead Tattan said: “Professor Mohamedally tasked us with creating something that would really make a difference in people’s lives – an application that could give those with fine motor skill conditions opportunities that typically abled people take for granted. The software uses and combines the latest open-source machine learning, natural language processing and computer vision software from globally recognised tech firms like Intel and Google, in a way that hasn’t been done before."
The student leads were guided by clinical lead and PhD candidate Sheena Visram (UCL Interaction Centre, UCLIC) and Dr Atia Rafiq, an NHS GP and UCL Computer Science honorary lecturer. They provided valuable insight into the humanitarian need for touchless computing software for those with mobility issues as well as medical staff.
Dr Rafiq said: "Giving patients who would otherwise rely on others the power and independence to navigate a computer and hence other worlds of shopping, gaming, access to services and every other freedom we take for granted is wonderful; and to do this with no expensive hardware or complicated gadgets is the icing on the cake. This, coupled with the big tech firms' commitment, like Intel, Microsoft and IBM, and university-driven motivation, can take accessibility to a whole new level."
Bernice You (General Manager, Strategy & Projects, Microsoft) said: “With technology we can empower everyone and make the impossible possible. Love the potential and possibilities that MotionInput brings to people with disabilities.”
Cathy Cummings (Executive Director, The International Alliance of ALS/MND Associations) said: “MotionInput improves quality of life and maintains independence and dignity by making things that matter accessible.”
As well as its clinical uses and for those with mobility issues, the software can also be used for teaching and presenting in schools, universities and businesses, and by people playing Windows computer games.
The software is freely available to download for individual users, and the team hope to see it widely rolled out across NHS sites as well as schools and other industries looking to invest in the technology. The team are also keen to connect with user groups with various accessibility and education requirements in supporting equitable computing access for all.
Source: UCL news release