Dr Simone Stumpf leads on Microsoft-supported AI for Accessibility project
Funded by Microsoft Corporation, Dr Simone Stumpf is the project leader on Meta Learning for personalised object recognition aimed at visually impaired people /The ORBIT (Object Recognition for Blind Image Training) dataset.
The team includes Dr Lida Theodorou as lead researcher, Dr Toby Harris as the app developer, Ms Luisa Zintgraf from University of Oxford to provide machine learning expertise, and Drs Cecily Morrison, Ed Cutrell, Daniela Massiceti and Katja Hofmann from Microsoft Research.
There are a number of emerging assistive technologies for blind users, particularly using some form of AI or computer vision. For example, the smartphone app Seeing AI permits blind users to take a picture and recognise objects, announcing what is in the picture, e.g. 'a dog on a sofa' or '10 US dollars'.
Recognition and personalisation
However, Dr Stumpf, a Senior Lecturer in the Centre for Human-Computer Interaction Design in the School of Mathematics, Computer Science & Engineering, points out that, “current approaches, do not allow object recognition to be personalised, especially to objects of importance to blind users, for example their white cane or their favourite mug”.
The overarching aim of her team’s work is to establish a large data set of images of labelled objects that can be used to train computer vision models to personalise object recognition. To do this, the team will be studying how blind users take pictures of a personal object so that it can be recognised from a variety of angles and in a range of different contexts.
Dr Stumpf says:
An additional aspect of this project is the development of a curriculum that can be used by teachers to educate people with visual impairments about how AI works and how they can be involved in creating these novel systems and applications, including how to contribute to this very important data set. This work presents a step change for including blind users in personalised object recognition, especially as they are often left out of technology innovation.
The ORBIT project is made possible through the Microsoft AI for Accessibility grant award in the ‘Daily life’ category which “builds modern solutions for people with disabilities by making software and devices smarter and more contextually relevant.”
For further information on the ORBIT project, please visit this weblink.