Digital Reading System Assists Vision Impaired to Read Graphs

Introduction

People who are blind can now read more than just words, such as graphs and graphics, following the development of an affordable digital reading system by Curtin University researchers.

Main Item

Opening up new career paths and educational opportunities for people with vision impairment, the system combines a number of pattern recognition technologies into a single platform and, for the first time, allows mathematics and graphical material to be extracted and described without sighted intervention.

Senior Lecturer Dr Iain Murray and PhD student Azadeh Nazemi of Curtin's Department of Electrical and Computing Engineering developed the device to handle the extraordinary number of complex issues faced by the vision impaired when needing to read graphics, graphs, bills, bank statements and more.

Continued below image.

Continued...

To develop the device, the team has made use of a number of technologies, mostly based on pattern recognition, machine learning and various segmentation methods.

"Many of us take for granted the number of graphics and statistics we see in our daily lives, especially at work. We love to have graphics and diagrams to convey information, for example, look at how many statistics and graphs are used in the sports section of the newspaper," Dr Murray said.

"People who are blind are often blocked from certain career paths and educational opportunities where graphs or graphics play a strong role. We hope this device will open up new opportunities for people with vision impairment - it's a matter of providing more independence, and not having to rely on sighted assistance to be able to read graphical and mathematical material."

The device works by using pattern recognition technology and other methods on any document to identify images, graphs, maths or text. From here it is then converted to audio format with navigation markup. Basically, the system takes a document, such as a pdf, bill, or scanned document, identifies blocks of text or pictures, segments these into related blocks and arranges these blocks in the correct reading order. Blocks are then identified as images, graphs, maths or text and recognized via optical character recognition or the utility for maths, Mathspeak. It is then converted to audio format with navigation markup.

The device is 20cm long, 15cm wide and 3cm thick. The controls are very much like a cassette player with a couple of additions for navigating through headings or chapters. Books can be downloaded or posted out on USB storage devices. These books are in a specific format that allows audio playback with navigation markup, with audio either in synthetic speech or human read. The device will have high contrast keys with tactile markings.

Dr Murray said the system runs on very inexpensive platforms, with an expected production cost as low as $100 per device, allowing it to be affordable to many people around the world and hopefully make a difference in third world countries.

He said previously there have been many methods to convert graphical material but all are very labor intensive and generally not easily transferable to other users.

"Our system is easily operated by people of all ages and abilities and it is open source, meaning anyone with the skill can use and modify the software to suit their application," Dr Murray said.

The player has built-in user instructions and a speech engine that converts to more than 120 different languages.

Dr Murray said he was now looking for philanthropic finance to set up production.