We aim to see that all blind people in the world can study tactile graphics independently. We created the mobile app READER to give them a virtual assistant. The app makes use of modern technologies to turn into the specialist above the shoulder, who can now be taken anywhere at any time. Here are the steps it makes every time it helps a blind person explore a tactile graphic independently!

1. Reading the QR Code
2. Reaching for information
3. Image Recognition and Interactive Augmented Reality

1. Reading the QR Code

Any self-describing tactile graphic needs a QR Code. The code is the bridge or the link between the physical and the virtual world. At first, the mobile phone must be placed above the tactile graphic so it can read the QR Code and recognize the graphic. (If you haven’t downloaded the app already, you can find it on the App Store free of charge).

If you want to know how to add a personalized code to your tactile drawing, read this article

2. Reaching for information

Once it read the QR Code, the READER app starts searching for the image in the LIBRARY. After having identified the image, it reaches for the server to get the audio description. The file is comprised of descriptive information written by the creators of the self-describing tactile drawings. You can become a creator yourself by joining the EDITOR.

3. Image Recognition and Interactive Augmented Reality

The READER app uses modern technologies such as Image Recognition and Interactive Augmented Reality. Image Recognition algorithms give you the freedom to walk your finger on the tactile graphic wherever you want, without it having to have the QR Code in sight all the time. It only needs to read it once, at the beginning. Then, you are free to explore – you can move the phone to follow your finger, or you can place the phone on a holder.

Augmented Reality allows the app to trace your finger on the tactile graphic. With the help of this technology, the READER recognizes different parts of the image, called areas of interest, and marked with green squares in the video. If you hold your finger on an area of interest for a couple of seconds, it beeps once for letting you know you should not move your finger. Then, three beeps follow, and with them comes the audio description of the area of interest, possible with the aid of VoiceOver.

The READER app can recognize and trace your finger and then give you information about the image in question. This is what Interactive Augmented Reality is – with it, the app turns into the specialist over the shoulder.

You can help us bring a virtual assistant for the independent exploration of tactile graphics to as many blind people as possible. The READER app is right now at the Alpha stage and available only for the iPhone. Help us improve it and create the Android version as well! You can do so by donating.