Problem Space

There are nearly 285 Million blind and visually impaired users in the world. Our aim for the project is to simplify the smartphone navigation for BVI community. Audio Sword gives a new smartphone navigation system which takes input using hand gestures and gives feedback using aural cues.

My Role

As a research assistant of Dr. Davide Bolchini, I conducted user studies and participatory design sessions with blind and visually impaired users and collected data that highlights their adaptiveness towards the prototype that we designed. on the where we studied how our prototype is conceptually fit for BVI users by doing test runs of the prototype. We are currently analyzing the research data that we have collected from 10 user interviews and writing a research paper on collected insights.


  • BVI users have to use both the hands while browsing through the cellphones, thus they can't use the smartphone while working or walking on the roads as both hands need to be free.
  • Voice input and output is tedious and may violate the privacy of the user while using.
  • Mobile navigation patterns are not consistent and can be complex for BVI users.


We focused on the hierarchical menu system, as it was easier to map in mind and easier to navigate.

Introducing Audio Sword
Making Prototype

We designed the prototype using Android, MYO armband, and Bone Conduction earphones. MYO tracks the muscle movements and recognizes the gesture that is performed, the android application takes that as input and further gives feedback to the user using Bone-Conduction headphones.

Input Gestures

Myo input gestures to navigate in the samrtphone.

For moving left in the navigation menu.

For moving right in the navigation menu.

For locking and unlocking the phone.

For selecting the current option from the menu.

For going back in the menu.
Next Steps

Right now we are in user evaluation process of the prototype. Our aim is to check if the prototype that we have collected will help the blind and visually impaired community in enhancing the browsing experience or not.

Other Case Studies