Sonification refers to the usage of non- speech audio to represent data by mapping parameters to sound. Using sonification, it is possible to listen to data, rather than merely visualize it. Earlier approaches have attempted to sonify images for data representation purposes and as an aid to the visually impaired, but have certain inefficiencies when used as an aid, which may be reduced or eliminated.
Common issues pertaining to sonification applications include usage of complex datasets and the subsequent processing overhead involved, maintaining image characteristics while attempting compression for storage and transmission, selection of parameters for mapping to an audio output, among others.
This model attempts to address some issues in designing sonification applications for the visually impaired. We deal with possible data set reduction by using image brightness levels instead of RGB bit depth, user defined data scaling for choice of image detail, use of melodic sounds to reduce mental fatigue when listening and enhance user comfort in comparison to non melodic sonification --which uses frequencies and tones in the audio range that are not necessarily aesthetically pleasing, hence leading to a degraded long term user experience.
This model aims to serve as an auditory aid to the viewing of image data,which is done by segmentation and conversion of the image to a text file for easy storage, representation, and processing of data.We perform systematic image compression and melodic sonification by mapping brightness levels of an image to musical notes in the A harmonic minor scale. This scale contains tonally stable notes according to the tonal heirarchy of notes as applied to human hearing. Ten characters are used to map the 256 levels of brightness in a grayscale image which is obtained by filtering an image from its hue, or RGB levels.We then segment the grayscale image according to user input to generate an ASCII text file and represent its important characteristics (pertaining to auditory perception) using tone-based ASCII art. Finally, we use this compressed version of the image containing its relevant characteristics and perform subsequent melodic sonification to represent the text file through musical tones.