People with aphasia, a condition that impairs the ability to understand or generate written or spoken language, are aided by assistive technology that helps them communicate through a vocabulary of icons. These systems are akin to language translation systems, translating icon arrangements into spoken or written language and vice versa. However, these icon-based systems have little vocabulary breadth or depth, making it difficult for people with aphasia to apply their usage to multiple real-world situations.
In the course of a day, an active individual has many reasons to communicate: reading a newspaper, visiting a doctor, shopping, or debating sports. Daily communication not only enriches peoples’ lives but also expands their vocabulary and enhances their language skills.
Icons are traditionally used in these assistive technologies because of their simplicity and clarity. Because they are painstakingly designed and engineered, icon-based systems remain unable to handle a large number of topics to the depth that a user might want. For example, the current icon-based systems are not flexible enough to handle the ever-changing topics of politics nor deep enough to accommodate the local and regional terms that appear in newspapers. It is also hard for other people to use these structured collections of icons to interact with people with aphasia; a caregiver may want to introduce a new medicine, but find it impossible to describe the medicine with existing icons and therefore need to add new icons to the system. This can lead to a frustrating and slow exchange as caregivers first search for desired icons and then create and add new ones.
The abundance and diversity of web images guarantee that a unique one can be found that captures the specific entity that is wanted for an image-based communication. However, compared to the icons which were carefully crafted to support communication, web images have inherent shortcomings. Their complex content may evoke several meanings, and different people might interpret them differently. Their quality and resolution vary significantly, and relatively few are labeled meaningfully.
To investigate this use of images, we conducted two studies to explore whether web images are as effective as a set of icons created specifically for people with aphasia. We concentrated on an aphasic individual’s perception of the pictorial representations instead of how they would use such representations in communicating, because, in practice, caregivers or family members involved in the care of the patients are expected to be the individuals who will select the images from the web and add them to the aphasic’s assistive language system.
Perception of Use of Icons
Icons are a type of practical, minimalist art. When the task of building visual vocabularies was one of simply moving from concept to image, with no ready-made alternatives available, they were a logical choice for presenting concepts. Icons retain contours, which are important for the perception of shapes but omit visual cues such as texture and color that are not as relevant to interpretation. Research has shown that line drawings depict the essentials of size, shape, and location of their subjects. It consists of a set of black and white symbols drawn on index cards. Each symbol represents a meaningful concept that a person with aphasia might wish to express.
Perception of Use of Images
Images are realistic and provide more interpretation cues than icons. Image attributes such as color, contrast, and segmentation influence the perception of the size and depth of an object. The distinguishing property of many real-world objects is also their distinctive color (e.g. lemon vs. lime). Likewise, the texture is useful in visualizing a surface; orientation, density, contrast, and size of the texture pattern are effective cues for recognizing the image. The presence of color, texture and visible discontinuities help observers identify basic features of the viewed image such as shape, orientation in space, light source, and size without the need for training or written annotation. Images are used to help people cope with memory, language, or speech deficiencies that accompany aging. Investigations show that communication is enhanced by pictures, especially realistic pictures, not only for people functioning at normal cognitive levels but also for people with cognitive degeneration or impairment. using images for communication can lead to significant improvements in learning for people with cognitive disabilities. Research, as well as anecdotal evidence, also shows that people with aphasia already use photographs to share memories, experiences, and information with friends, family, and strangers.
