Helen Petrie(1), Sarah Morley(1), Gerhard Weber(2).
The increasing use of graphical user interfaces is making computer systems more,
rather than less difficult to interact with for blind users. One solution is to create
interfaces for blind users which are based on auditory and tactile information rather than
visual information. The GUIB Project is developing such an interface making use of a
number of different information sources: synthetic and digitised speech, non-speech
sounds, braille and a touchpad. This video illustrates how the GUIB interface allows
users to directly manipulate interface objects using a touchpad and braille display with
cursor routing buttons. This gives blind users the same sense of engagement with the
interface which sighted users gain from using a mouse.
users with special needs, blind users, alternative interaction methods.
For blind users, the development of inexpensive machines with command-line
interfaces has allowed them to use computers very successfully. A one line command
maps easily to synthetic speech or a braille display for the output medium, and blind
users are proficient with standard keyboards for input. Screenreading software is also
required which produces the screen text in either synthetic speech or braille and
provides functionality to explore and review the screen contents without actually
executing commands.
The increased popularity of graphical user interfaces (GUIs) may have made computing
easier for many sighted users, but for their blind colleagues this development has been a
drastic step backwards. Many blind users and sighted experts believe that the
interaction techniques and visual metaphors will make GUIs extremely difficult to
access effectively for this group of users.
One of the great advantages of GUIs for sighted users is the ability to directly
manipulate interface objects. How can we create interfaces for blind users in which
direct manipulation can be carried out substituting auditory and tactile information for
the visual information available to sighted users?
One of the problems in creating alternative interfaces is that the characteristics of visual
information are very different from those of auditory or tactile information. Visual
information gives us a rich, spatially organised representation of a scene. It allows us
easy access to both an overview or to specific details. Tactile information is also spatial
in representation, giving us a detailed knowledge of the world at our fingertips, but is
unable to provide the same overview of a scene obtained from a visual glance.
Auditory information on the other hand, has a more temporal organisation, and we can
only attend to a limited number of sounds simultaneously. These different
characteristics of sensory modalities need to be considered in developing intuitive and
efficient non-visual interfaces for blind users.
The overall aim of the GUIB Project (Graphical User Interfaces for Blind Persons) is to
develop multi-modal interfaces for blind users which will give the same types of
advantages which sighted users have already experienced with GUIs. In particular we
want to create the sense of engagement with a system which comes from the direct
manipulation of interface objects. To do this, users need certain kinds of information
about the nature of the visual interface, for example the spatial layout of the interface.
Thus in the GUIB interface this information is preserved rather than presenting the
interface in an alternative form such as tree structure. In the project, we are
investigating how the visual information can be most effectively translated into a
number of different auditory and tactile forms, in order to cope with the richness of the
original visual display. We are exploring the use of synthetic and digitised speech, non-
speech sounds, both natural and abstract, musical sounds, and braille and a touch-pad
to produce a multi-modal display to replace the visual one.
In order to give the blind users access to the information in a GUI, the bit mapped
visual presentation must be converted into a form with which blind users can interact.
To do this, the GUIB project has developed an "Off Screen Model" (OSM), a
representation mechanism which describes interaction objects and the dynamic changes
in these objects. This is used to generate a text-based presentation suitable for non-
visual input and output. Graphical objects are represented with ASCII characters such
as brackets and text labels. It can also produce a visual form of the presentation to
enable successful collaboration between blind and sighted users, for example at work.
Two OSMs have been developed, one for Microsoft Windows 3.1 and one for X
Windows.
In the video we illustrate how the GUIB interface attempts to create direct manipulation
using two types of tactile interaction. Firstly, the touch-pad gives the blind user the
ability to obtain a spatial overview of the screen similar to the visual glance at the screen
by the sighted user. Touching the pad lightly produces speech output, informing the
user of the interaction object at that location on the screen. Pressing more firmly
emulates moving the mouse to that object, which can then be activated with a key just
below the touchpad.
DYNAMIC FIGURE 2 (QuickTime Movie, about 4 mb)
Direct manipulation at a much finer level of detail can be achieved using the braille
display. A braille module comprises 8 independent pins in a 2 by 4 matrix which
displays an ASCII character by moving the pins up and down. There is also a small
button above each braille module, and pressing this button moves the cursor to this
location, a process known as cursor routing. The GUIB braille display includes of 2
rows of 80 braille modules which can display two complete lines of the text-based
presentation.
A fundamental difference between exploring the display via a visual screen and a braille
display is that whereas the sighted user may move the mouse around the screen freely,
the blind user can only move the mouse cursor in discrete steps by clicking on different
routing buttons. For example in Microsoft Windows, to move an inactive window, the
sighted user first moves the mouse into the window and clicks once, making the
window active. The blind user must click twice on the routing button, once to move the
cursor into the window title and again to make the window active.
DYNAMIC FIGURE 4 (QuickTime Movie, about 2 mb)
To pick up the window and move it, the sighted user presses the mouse button and
holds it down while moving the mouse to the desired destination. She then release the
button which drops the window. The blind user executes the pick up by doubled
clicking on any other routing button. To move the window she then scrolls down the
display and double clicks on the destination routing button to drop it.
By these actions blind users gain a sense of directly manipulating the window in a
manner similar to manipulating it with a mouse.
DYNAMIC FIGURE 5 (QuickTime Movie, about 2 mb)
These forms of interaction have been assessed along with other components of the
GUIB interface in a systematic evaluation. The evaluation uses standard sets of tasks
based on common word processing activities. Sixteen blind users have each
undertaken eight hours of training in GUI concepts, the GUIB interface and evaluation.
Extensive objective measures such as time to complete tasks and error rates, and
subjective measures such as ratings of the components are currently being analysed.
For example, users were asked to rate the different interaction methods available,
including different combinations of input methods. They rated input from the keyboard
the most favourably (mean rating = 6.67, on a scale from 1 = least acceptable to 7 =
most acceptable), probably because they are already familiar with this method and it is a
very reliable means of interaction. The touchpad was not rated as highly (mean rating =
2.50), perhaps because users often found it difficult to locate individual interaction
objects on the small touchpad. Further design work needs to be undertaken to try to
improve its usability.
The routing buttons were also very highly rated (mean rating = 6.00) and many positive
comments were made about this method of interaction. For example, one user said that
he liked using the routing keys in dialogue boxes as gave him a real hands-on feeling.
Several other users commented on how efficient and simple the routing buttons were to
use. These comments are similar to those expressed by sighted users about direct
manipulation techniques and indicate that blind users are achieving the same sense of
engagement with the system.
Other work in the GUIB Project is complementing the current research on direct
manipulation. We are investigating the potential for a number of novel interaction
devices, the use of three dimensional speech and sound output to create a virtual "sound
display" and the use of non-speech sounds, both natural and abstract to enhance the
interface.
In conclusion, the GUIB Project has shown that there is reason to be optimistic about
the potential of accessing GUIs by blind users. With appropriate training, particularly
in the metaphors used in GUIs, and suitable interaction methods based on auditory and
tactile information, blind users may well be able to have at least some of the advantages
of GUIs currently enjoyed by sighted users.
The GUIB Project has been supported by the Pilot and Bridge Phases of the TIDE
Programme (Technology Initiative for Disabled and Elderly People) of DG XIII of the
Commission of the European Union. We are grateful for the work and support of all
the members of the GUIB Consortium. We are also very grateful to John Bain and
Julia Pontin for their assistance in the production of the video.
Abstract
Keywords:
ACKNOWLEDGEMENTS