An increasing number of telephone services are offered in a fully automatic way with the help of speech technology. The underlying systems, called spoken dialogue systems, provide speech recognition, speech understanding, dialogue management, and speech generation capabilities. They enable a more-or-less natural spoken interaction with the human user, which nevertheless differs from a normal telephone conversation.
This book gives a systematic overview of assessment, evaluation, and prediction methods for the quality of these innovative services. A new taxonomy of quality-of-service aspects is developed, which takes into account the behavior of the dialogue system, the interaction environment, the task offered by the service, as well as the context of use. The taxonomy serves as a tool for classifying assessment and evaluation methods, for planning and interpreting evaluation experiments, and for estimating quality on the basis of system characteristics.
Whereas the behavior of the system can be quantified in a parametric way, the perception of the user can only be quantified via interaction experiments with human test subjects. A broad overview of parameters and evaluation methods is given, both on a system-component level and for the fully integrated system. Three new experimental investigations illustrate the relationships between system characteristics and perceived quality. The resulting information is needed in all phases of system specification, design, implementation, and operation. It helps to perform assessment and evaluation experiments in an efficient way, and to set up systems which offer a high quality to their users.
Although being written from the perspective of an engineer in telecommunications, the book is an invaluable source of information for a wide audience, from experts in telecommunications and signal processing, communication acoustics, computational linguistics, speech and language sciences, up to psychophysics, human factor design and ergonomics.
