|
The Effects of Explanation
Type and User Involvement on Learning from and Satisfaction with Expert
Systems
Paul John Steinbart Wilton L. Accola |
| ABSTRACT:
This article reports the results of an experiment that examines how the design of an expert system affects user learning. Four versions of an expert system were built to compare the relative effectiveness of different explanation types and levels of user involvement on user learning. Subjects were randomly assigned to use an expert system that presented either rule-trace or justification explanations; one-half of the subjects, seeing each type of explanation, were also required to explicitly evaluate the system's recommendations. Subjects used the expert system to practice evaluating internal controls for a number of different scenarios. Subjects' satisfaction was measured immediately after they used the expert system. Several days later subjects were tested on their knowledge about the relationships between the presence (absence) of various control procedures and the risk of different threats to measure what they had learned through their use of the system. The
results indicate that explanation type did not affect either learning
or user satisfaction. The manipulation of user involvement also failed
to affect learning, but it did affect user satisfaction. Although all
subjects expressed satisfaction with the version of the expert system
that they used, subjects who were required to explicitly evaluate the
expert system's recommendations were less satisfied than were subjects
who only read the system's recommendations
|