|
Chapter
9
Measuring Learning Outcomes
This chapter describes the development of measures to assess the learning outcomes of the curriculum. Specifically the chapter:
- Identifies criteria for measuring program-level outcomes
- Describes procedures for the design and validation of learning outcome measures
- Describes and illustrates methods for assessing expanded learning outcomes in three categories identified by advocates of change in accounting education:
- Knowledge (including the ability to adapt and apply knowledge in new ways)
- Skills (intellectual, interpersonal, communication, and ethical reasoning)
- Professional orientation (values and attitudes)
A fundamental premise of the Objectives is that "To attain and maintain the status of a professional accountant requires continual learning" (p. 1). The curriculum should help students learn how to learn, building on three major components: knowledge, skills, and professional orientation. Assessment of each of these categories is discussed separately in the sections that follow. In addition, assessment of "learning to learn" is discussed with attention to both the skills and attitudes believed to play a part in fostering lifelong learning.
In preparing to examine learning outcomes at the program level, it is useful to keep in mind some key differences between "classroom-level" and "program-level" measures of learning outcomes. In the classroom, many measures of different outcomes are used to make global judgments about the achievements and relative ranking of individual students within a subject area. At the program level, one or more measures of a single outcome or family of outcomes may be used to make judgments about the success of the program. Measures used for program-level assessment are likely to be developed collaboratively and systematically by teams of faculty rather than by individuals.
A further difference is that program measures are expected to demonstrate degrees of reliability and validity not usually associated with classroom measures. Both classroom- and program-level measurement should, however, be held to rigorous standards because important decisions depend on the results they yield.
Finally, new expectations arising from changes in the accounting profession suggest that program-level assessments will increasingly emphasize use of knowledge and skills of learning to learn, a trend which will, in turn, affect assessment at the classroom level.
In spite of their differences, classroom- and program-level measurement should be coherently related and in some cases even intertwined as in the case of portfolio assessment or other course-embedded approaches. Measurement at both levels should address the goals, objectives, and performance criteria established by the faculty (as described in Chapters 6 and 7). These essential outcomes, which shape and direct the curriculum and the design of outcome measures, should be rooted in the instructional materials used by faculty (syllabi, examinations, projects, cases, problems). Some program-level measures may therefore be derived from materials faculty use in their courses. Conversely, development of outcome measures may lead faculty to clarify and refine the goals and objectives, with corresponding changes in instructional materials, methods, or even curricular structure.
Commonly used methods for assessing knowledge and skills (cognitive outcomes) and attitudes and values (affective outcomes) are listed in Figure 9.1.
FIGURE 9.1
Assessing Knowledge and Skills
- Tests and examinations:
- Scores on in-class examinations
- Standardized or locally developed achievement tests
- Oral examinations
- Course grades
- Grade-point averages
- Faculty ratings
- Student self-reports
- Learning logs (question/connect/apply)
- Performance measures and simulations
- Portfolios (collected work of students over time)
- Internship supervisor/employer evaluations
- Assessment center method
Assessing Values and Attitudes
- Questionnaires and rating scales
- Observation/reflection logs
- Focus groups
- Unobtrusive measures (for example, participation rates, choice behavior)
- Performance simulations
- Research instruments
Section 9.1: Basic Principles and Procedures for Designing Learning Outcome Measures
Section 9.2: Measuring Knowledge Outcomes
Section 9.3: Measuring Skill Outcomes
Section 9.4: Measuring Professional Orientation: Values and Attitudes
Section 9.5: Learning to Learn: Integrating Knowledge, Skills, and Professional Orientation
Section 9.6: Conclusion: The Challenges of Measuring Learning Outcomes
