|
Introduction
The Promise of Assessment:
Resource for Change in Accounting
Education
One of the objectives of the Accounting Education Change Commission (AECC) is to increase the effectiveness of academic programs in preparing students to enter a "dynamic, complex, expanding and constantly changing profession" (AECC, Objectives, 1990, p. 1). The AECC further notes that accounting graduates must function in a global economy which is increasingly sophisticated and interdependent. The current effort to change accounting education is directed toward the achievement of this AECC objective.The AECC and the accounting community in general call for a change of focus in the philosophy of accounting education. Currently, programs are aimed at transmitting technical knowledge, while the new focus envisions helping students to develop a deep understanding of concepts and policies, skills and attitudes necessary for the successful practice of accounting. This transformation is intended to produce graduates better prepared to function within the profession by:
- Developing the knowledge, skills and values of the discipline
- Knowing and conforming to the ethics of the profession
- Being able to make better value judgments
- Maintaining a high level of integrity, objectivity, competence, and concern for public interest (Objectives, pp. 2-3)
The six major accounting firms have invested $5 million since 1989 to help advance the AECC's vision. This investment has taken the form of direct grants to accounting programs, participation in accounting publication efforts, and funding of general information programs to disseminate the changing concept of accounting education. It soon became clear that few schools or accounting programs had suitable metrics in place to measure the success of their efforts in achieving these new learning objectives. In particular, the new emphasis on lifelong learning challenged accounting educators to create new assessment methods and measures.
After visiting schools that have recently revised their curricula, one observer noted that "Dedicated academics are striving to design outstanding learning environments. Visitors can only wonder at the diversity of solutions."
This diversity raises the questions. "Is one [solution] better than others? Will the outcomes differ? Will the accounting environment in the 21st Century resemble any of these solutions?"
Continuous improvement requires continuous evaluation based on credible evidence systematically obtained. Recognizing this, the AECC:
- Encouraged grant recipients to develop and implement assessment plans
- Sponsored workshops on assessment for grant institutions
- Commissioned this guide to increase the use of assessment in program improvement
EDUCATIONAL ASSESSMENT
is the systematic collection, interpretation, and use of information about
Student Characteristics,
Educational Environments,
Learning Outcomes
and Client Satisfaction
to improve student performance and professional success.0.1 Purposes of this Guide
This guide was commissioned by the AECC to help accounting educators develop practical, effective assessment programs to provide information for curricular and instructional improvement. "Improvement" in this context is broadly defined to reflect the diverse educational environments, curricular goals, approaches to curricular change, and organizational cultures of accounting programs nationally.
The guide responds to recurring faculty questions and concerns about assessment. It has four objectives:
- To inform accounting educators about the imperatives driving the assessment movement
- To provide practical guidance for faculty in the development of methods to assess expanded learning outcomes (often referred to as outcomes) such as those advocated by the AECC, professional accounting associations, and AACSB
- To identify student characteristics and factors in the educational environment that influence those outcomes
- To illustrate the use of assessment as a tool for continuous improvement of learning outcomes and client satisfaction
The guide outlines a model for developing the department's assessment program. It suggests ways to establish assessment priorities, and outlines methods for clarifying curricular goals to facilitate curriculum development and assessment. It provides suggestions for framing questions about the curriculum and assessment, measuring improvement, and using what is learned to improve program outcomes and client satisfaction. Throughout the guide, readers will find examples from current assessment practices at AECC grant-funded schools and other institutions in the forefront of curricular change and assessment in accounting education.
0.2 Structures of the Guide
The remainder of the guide is in two parts.
Part I outlines the planning framework.
Chapter 1 puts assessment into a larger educational context.
Chapter 2 identifies key stakeholders and participants in the process.
Chapter 3 previews the model for accounting education assessment planning.Part II elaborates on this framework.
Chapter 4 discusses the process of establishing purposes and priorities for the assessment program.
Chapter 5 identifies budgeting and resource allocation issues.
Chapter 6 describes the essential process of articulating program goals, as well as clarifying the distinction between goals and objectives or performance outcomes.
Chapter 7 explains how to translate goals into measurable objectives.
Chapter 8 identifies research design topics and suggests way to initiate an assessment program.
Chapter 9 describes the development of learning outcome measurement. It is structured to mirror the three major categories of learning outcomes endorsed by the AECC:
- Expanded professional knowledge
- Improved skills (intellectual, interpersonal, communication, and ethical reasoning)
- Enhanced professional orientation (values and attitudes essential for professional integrity and lifelong learning)
This chapter provides guidance in designing more reliable performance measures. It also suggests practical methods for data collection.
Chapter 10 presents strategies to assess contributions of the educational environment.
Chapter 11 discusses reporting and use of assessment results.
Chapter 12 advocates review and improvement of the assessment process itself.
The guide concludes with a Glossary and Bibliography of assessment resources.Throughout the text, Figures summarize key points and present tools from the literature on assessment. Tables at the end of each chapter illustrate application of concepts presented in that chapter. Appendices present materials drawn from current assessment practice in accounting.
Designing an assessment program is an iterative process. Each new phase leads to insights that stimulate rethinking of earlier phases. Assessment facilitates improvement by sharpening curricular focus and academic effectiveness. It also establishes a sense of common purpose among faculty members as they come to appreciate their individual and collective contributions to the success of their programs' graduates.
0.3 Definition of Assessment
Assessment is the systematic collection, interpretation, and use of information on student characteristics, the educational environment, and learning outcomes to improve student learning and satisfaction.
This document emphasizes the comparison of expected with actual student learning outcomes, and the continuous use of process and outcome information, to identify strengths and weaknesses in students' performance and influences on the outcomes observed.
"Expected outcomes" are often stated as "performance criteria" (Chapter 9). In statistical analyses, they take into account entering student characteristics. "Actual outcomes" take into account both student characteristics and features of the educational environment to which individual students are exposed.
The assessment framework is shown below:
The mission and goals of the program, institution, and profession influence these elements (Baker and others, 1993; Gardiner, 1989). Learning outcomes, in particular, reflect these objectives, and those of the faculty and students.
Faculty regularly assess the performance of individual students, but assessment involves compiling evidence about individual student's performance from multiple perspectives to understand learning at the program level. In program-level assessment, faculty and other stakeholders clarify learning goals and set performance parameters. These criteria serve as internal benchmarks against which students' performance can be measured. Evidence from multiple sources is integrated over time to understand relationships among learning outcomes, student characteristics, and contributions of the educational environment. Information from these sources guides revision of current programs or development of new initiatives aimed at improving student learning.
0.4 Assessment: Timely, Practical Information for Educational Decision Making
Assessment adds the greatest value to educational programs when it provides current information that can be quickly integrated into the ongoing process of instruction and curriculum revision. Assessment should enable faculty to monitor students' performance and client satisfaction continuously so that program effectiveness can be enhanced. Delaying assessment until the end of a curricular "experiment" (particularly if the end is defined as the point when the first class of students graduates from a new or revised program) eliminates opportunities to improve the curriculum for students within that program. From this perspective, assessment becomes an information function for educational decision-making, much as accounting is now understood as an information function for financial decision-making (Bedford Report, 1986; Diamond and Pincus, 1994; Elliott, 1991). Assessment, therefore, encourages educators to generate new information about what and how students are learning in the current program, and what and how they should be learning from the program in the future.
Unlike traditional laboratory research, assessment is problem-oriented and field-based. Its primary purpose is to help educators improve program processes and outcomes, rather than to test theoretical knowledge. It therefore works on different principles of design from those associated with laboratory research. Realistically, formal experiments involving random assignment to experimental and control groups are seldom possible. Still, it is possible to obtain valid, useful information working within the constraints of the educational environment. This guide suggests practical ways to acquire and use such information for thoughtful and continuous program improvement.
Although this guide describes a comprehensive model for planning assessment, implementation is inevitably incremental. Once the key players and stakeholders have been identified (Chapter 2), and the basic framework for assessment is clearly understood (Chapter 3), the assessment committee should clarify the purposes and scope of the assessment (Chapter 4) and (Chapter 5), and goals of the curriculum (Chapter 6) before discussing specific measures. Program documents should be consulted to identify priorities for assessment.
Once the faculty have identified essential goals, the department should decide on a strategy for organizing the assessment. Options include:
- Focusing assessment on a particular area of the curriculum, such as communication skills or proficiency in using technological information
- Pilot testing a particular approach to assessment, for example compiling portfolios of students' work, or establishing a database to track students' progress and identify contributing factors
- Targeting a specific student population to study closely
In any case, the assessment program should be consistent with the interest, culture, and resources of the department and school.
In the long run, the value of assessment is not to provide numerical indicators of success or failure, nor to design the definitive curricular experiment. Its value is to create a "culture of evidence" (Wolff, 1990) that prompts the institution to ask questions, questions that lead to the development of information to improve the learning outcomes for all entering students. The goal is to develop an inquiring spirit into questions about the program that translate into action research projects and ongoing commitment to self-scrutiny.
In this culture, assessment is a tool that will help accounting educators create educational environments for the 21st century, based on evidence of what works for the students they are responsible to educate.
