The Committee's charge is to: (1) determine best practice
criteria for assessing the learning outcomes of accounting programs, (2)
identify best practices among accounting programs, and (3) prepare a
publishable report that documents our work. We expect to complete this project
near the end of the 2002–2003 academic year.
The Committee is using the four-phrase process described below to execute
its charge, and its final report will include a section that documents the
results of each phase.
1. Briefly define the outcomes assessment, review the
literature, and prepare a list of guidelines for effective assessment practices
derived from the literature. We will use the list as "best practices
criteria (BPC)." The literature review is very concise and focuses on
letting the audience know about the resources that are available.
We have completed a first draft of the literature review. Some of our
observations are presented here.
Prior research suggest that only a minority of accounting programs have
assessment programs in place and that the accreditation status of a program may
not be associated with whether a department has an assessment program. These
studies are, however, dated and do not reflect the impact of the AACSB's new
outcomes assessment initiatives on the assessment practices of accredited
programs.
Despite the potential for many different types of assessment practices to
exist, there is very little current knowledge of the assessment practices of
accounting programs. This knowledge is particularly relevant, as it can be used
to identify those practices that are more effective and/or efficient in
engaging faculty, students, administrators, and external stakeholders. It is
generally believed that failure to engage faculty, students, and administrators
can marginalize the potential benefits of assessment activities as a tool for
continuous improvement.
Based on a survey of U.S. and Canadian accounting departments, Chang and
Chow (1999) suggested that the Balanced Scorecard is potentially useful to
guide and engage the accreditation and assessment efforts of accounting
programs. There are no reports on the extent of use of the Balanced Scorecard
or other similar coherent frameworks as a framework for assessment and
strategic management of accounting programs.
The assessment literature suggest that an effective assessment program will
have the following characteristics:
a. Assessment activities focus on learning outcomes that align
with the mission, goals, and objectives of the accounting program;
b. The assessment program involves collection of quantitative and
qualitative data;
c. The data is stored in a systematic manner and can be retrieved without
undertaking a major project;
d. Data generated by the assessment program is shared and used to engage
faculty, administrators, and stakeholders (where appropriate) in discussions
about student learning within the accounting program;
e. The assessment program involves and engages more than just a single
faculty member and can be replicated at other institutions;
f. There is evidence that the assessment program has motivated (or is very
likely to motivate) improvement in accounting programs.
A highly desirable characteristic of assessment programs is that they should
flow from a strategy map (Kaplan and Norton 2000) for the accounting program
and use a comprehensive framework such as Kaplan and Norton's Balanced
Scorecard.
2. Survey program chairs at U.S. universities and colleges to learn the
extent to which departments satisfy the BPC. Data from this survey will be used
to identify roughly eight best practices schools (BPS).
We surveyed department chairs and have completed a draft report of our
findings. The survey has at least two deliverables. First, it documents current
assessment practices and provides insight into the state of the art. Second,
the survey instrument can be used as an assessment checklist by accounting
programs that are currently designing a new assessment program or evaluating
their existing assessment activities.
Our results show that the most common assessment practice reported by
respondents is success of recent graduates in securing employment (77 percent).
This is followed by professional exam pass rates (74 percent), surveys of
graduating students (71 percent), and alumni surveys (71 percent) at least
every five years. Despite observations by Chang and Chow (1999), Accounting
departments seldom use a broad framework such as the Balanced Scorecard (Kaplan
and Norton 1992, 1996, 2001a, 2001b) or a key success factor model to guide
their assessment practices.
3. Invite program chairs and faculty to write detailed reports on assessment
practices at BPS. These reports will use a common format to document the
assessment activities at these schools. The Committee's report will contain
approximately eight BPA cases.
This phase of the process is in progress. Each BPS case will contain a
strict maximum of 10 pages on an assessment activity that works for them.
Criteria for selecting BPS cases are:
a. The activity assesses a learning outcome that aligns with the
goals and objectives of the accounting program;
b. The activity involves collection of quantitative or qualitative data;
c. The data is stored in a systematic manner and can be retrieved without
undertaking a major project;
d. The data generated by the activity is shared and used to engage faculty,
administrators, and stakeholders (where appropriate) in discussions about
student learning in the accounting program;
e. The activity involves and engages more than just a single faculty member
and can be replicated at other institutions;
f. There is evidence that the assessment activity has motivated (or is
likely to motivate) improvement in accounting programs.
4. Review AACSB accreditation reports of newly accredited or reaffirmed
accounting programs (within the last two or three years) and document, using a
consistent format, their assessment practices.
The Committee has completed an initial draft of this part of its charge. In
executing this part of its charge, the Committee reviewed the AACSB
accreditation reports of accounting programs that received initial
accreditation or reaffirmation within the last two years. This review was
considered necessary in order to present in a single document the assessment
practices of programs that recently received separate accounting accreditation.
AACSB requires assessment of learning outcomes as evidence of program quality.
We assume that these assessment practices are reviewed and validated by
external teams of deans and department chairs as part of the accreditation
process. The Committee thought that an analysis of AACSB accreditation reports
would provide insight into assessment practices that work for accreditation
purposes. Furthermore, Committee members were not aware of any similar any
similar analysis and documentation of assessment practices. A preliminary list
of 77 different practices was identified in this process. The Committee grouped
these practices into six categories that cover assessment of programs,
students, curricula, faculty, resources, and external department relationships.
Thomas Calderon, Nrian Green, and Mike Harkness took the initial lead roles
in Phases 1 and 2, which are prerequisites for Phase 3. Paul Bayes, Lynda
Murray, Leslie Kren, Kathleen Sinning, Terry Ward, and Mark Mitschow worked on
phase 3 of the project. Most committee members have helped with reviewing and
commenting on work in progress, particularly the online survey instrument. Paul
Bayes and Lynda Murray are coordinating Phase 4 of the project.
The Committee will continue working on its charge during 2002–03. We
invite department chairs and faculty to submit best practice cases for review
and consideration for inclusion in our final report.
Chang, O. H., and C. W. Chow. 1999. The Balanced Scorecard: A
potential tool for supporting change and continuous improvement in accounting
education. Issues in Accounting Education 14 (August): 395-412.
Kaplan, R. S., and D. P. Norton. 1992. The Balanced Scorecard-Measures that
drive performance. Harvard Business Review 70 (Jan.-Feb.): 71-79.
________, and ________. 1996. Using the Balanced Scorecard as a strategic
management system. Harvard Business Review 74 (Jan.-Feb.): 75-85.
________, and ________. 2000. Having trouble with your strategy? Then map
it. Harvard Business Review 78 (Sept.-Oct.).
________, and ________. 2001a. Transforming the Balanced Scorecard from
performance measurement to strategic management: Part I. Accounting
Horizons 15 (March): 87-104.
________, and ________. 2001b. Transforming the Balanced Scorecard from
performance measurement to strategic management: Part II. Accounting
Horizons 15 (June): 147-160.