|
Chapter
8
MEASURING AND ASSESSING CHANGE
From its first meeting, the AECC was concerned with measuring the impacts of educational changes. In fact, two task forces addressed measurement. One task force focused on how to measure the impact of program changes on the skills and capabilities of students, and the other focused on measuring the role of the Commission itself in bringing about changes in accounting programs.Measuring Changes in Student Capabilities
Changes in educational processes are often accepted simply on the basis of logic and ex ante reasoning. The Commission wanted more evidence that changes being undertaken in accounting programs truly generated increased capabilities in students. This concern was evident in many of the Commission's activities. Grant proposals were required to specify plans for measuring the impacts of program revisions. Such assessment plans were thoroughly discussed in evaluating proposals. Several meetings were held so that representatives of the grant schools could compare notes on their assessment plans and activities. For one of these meetings, all grant recipients were asked to write a two-page document on the methods used (or proposed) to measure outcomes. Site visits by the AECC liaisons included discussions of the measurement of outcomes. Nevertheless, outcome assessment remained an elusive topic.
The need for assessment was not unique to accounting programs. Educational institutions had focused for years on measuring inputs; measuring outputs, and especially outcomes, did not come easily. Accountability and related assessment was a rallying cry in all of education, from kindergarten to graduate programs. Whether imposed by government agencies or initiated internally, nearly all educational institutions were undertaking some kind of assessment. The Commission believed that accountants, with their expertise in measurement, could lead the way in educational assessment. Therefore, the AECC undertook its own project in assessment.
A well-received monograph was the result: Assessment for the New Curriculum: A Guide for Professional Accounting Programs, by Joanne Gainen and Paul Locatelli. Published in 1995, this monograph provides background on the assessment movement in the United States. It outlines a model for developing an assessment program and provides guidance for faculty to assess not only the traditional learning outcomes, but also the expanded learning outcomes advocated by the AECC and others. It also illustrates the use of assessment as a tool for continuous improvement of learning outcomes and client satisfaction.
The Process of Change
The process of change is seldom smooth. The accounting education changes in the 1990s were no exception. The process of change undertaken by the Commission had three main phases: (1) create acceptance of the need for change, (2) develop the direction of change and the means of achieving it, and (3) implement the changes throughout accounting programs in the United States. I don't think this three-pronged approach was ever explicit in the strategy of the Commission, but it is a logical way to summarize the process. All three phases began in the early stages of the Commission's life, and they were accomplished nearly sequentially.
The Commission did not have to convince faculty of the need for change as much as to emphasize its urgency. As mentioned earlier, changes in accounting programs had been discussed for years. In the 1980s it became clear that something needed to be done. The Commission focused on coalescing opinions on what changes to make and the necessity to make them sooner rather than later.
A factor that made change especially difficult was the magnitude of the required change. Accounting education flourished in the 1970s and early 1980s. Accounting programs kept attracting more and better students. The marketplace seemed to have an insatiable demand for accounting graduates. Some years were better than others were, but the over-all trend was always positive. Programs were growing and external funding was steadily increasing. Accounting programs seemed isolated from the profession's rocky roads of litigation and increasing competition. The prevailing view in accounting academe, though seldom explicitly articulated, seemed to be "don't rock the boat."
Accounting faculty were not blind to the need to continually evolve their programs. In fact the 1970s and 1980s were times of great change in the accounting research programs in most schools. A growing number of colleges and universities instituted research and publication requirements for their faculty, and these requirements seemed to be continually increasing. Unfortunately, much of the research program changes did not affect the teaching programs. Contrary to the synergy between teaching and research that drove the development of the research universities after World War II, the growth of research in accounting seemed to be parallel to, not integrated with, teaching. Nevertheless, accounting departments were quite different in 1990 from what they were in 1970.
At the same time, certain segments of the accounting curriculum also underwent major changes. The teaching of auditing is a major example, moving in most universities from a focus on how to do an audit to an understanding of the audit process. A similar move to a more conceptual orientation took place in many tax courses. However, the core of the accounting curriculum-financial and management accounting-experienced little change.
The Commission interpreted the slow pace of change in accounting curricula as a sign of complacency in the accounting academy. A majority of the Commission felt that a bold step was necessary to break through this complacency. This led to the most controversial action taken by the Commission, the issuance of Issues Statement No. 1, AECC Urges Priority for Teaching in Higher Education. The Statement upset some members of the accounting academic community-an outcome that was both good and bad. A wake-up call was needed, and the Statement grabbed the attention of most faculty. However, by antagonizing a segment of the accounting academic community that could have been leaders in developing the needed changes, it also hindered the process of change.
While Issues Statement No. 1 was an attention-grabber, a more grass-roots effort, which involved speaking at a large number of conferences and universities and publishing various articles, was also an important part of selling the need for change. A combination of Commission efforts and market forces led to widespread acceptance of the need to change by the early 1990s.
The direction of change was set primarily by the Commission's Position Statement No. One, Objectives of Education for Accountants. Methods of change were developed primarily by the grant schools. Yet, it is clear that the Commission did not have a monopoly on ideas for and models of change. Many colleges and universities proceeded with change activities independent of AECC support. Some received support elsewhere. For example, Coopers & Lybrand supported changes at the University of Southern California, and California State University, Chico received government funding from FIPSE (Fund for the Improvement of Postsecondary Education). But the majority tackled change activities with no outside funding. Some of these efforts were described in the special AECC issues of Accounting Education News, but the majority were simply program improvements that did not receive notice outside the college or university implementing them.
Would models of change have arisen without the AECC? Probably many of them would have. But the Commission created projects that had an explicit obligation to widely share their change experiences with others. In addition, the Commission helped create an atmosphere where curriculum change and experimentation were viewed more positively than previously. Finally, the Commission provided forums for sharing experiences, speeding the process of learning what might work and what might not work.
The final phase, widespread implementation of changes, is an ongoing process. Few colleges and universities have not made some changes in their curricula. But so far, the widespread changes have been primarily those that were easy to accomplish rather than those with the greatest long-term benefit. Pedagogy has changed more than content. I believe that the success of the AAA's faculty and program development activities will play a major role in the successful implementation of the more fundamental changes needed. Changing faculty capabilities and their comfort with new approaches to the teaching of accounting is essential to implementing the changes. While the AECC set the stage for this phase, its accomplishment is beyond the Commission's purview.
One benchmark for judging the widespread implementation of AECC-supported changes is changes in textbooks. On the surface, the Commission has had a noticeable affect on accounting textbooks. Almost every introductory accounting text has a section in the preface telling how it meets AECC suggestions. End-of-chapter materials invariably include problems, exercises, and cases designed to foster problem-solving, communication, and team-building skills. But, like most accounting programs, most textbooks have addressed the easy changes and have ignored the more fundamental ones. Some texts with significantly different approaches-most consistent with AECC objectives-have been published, but they generally have not made major inroads in the marketplace.
One positive factor from the textbook market is revealed in Sullivan and Benke (1997). They categorized introductory financial accounting textbooks from traditional to revolutionary (where revolutionary books are generally more consistent with AECC objectives), and then asked authors to comment on the categorization. Most authors did not agree with their categorization, and all but one author thought his or her book should be ranked further toward the revolutionary end of the scale. From this, Sullivan and Benke (1997, 199) concluded, "authors are trying to move toward the Revolutionary category." If this attitude carries over into their next revisions, textbooks will gradually incorporate more of the Commission's suggestions, and the entire accounting textbook market will move in that direction. On the other hand, if authors already believe that their text is more revolutionary than it is, maybe they will not see the need to move further in that direction.
Resistance to Change
The process of change is not easy. Often those with the most at stake are the most resistant to changes. This was the case with accounting education changes. Faculty, students, and parents all had reasons for opposing change. Change agents need to anticipate this opposition and develop strategies to deal with it. Lessons on resistance to change learned primarily from the experience of the grant schools were presented to the Commission by a committee of the Federation of Schools of Accountancy in August 1993. A related paper (Pincus et al. 1993) reported the results. I will summarize only the main points in this section.
Faculty were the most threatened by the changes. Among the reasons for their resistance are:
- They do not agree with the reasons for change.
- Reluctance to change programs that have worked well in the past.
- Mixed signals from practice on what is desired.
- Changes will leave students less well prepared for CPA examination.
- They fear an inability to succeed in a changed environment.
- Loss of "ownership" of classes.
- Skills, such as dynamic lecturing, may not be rewarded in the future.
- Feel incompetent to teach broader skills.8
- They resist the great effort required to change.
- Familiar textbooks will be replaced with new ones.
- Curricular reform takes much effort and offers little reward.
- University approval processes are long and time-consuming.
- Lack of administrative and resource support for changes.
- They wish to avoid conflict with faculty with opposing views.
- Integrated curriculum requires agreement across faculty.
- Lack of cooperation of faculty outside of accounting.
Student concerns centered on how the changes would affect their chances of success in accounting courses and in an accounting career:
- Change brings uncertainty in expectations.
- Unknown level of difficulty in the new curriculum.
- New program differs greatly from expectations developed in high school accounting classes and from previous accounting students.
- Learning in teams conflicts with the methods learned to compete against each other.
- Rumors bring fear of the unknown, especially if some faculty express reservations about the changes.
- Materials are more varied and are not always available in standard (textbook) format.
- The fear of uncertainty is especially great in grading.
- Grading on communications and team exercises adds subjectivity.
- Loss of control over grades as expectations change.
- Textbooks do not provide a boundary around what is expected to be learned.
- The fear that they will be less prepared to succeed in the accounting profession.
- Not be technically competent to pass the CPA (or CMA or CIA) examination.
- Mixed signals from recruiters about the new curriculum.
Student resistance to change was especially strong when a pilot program was tested. The students in the pilot program were often concerned that they were not learning the traditional materials that their peers in the regular curriculum learned. Those in the traditional curriculum felt they were not getting as much attention as were those in the pilot program. This resulted in unhappy students in both groups.
The good thing about student resistance is that it generally disappears after about three years of experience with the revised curriculum. Students turn over every year, so in a short period of time the collective student body has no knowledge about how the program used to be. The "new" curriculum is no longer new to them-it is just what is expected.
In some programs, resistance to change came from an unexpected source-parents of students. Two factors seemed to bring this out: (1) parents whose children did not do well in the new curriculum, and (2) parents who studied accounting themselves and were concerned that their children were not learning the technical detail that they did as students.
Overcoming Resistance to Change
There are many things that can be done to overcome resistance to change, if change is correctly anticipated. Most important is to make sure that everyone has full information. Many schools have found it worthwhile to bring in outside experts, recruiters, employers, and alumni to address both faculty and students on the need for change and how the changes are likely to improve the program. Both written and oral communication to students is an obvious step, but communication with faculty, including those outside accounting is often overlooked. Sometimes such communication comes only after resistance is encountered, which often is too late.
Communication with students, their parents, high school and two-year-college counselors, employers, and others affected by the changes should take the form of a marketing campaign. Not only will this reduce misinformation, it will focus on the strengths and benefits of the revised program to students entering the business world. Especially important is bringing employers and alumni to the classroom and other student events to address how the changes will affect students after they graduate and enter the job market.
In addition to mere communication, participation in the change process is important. For faculty this is probably obvious. If faculty are to change how they teach, they need to be involved in determining how such changes will be decided and implemented. But other stakeholder involvement can also help the change process. Advisory boards are especially valuable because they bring the perspective of the marketplace to the deliberations. Students might be represented on planning committees, but more important is the use of student focus groups to react to planned changes. Not only will this provide insights into where student resistance may be encountered and how the program might be designed to minimize such resistance, it will often yield a set of informed students who can sell the concept of change to the other students.
Providing support to both faculty and students is also important, and it can be one of the most expensive parts of the change process. Faculty support comes mainly during the planning phases. Retreats, training sessions, and sending faculty to programs that will enhance their skills are ways of providing support. At some point, the hiring of an independent mediator may be helpful. Incentive schemes that reward faculty for devoting the time and effort needed to participate in the change process are powerful but possibly expensive methods of support. Some schools find that providing release time from other obligations, bonuses, or summer support for curriculum development is sufficient reward; others add a curriculum development component to faculty performance evaluation systems. Protection from low student evaluations when experimenting with curricular change is also helpful.
Student support includes a good orientation program, an open-door policy for counseling and guidance, open discussion sessions where students can voice their concerns and suggest improvements, and extra help in areas not traditionally considered part of the accounting curriculum. For the latter, writing centers are a very common support facility provided.
Sometimes gaining acceptance of changes requires negotiated compromises. Reduced class size might be the cost of getting a changed curriculum. Keeping some traditional materials may be necessary to introduce some new materials. Nontenured faculty might be protected from the efforts of curriculum revision so that they can devote their effort to the research necessary for promotion. In some cases, a small number of faculty may be exempted from the change activities and allowed to teach some courses using traditional methods so that they do not undercut the entire change process. All of these may reduce the complete commitment to change, but they may be necessary to accomplish any change at all.
Compromises on the student side usually involve allowing choices between old and new courses during a transition period. Students who choose to participate in a new curriculum have already made a step toward accepting change.
Resistance is a natural part of the change process, and it can be healthy as well as an obstacle. By anticipating and dealing with resistance, programs can alter plans to accommodate legitimate concerns and overcome some of the obstacles that are potential threats to change.
AECC in the Literature-Publications
One measure of the impact of the AECC is the degree to which articles by or about the Commission appear in the accounting literature and the number of times AECC publications are cited. By these measures, it is clear that the Commission has at least been noticed by authors and readers of the literature.
One way the AECC tried to reach a variety of audiences was through publications. Both of the Position Statements and the first four Issues Statements were published in Issues in Accounting Education before being collected in a special volume published by the AAA (AECC 1996). In addition, the Commission published three articles describing the grant proposals that were accepted for publication (Williams and Sundem 1990, 1991; Williams 1992c).
Listing all publications by all Commission members that made reference to the Commission would be a waste of space. However, some examples should suffice. The Chairman and Executive Director wrote many short pieces for a variety of newsletters (e.g. Williams 1992a, 1992b; Sundem 1991b, 1991c, 1991d). They also addressed practitioner and student audiences (e.g., Chironna et al. 1990; Sundem 1991a; Williams 1993). Many other Commission members also carried forth the message in print (e.g., Elliott 1992; Elliott and Jacobson 1992; Kieso 1992a, 1992b; Strait 1992).
In the later years of the Commission, the grant schools carried much of the publication burden. Some of the grant school publications are listed in exhibit 8-1. These publications are in addition to the descriptions published by the AECC (Williams and Sundem 1990, 1991; Williams 1992c; Flaherty 1998). Although not officially "published," the average grant school also provided written information about its project to more than 70 parties that requested such information. In addition, several put information on their web sites.
AECC in the Literature-Citations
Citations to the AECC in the accounting education literature have been extensive.9 Citations began appearing in about 1992, and they have been relatively constant through mid-1998. To assess the extent of these citations, I examined the two largest accounting education journals, Issues in Accounting Education and Journal of Accounting Education. From 1992 through mid-1998, 47 percent of the main articles (107 out of 229 articles) referred to one or more AECC publications. (In the tally, I omitted cases and other special sections that had few, if any, citations to the literature.) There is little difference between the two journals, with 46 percent of the articles in Issues in Accounting Education citing the Commission and 48 percent of those in Journal of Accounting Education.
The number of citations has stayed relatively constant across time. From 1996 to mid-1998, 51 percent of the articles cited the Commission, compared with 43 percent from 1992 through 1995. Annual citation percentages are:
1992
1993
1994
1995
1996
1997
1998
38%
55%
32%
46%
53%
45%
56%
For the Commission to be cited in nearly 50 percent of the articles over a period of seven years is a clear indication that authors of the articles were paying attention to the Commission. And there is no indication that references to the Commission are declining-if anything they have increased slightly in later years. Thus, one would expect such citations to continue.
EXHIBIT 8-1
Publications by the AECC Grant SchoolsAinsworth, P. 1992. Flint Hills Salon: A Case Analysis. John Wiley & Sons.
---, and D. Plumlee. 1993. Restructuring the accounting curriculum content sequence: The KSU experience. Issues in Accounting Education (Spring): 112-127.
---. 1994. Restructuring the introductory accounting courses: The Kansas State University experience. Journal of Accounting Education (Fall): 305-323.
---, D. Deines, D. Plumlee, and C. Larson. 1997. Introduction to Accounting: An Integrated Approach. Richard D. Irwin.
Albrecht, S. 1991. Implementing a new curriculum: The BYU experience. FSA Proceedings, 1991 Annual Meeting (December): 95-102.
---. 1992. Education update on BYU's new curriculum. Education Update (Spring).
---, and J. Smith. 1993. Integrating auditing across the undergraduate curriculum. Proceedings of Auditing Education Conference, Lehigh University, May 20-21.
---, and --- . 1994. Integrating auditing across the curriculum. Proceedings of Auditing Education Conference, SUNY at Binghamton.
---, C. Clark, J. Smith, K. Stocks, and L. Woodfield. 1994. An accounting curriculum for the next century. Issues in Accounting Education (Fall): 401-425.
--- . 1997. U.S. educators look to the big picture. Australian Accountant (May): 52-54.
Chronicle of Higher Education. 1997. The new accounting. (January 31): A10.
Cooper, W., G. Faucette, and C. Malone. 1995. Introducing practical experience into accounting education. Accounting Forum (March): 69-78.
---, L. Griffin, and C. Malone. 1996. Advisory Services Ltd.: An interdisciplinary project involving accounting and technology students. Mid-American Journal of Business (Fall): 48-58.
DeMong, R., J. Lindgren, and S. Perry. 1994. Designing an assessment program for accounting. Issues in Accounting Education (Spring): 11-27.
Deppe, L., E. Sonderegger, J. Stice, D. Clark, and F. Streuling. 1991. Emerging competencies for the practice of accountancy. Journal of Accounting Education (Fall): 257-290.
Hill, N., S. Perry, and D. Stein. 1998. Using accounting student surveys in an outcomes assessment program. Issues in Accounting Education (February): 65-78.
Insight. 1995-1996. Educators stop spoon feeding students. (December -January): 20-24.
Jones, K., J. Price, M. Werner, and M. Doran. 1996. Introduction to Financial Accounting: A User Perspective. Prentice-Hall.
Lewis, C. 1995. Critical thinking and the introductory accounting curriculum. American Accounting Association Communicator (February): 17-18.
Pattison, D., P. McKenzie, and R. Birney. 1995. Interactive Managerial Accounting Lab. McGraw Hill.
Romney, M., J. O. Cherrington, and E. L. Denna. 1996. Using information systems as a basis for teaching accounting. Journal of Accounting Education (Spring): 57-68.
Smith, R., and R. Birney. 1995. Interactive Financial Accounting Lab. McGraw Hill.
Stone, D., and J. Shelley. 1997. Educating for accounting expertise: A field study. Journal of Accounting Research (Supplement): 35-61.
By far the most often cited publication is Position Statement No. One, Objectives of Education for Accountants. Of the 107 articles citing the Commission, 82 percent cite the Objectives Statement. Next most often cited is Position Statement No. Two, The First Course in Accounting, cited by 17 percent of the 107 articles. The only other AECC publications with more than five citations are Issues Statement No. 1, AECC Urges Priority for Teaching in Higher Education, and Issues Statement No. 5, Evaluating and Rewarding Effective Teaching.
Beyond the U.S. Accounting Literature
Although the Commission's main goal was to affect accounting education, its activities reached beyond the borders of accounting. For example, Jean Wyer's article in Change magazine (Wyer 1993) introduced the Commission and its activities to a broad, interdisciplinary audience. The Commission has also been referenced in academic business journals outside accounting such as Journal of Applied Business Research and Business Communications Quarterly.
International journals have also shown interest in articles on the AECC. The British journal Accounting Education includes many references to the AECC, including two full articles. Sundem and Williams (1992) described the basis for the changes advocated by the Commission. Later, Mathews (1994) described the activities of the Commission, criticisms of its activities, and implications for other countries. He concluded that "there is a very high level of support for the work of the AECC" and that the grant program "as a whole appears to be working although it will take several more years to establish whether the entire scheme has been a success" (Mathews 1994, 200, 202). Journals in several other countries, including Canada, Ireland, Germany, Japan and New Zealand, have also had articles on the Commission.
Awards to AECC Grant Projects
One measure of the quality of the AECC grant projects is the awards they have received. Three of the projects have won the AAA Innovation in Accounting Education Award: 1993-Brigham Young University School of Accountancy & Information Systems; 1995-School of Accountancy, Arizona State University; and 1997-David Croll and Anthony Catanach, Jr. The first two are self-explanatory, but the third needs a bit of elaboration.
David Croll and Anthony Catanach, Jr. are on the faculty of the McIntire School at the University of Virginia. They developed a new approach to the intermediate accounting course as part of Virginia's AECC grant project. Their "Business Activity Model" (BAM) replaced the traditional lectures and textbook assignments of intermediate accounting with a two-semester focus on accounting for a business from its inception through seven years of operation. Jensen (1998) calls their approach "revolutionary," and indicates that "the BAM approach makes students search for answers on their own or in teams….The main innovation of the BAM pedagogy is that students teach themselves in a discovery learning pedagogy."
8This fear was articulated by Richards (1992, 15): "Certainly accounting educators can teach the base level of skills, but can we teach the second-level skills? It is difficult to see how we can, either now or ever….We do not have the education in these areas, and it is not likely that we ever will. If skills beyond the base level are to be taught, they should not, indeed cannot, be taught by the accounting faculty….Accounting educators are teachers of accounting and nothing more." This is a very pessimistic view of the capabilities of accounting faculty.
9As expected, there are few citations of the AECC in the accounting research literature.
