Evaluating Faculty: A toolkit of resources

The excerpts below are from the AAA’s Faculty Evaluation Toolkit, developed for the AAA’s Academic Partner Schools by William E. Cashin. The toolkit contains a handbook developed by Cashin, author of several reviews of literature on the topic; the book Assessing Faculty Work by Braskamp and Ory; and a collection of annotated bibliographies on the topic. The purpose of the toolkit is to guide busy faculty and department chairs to some of the most useful resources in the research literature on faculty evaluation. Sections in the toolkit cover the information below as well as student ratings of teaching, quantifying or weighting evaluations, the role of evaluation processes in merit pay, interpreting data, the relationship between faculty evaluation and improving teaching, and web resources on the subject. Over 150 resources are annotated in the toolkit handbook. Examples are shown in the excerpts below.

Foreword
The topic of this toolkit is faculty evaluation, the making of personnel decisions about retention, promotion, tenure, and salary increases. To accurately assess a faculty member’s performance takes considerable time and effort. If we go to that much trouble, I will suggest that the data also be used to help the faculty member improve.

A faculty member’s responsibilities, and therefore his or her performance, are extremely complex (as, I suggest, are a student’s). Some faculty take the position that what they do is too complex to be meaningfully evaluated, let alone reduced to numbers, especially a single number (the way we do with grades and GPAs). My position—based on both the research and experience—is that, although we cannot perfectly measure a faculty member’s performance, given the appropriate time and effort we can reach a meaningful and useful approximation. We can with reasonable accuracy assign a faculty member’s performance during a given period to a meaningful group, e.g., “satisfactory” (something like assigning students letter grades). We realize that the “grade” certainly does not define the person, but represents—when done properly—an informed opinion about a specific performance.

Why is faculty evaluation important?
The simplest answer is because every college and university evaluates its faculty. Any time an institution decides to retain or dismiss a faculty member, to promote or not, to grant tenure or not, to give this salary increase or that, it is evaluating. Colleges and universities have always evaluated; more importantly, they always will! The question has never been whether faculty will be evaluated, but how will they be evaluated.

The fact that evaluation is always done, however, does not mean that it is necessarily done well. In the past especially it may have been done rather superficially and subjectively. Sometimes evaluations were based on the complaints of a few students, or bits of a lecture overheard through an open classroom door, or gossip in the faculty lounge, perhaps even on one’s appearance or how one dressed. But—based on some kind of information—judgments were made. How well a college or university, or a department or some other academic subunit, evaluates its faculty is extremely important both for the future of its individual and collective faculty and of that academic unit, and most importantly, for its students and others served by its educational mission.

Comment: I would suggest that it is important in another way. For decades higher education has in effect been saying, we are doing a good job, trust us. Students, parents, legislators, the public in general—our stakeholders—are no longer willing to simply take our word for it. I believe that we are not doing as bad a job as the public thinks—nor as good a job as we say—but that an accurate assessment of U.S. higher education would show that in the main we are doing a satisfactory job. A faculty evaluation system that yields accurate information about faculty performance could also provide data to support that claim.

What Should Faculty Evaluation Be?
At its worst, faculty evaluation is making a judgment about a faculty member’s performance based on the slimmest of and possibly rather irrelevant information. But some kind of information or data are always used. Effective evaluation should be systematic, comprehensive, public, and flexible:

systematic–the data should be collected in an organized way,

comprehensive—all major responsibilities should be included in the evaluation,

public—procedures should be known beforehand and in writing,

flexible–the evaluation system must fit varying circumstances of different academic units.

What Is Legal?
Does even your college or university attorney know? Probably not, because there are no absolute or definitive answers. What is allowed depends on whether you are a public or a private institution, whether you are unionized, what state you are in, whether the faculty member is a member of a protected group, whether you followed your—hopefully written—policies, and on and on. However, although there are no definitive answers that apply universally, there are broad—and useful—principles and a growing body of case law.

Suggested Reading
15. Baez, B., and Centra, J. A. (1995). Tenure, promotion, and reappointment: Legal and administrative implications. ASHE-ERIC Higher Education Research Report, No 1. Washington, D.C.: Association for the Study of Higher Education.

This report, as its title indicates, deals specifically with the legal implications of tenure, promotion, and reappointment, including such topics as Affirmative Action, diversity, individual rights, and peer review. The authors do so in detail but in language understandable to the layperson. Every college and university library—and their attorney—should have a copy of this book, and most administrators and faculty should read it.

What Should Be the Scope of the Evaluation?
In principle this is easy to state: every important responsibility of a faculty member should be evaluated, which means that data must be gathered about every important responsibility. The problem comes when trying to put the principle into practice. If service and advising, for example, are important—and they are—then meaningful data must be collected to provide a basis from evaluating these responsibilities. If attending student functions is important at a given institution, then that institution must gather data on faculty attendance. To base evaluations on data that are obtained only by chance leads to inaccurate, and therefore unfair, evaluations.

How frequent and how detailed should evaluations be? This really depends on the purpose of the evaluation. Data for promotion and tenure do not need to be reviewed every year. However, for improvement as well as evaluation, it is recommended that every faculty member meet with his or her supervisor (usually a department head or chair) for a formal performance appraisal every year. One purpose of such a meeting should be to give untenured faculty an assessment of their progress toward achieving tenure, or being retained; for most faculty, feedback on progress toward promotion; and with merit pay, a rationale for next salary increase. Often such meetings also include discussion of improvement (except of course for the absolutely perfect faculty). At some institutions a separate but formally scheduled meeting is held specifically to discuss improvement. (This will be discussed in more detail in Part VIII. Feedback Process.)

What Are the Faculty’s Responsibilities?
Everybody in higher education knows the answer to that question: teaching, research, and service. Well maybe research should be “scholarship/research/creative activity.” Personally I think advising is so important that it should be listed separately. Although professional competence, professional behavior, and possibly professional development have usually been included implicitly, I suggest that the time has come to explicitly discuss them. These will be treated below after the more general question of criteria.

What Criteria Will Be Used?
The term “criteria” is used loosely and variously by different institutions. For some it is simply teaching, research, and service; for others it is “excellence” in these three. Excellence is almost never defined, to the chagrin of faculty. As far as faculty evaluation is concerned, criteria should involve a standard or rule upon which a decision can be based. The ultimate goal should be to have specific, operational definitions. For example, using the institution’s student-rating items, average ratings (from all courses taught during the evaluation period) on a global instructor item (e.g., This instructor is an excellent teacher) of 3.5—with no average being below 3.0—might be considered “satisfactory.”

Some of the other issues that need to be considered are expectations for the faculty. Should they be uniform (for the entire institution, for a given department), or may they be individual (differing from person to person within a department, or even for the same individual for different years)? How comparable should the workload be, say comparing graduate and undergraduate teaching?

Suggested Reading
27. Meyer, K. A. (1998). Faculty workload studies: Perspectives, needs, and future directions. ASHE-ERIC Higher Washington University, Graduate School of Education and Human Development.

This report is an up-to-date study of faculty workload and the various approaches and problems in defining—and increasing—workload. Concludes with a section on useful solutions, e.g., we don’t need faculty to work longer, but smarter.

Procedures for Evaluation
The general literature on faculty evaluation… contains some discussion of faculty evaluation procedures. However, there is very little discussion of how an institution should go about changing its procedures. To fill this gap some principles from the literature on educational change will be suggested. Also some of the psychology and the communication literature is relevant.

What Should the Statement of Procedures Cover?

1. What purposes will evaluation serve?
2. Which faculty responsibilities will be evaluated?
3. What data will be collected for each responsibility?
4. Who will supply data?
5. Who will serve as evaluators?
6. What criteria will they use for each responsibility?
7. How will decisions be made?
8. To whom and how will decisions be communicated?  
9. What will be the appeal procedures?
10. What are the time lines?

The first six have been discussed in the previous Parts of this toolkit… The timelines for the evaluation procedures vary so widely across institutions, that they will notbe discussed in this toolkit.

What Process Is Best for Developing an Evaluation Plan?
Comment: There is no research evidence that one strategy works better than another. Experience suggests that pure reason does not work, nor does coercive power. The problem is that in trying to change a faculty evaluation system, one is trying to change not just ideas, but feelings, attitudes, values.

Suggested Reading
14. Diamond, R. M., and Adam, B. E. (Eds.). (1993). Recognizing faculty work: Rewards systems for the Year 2000: New Directions for Higher Education, No. 81. San Francisco, CA: Jossey-Bass.

This entire issue deals with faculty rewards. The following two chapters especially focus on fitting rewards to the institutions goals: Changing Priorities and the Faculty Reward System (pp. 5–12) and Instituting Change in the Faculty Reward System (pp. 13–22).

The complete Faculty Evaluation Toolkit was developed for participants in the AAA’s Academic Partners program. Academic Partner Schools receive their choice of two toolkits as part of their membership. AAA members who are not Academic Partners can purchase the toolkit for $100, non-AAA members for $150.

The Academic Partners Program has since been discontinued.