A Tool for Effective Grading:
Primary Trait Analysis

In their book Effective Grading: A Tool for Learning and Assessment, Barbara Walvoord and Virginia Anderson introduce a method for establishing criteria for grading that brings rigor to the classroom and allows grading to be used, if desired, as the basis for departmental or programmatic assessment. Describing grading as a complex process involving evaluation, communication, motivation, and organization, the authors cover both conceptual and practical issues related to designing assignments and effective grading with examples from many disciplines. Here they share a method for making course expectations explicit and easily understood.

The method is called Primary Trait Analysis (PTA) (Lloyd-Jones 1977). It creates a scoring rubric that can be used to assess any student performance or portfolio of student performances—written, oral, clinical, artistic and so on. PTA is assignment-specific; that is, the criteria are different for each assignment or test. … As we apply it here, PTA is a way of explicitly stating the teacher’s criteria, and it is used in the classroom to make grading criteria very clear and specific. … PTA works for departmental or institutional assessment because it provides a common format for stating various teachers’ criteria and standards. PTA, because it is very explicit, allows criteria to be made public and understandable to outside audiences. Here, however, we show how teachers can use PTA inside their classrooms to make criteria and standards clear to themselves and their students and to guide classroom teaching and learning.

Characteristics of Primary Trait Analysis
… PTA is both highly explicit and criterion-referenced. To construct a PTA scale, the teacher (1) identifies the factors or traits that will count for the scoring (such as “thesis,” “materials and methods,” “use of color,” “eye contact with client,” and so on); (2) builds a scale for scoring the student’s performance on that trait; and (3) evaluates the student’s performance against those criteria. A Sample PTA Scale … Anderson asked her students in an upper-level biology course to design and carry out an original scientific experiment comparing two commercially available products and to present the comparison in scientific report format.

Anderson used PTA to assess student work on that assignment. She began by choosing ten traits she wanted to measure:

1. Title
2. Introduction
3. Scientific format demands
4. Methods and materials section
5. Nonexperimental information
6. Experimental design
7. Operational definitions
8. Control of variables
9. Collection of data and communication of results
10. Interpretation of data: conclusions and implications

Depending on her purposes, Anderson could have decided to measure only one or two traits, or more than ten. … [her] second step was to build a two- to five-point scale for each trait, describing each performance level. For example, Exhibit 5.1 is Anderson’s scale for one of her traits, the “methods and materials section.” She used a five-level scale, with five as the highest score. It is possible to use four, three, or two levels, depending on your purposes. A two-level scale is essentially a pass-fail or yes-no decision and can be used where you need only that level of judgment.

Anderson constructed a similar scale for each of the other nine traits she had identified.

How to Construct a PTA Scale
To begin her PTA scale for the scientific experiment assignment, Anderson selected a sample of former student papers, across a range of quality. (You don’t have to have former papers, but they can be useful. If you don’t have them, then try to imagine what student papers might contain.)

Next Anderson asked herself, What are the factors or primary traits that I want to measure? The traits are the factors that will count for grading or scoring the student’s work. A glance at Anderson’s ten traits presented earlier will show that traits are expressed as nouns or noun phrases.

Traits may already by stated in your assignment sheets, grading checklists … or other material. If you cannot immediately come up with a list of traits, then begin by describing an A paper or a C paper. The traits may emerge within your descriptions.

To develop traits, it also helps to talk with a colleague. If you have students’ earlier work, let a colleague outside your discipline read a few examples. Then try to tell that colleague exactly why you gave that paper a B or C. Anderson, a biology teacher, found it perplexing but very helpful to explain these things to Walvoord, an English teacher. For example, when Walvoord complimented one student’s research report for beginning with such an “effective quotation,” Anderson had to find the words to explain concretely what makes a good opening for a science report and why a good opening does not contain a quotation. Once you have your traits, construct a scale for each trait, as Anderson did for the Materials and Methods section (in Exhibit 5.1).

Scoring with a PTA Scale
Having explained how to construct a PTA scale, we move to the next topic, which is how to score student work using a scale. Anderson’s “title” trait for her students’ scientific reports in biology provides a quick and easy example (Exhibit 5.3) [With her criteria] try using the PTA scale to rate each of the following titles. They were written by [her] students before she developed her PTA scale.…

• A A Comparison of Prell and Suave Shampoo
• B The Battle of the Suds: Budweiser and Weiderman Beer
• C Would You Eat Machine-Made or Homemade Cookies?
• D A Comparison of Arizona and Snapple Ice Tea for pH, Residue, Light Absorbency, and Taste
• E Research to Determine the Better Paper Towel
• F A Comparison of Amway Laundry Detergent and Tide Laundry Detergent for characteristics of Stain Removal, Fading, Freshness, and Cloth Strength

Let’s see how you did. Here are Anderson’s scores:

• A 3 Prell and Suave identify the brand names. The word comparison vaguely hints at design and function but without specificity.
• B 2 Only the brand names are explicit, and the title is almost misleading.
• C 1 Perhaps it is modeled after a Speech 101 title that worked, but it doesn’t fit this upper-level biology assignment.
• D 5 The design is clearly specified, and the writer includes all the key words that will accurately classify this report in permuterm indexes or electronic databases.
• E 2 As perfunctory as “Book Report on Silas Marner.”
• F 4 Very good, but wordy.

After trying this scoring, you might recommend changes to the scale. Perhaps, after trying to score title F, you would recommend that the words is concise be added to Level 5 and meets all criteria for 5 but may be wordy be added to Level 4. … PTA scales tend to be revised as you use them, and they should be. The benefit of doing a PTA scale lies as much in the hard thinking it forces the teacher to do, and in the influences it exerts on teaching and learning, as in the final scale that emerges.

Why Take the Time to Do PTA
[Anderson adopted PTA scales] because she wanted to:

• Make grading more consistent and fair
• Save time in the grading process. Once she was very clear about what she was looking for and had the PTA scoring rubric, she could move quickly through the students’ work.
• Diagnose her students’ strengths and weaknesses very specifically in order to teach more effectively.
• Track changes in her students’ performance over several semesters so she could see how changes in her teaching affected student performance (see Anderson and Walvoord, 1991).

Here are some reasons why other faculty in our experience have found it worthwhile to do PTA:

• To help teaching assistants grade papers consistently
• To reach agreement with colleagues on criteria for common exams, for multiple sections, or for sequenced courses
• To introduce greater distinctions into one’s grading (For example, a psychologist had written a set of loosely stated criteria, but she found herself giving A’s to papers she felt did not deserve an A. Somehow, she had not captured in her loose list the full range of criteria she wanted to use to make distinctions. A primary trait analysis, with its greater specificity, helped her tease out for herself what those criteria were and then to distinguish the truly excellent papers from others.)
• As data for departmental and general education assessment.

Note: Entire assignments and grading criteria scales are included in the book’s appendices.

Full citation: Walvoord, B. E., and V. J. Anderson. 1998. Effective Grading: A Tool for Learning and Assessment. San Francisco, CA: Jossey-Bass Publishers.

References: The full list of references cited in this excerpt can be obtained in hardcopy by contacting the office at (941) 921-7747.