The Auditors Report

Acquiring New Intellectual Capital:
Experimental Research Methods

Steve Salterio, Professor
Queen’s School of Business, on behalf of the Research Committee

The first few years as an assistant professor fly by quickly—suddenly, you are asked to submit your tenure package, and if all has gone well, soon after that you are a tenured associate professor. What you probably realize, after the tenure celebration is over, is that you have been drawing on the stock of intellectual capital that you acquired in your Ph.D. coursework for these many years. That coursework is now some eight to eleven years old, depending on how long it took from coursework completion until you started your first academic appointment and then completed the tenure process. You are beginning to notice new assistant professors using research methods that were unheard of in your days as a Ph.D. student. You resolve to catch up—after all, more research is expected before you are a full professor. But how? This article explores one tenured faculty member’s approach to catching up in research methods used by experimental researchers. Future articles by the Research Committee will discuss approaches to learning new theories and archival research methods.

Shortly after he graduated from his Ph.D. program, our fictional new associate professor, Barely Begun (BB for short), began to notice that a statistical technique he had never heard of, Structural Equations Modeling (SEM), was being featured frequently in audit experimental research (e.g., Libby and Tan 1994). It really became an important issue when BB was asked to review a paper that featured SEM prominently as its principal method of analysis. He found himself eagerly reading a short monograph entitled “Structural Equations Modeling for the Disenfranchised” (or some similar name, the exact reference is lost with the passage of time) so he would not embarrass himself in the review process. With the review safely out of the way, BB considered how he was going to deal with this new-found deficiency in his research toolkit.

A number of approaches occurred to BB:

  1. He could read an SEM book (e.g., Bollen 1989) and try to teach himself this research method;
    Pros: Cheap—only need to buy the software (e.g., Amos) and the textbook.
    Cons: Self-motivation might result in giving up. It is potentially inefficient to learn a statistical method on your own.
  2. He could audit an SEM course at his own university;
    Pros: Relatively cheap—often can be done tuition-free, so he would need only a text and software. He might be able to include it as part of the justification for spending a sabbatical term at his home university.
    Cons: Normally, courses are held during the principal teaching terms. He might have to take the course with his own graduate students. A rigid schedule for a 13- to 17-week term can be hard to fit into a busy associate professor’s life. Furthermore, many cutting-edge statistical methods courses are over-subscribed by graduate students, or only rarely offered, because of resource constraints.
  3. He could attempt to take a course over the summer;
    Pro: Timing works well.
    Con: Reduced opportunity to obtain teaching-based summer funding. Very limited selection of methods courses offered at most universities during the summer.

BB’s final analysis was: Teaching yourself any statistical tool is fraught with difficulty and is likely to be inefficient and result in less knowledge than could be gained with the same time committed to guided study. Taking a course during a regular term could be difficult to commit to with the pressures of teaching, research, and service. Finally, next to no statistical methods courses tend to be offered in the summer at many universities—even at research-intensive universities. On the verge of giving up, BB vaguely remembered hearing about summer statistical institutes held at a variety of universities throughout the U.S. and Canada, but he did not know much about them.

After some Internet research, BB found a number of such institutes that were sponsored by survey research centers or academic statistics departments at various public universities. BB found that one of the oldest of these summer research institutes is sponsored by the University of Michigan’s Institute for Social Research (http://www.isr.umich.edu/src/si/icpsr.html). Two summer programs are offered: the Summer Institute in Survey Research Techniques (in operation for over 50 years) and the Summer Program in Quantitative Methods (in operation for over 40 years). Between the two programs, almost every imaginable type of behavioral research method is covered—simple ANOVAs, experimental design, survey research design and analysis, survival analysis, sophisticated time-series analysis of data, as well as various types of qualitative research, including computer methods for analyzing qualitative data. Indeed, the SEM course BB was looking for was offered in three different formats, a mathematically intensive three-week version (two hours of class time and one hour of lab time daily), an applications-intensive three-week version (similar class time commitments), and a one-week intensive applications-oriented version (four hours of class time and three hours of lab time daily). In the two three-week summer periods, almost 40 different statistical analysis and design courses were offered by the Institute, including complimentary brush-up seminars in linear algebra, calculus, and computer-intensive statistical analysis for those who felt the need for additional background preparation (one or two hours daily for 10 days).

Although somewhat expensive—BB paid $1,000, a discounted tuition rate because of BB’s university affiliation with UM’s ISR—to take the course as well as pay for accommodations and meals, taking the one-week intensive course was a very efficient use of BB’s time. Given the tuition level and the one-week, intensive nature of the course, class size was limited to 20 students. Furthermore, the instructor, Kenneth Bollen, had written the definitive text in the SEM field (Bollen 1989) and brought his research assistant with him to give additional opportunity for one-on-one coaching.

However, the one-week intensive nature of the course made BB very glad that he had read all the suggested reference materials prior to the course. It was a challenge to keep up with just the day-to-day readings and assignments during the course itself. Furthermore, despite almost half of the class being faculty members, the instructor was serious about all students doing and handing in homework. BB found that having his own data set to use during the lab times enhanced his experience over those that had to use the canned data sets provided by the instructor. Using his own data allowed BB to receive input from the instructor on issues that he would normally face with the type of data he typically collects.

Overall, BB found this to be a highly efficient way of dealing with what he perceived to be a major deficiency in his behavioral research methods toolkit. Furthermore, he learned that several other universities had some form of summer statistical research institute, although none seemed as comprehensive as UM’s ISR (e.g., York University in Canada’s Survey Research Center offers several courses every May and June). While BB recognized that this summer course approach is not the only way to acquire new intellectual capital, BB found this approach relatively painless and a most useful approach when retooling in behavioral research methods.

Note: If you have any retooling experiences in behavioral research methods that you would like to share with other Auditing Section members, please submit them to Steve Salterio at sesalter@uwaterloo.ca. Some of these ideas will be posted on the Section’s website under the Research heading—no commercial endorsements please. If you have archival research retooling experiences, please forward them to Michael Willenborg at m.willenborg@uconn.edu. He is in the process of preparing an article on archival research methods retooling that will appear in the next issue of The Auditor’s Report.

References:
Bollen, K. A. 1989. Structural Equations with Latent Variables. New York, NY: John Wiley.
Libby, R., and H. Tan. 1994. Modeling the determinants of audit expertise. Accounting, Organizations and Society (November): 701–716.


Back to Contents Page