Performing a Literature Synthesis Project for the PCAOB
Dana R. Hermanson
Kennesaw State University
The Auditing Section recently solicited volunteers to participate in "syntheses of existing research related to key PCAOB standards-setting projects" (see http://aaahq.org/audit/call_for_participants_in_researc.htm). The future projects include:
- Audit Confirmations
- Audit Firm Quality Control
- Audit Reporting Model
- Auditing Related Party Transactions
- Communications with Audit Committees
- Engagement Quality Review (concurring or second partner review)
- Financial Fraud.
I recently worked with an outstanding team of researchers (Bob Allen, Tom Kozloski, and Bob Ramsay) on one of the first two projects for the PCAOB — a literature synthesis on Auditor Risk Assessment. As of this writing, our project is nearing completion. The purpose of this brief piece is to share the nature of our process and my experience, so that others can have an idea of what to expect on such a project.
Assembling the Team
Going forward, the Auditing Section will be creating the research teams. Our team was a subset of the Auditing Standards Committee members — those who expressed an interest in this project. Bob Allen, Tom Kozloski, and Bob Ramsay all had significant knowledge of the subject area, while I viewed my role as oriented more toward project management, editing, and interacting with the PCAOB staff. I believe that it is important to identify, right at the beginning, a person to take on the project management role.
Organizing and Dividing the Work
The first question our team addressed was, "What exactly does the PCAOB staff want to know about auditor risk assessment?" Our fear, which was later confirmed, was that there would be far too much literature to synthesize. To answer this question, we went to the PCAOB Briefing Paper on risk assessment that had been prepared for a Standing Advisory Group (SAG) meeting (available on the PCAOB website). This briefing paper presented 10 broad discussion questions for the SAG, and we ultimately used this framework to organize our literature review. This approach may not always be optimal, but it seemed to work well for our group.
Once we had the organizing framework laid out, we divided responsibility for the literature searches. Bob Ramsay was in charge of "business risk" issues, including innovations in the audit approach to consider strategic-level client risks. Tom Kozloski was in charge of "fraud risk" issues, which were later integrated throughout the paper. Bob Allen and I took responsibility for "inherent risk," "control risk," and "audit risk model" issues, including the linkage between auditors' risk assessments and the nature of subsequent audit testing.
The literature searches included databases of published papers, as well as scans of the AAA Annual Meeting program, Midyear Meeting program, and SSRN. In addition, we contacted certain authors in the area and asked about current working papers. The searches produced much more than we could reasonably synthesize, so we went through a process of eliminating papers that we did not believe were directly related to the 10 PCAOB questions, as well as some papers that had findings similar to other papers. Once we had a reasonable list of papers, we organized them by the 10 areas and began to write the sections of the paper. We then pulled the sections together and identified the key messages from our review. Finally, we identified future research directions.
Much of our group's interaction was through email. We held a few four-way conference calls at key points in the project, and we often had an informal agenda or proposal to guide these discussions.
Timing and Workload
Our work on this project began around March 1, 2005 and involved fairly steady effort for just over four months. We did not track our hours, but I would equate this project with other papers targeted to Journal of Accounting Literature or a similar outlet. Obviously, our "data" was the results found in others' research, so this type of project can be done more quickly than a large-scale empirical project. The relatively tight timeframe on these projects, though, does require that everyone on the team be able to jump in immediately and hit all the key deadlines. Our team set several intermediate deadlines, especially in April – June to ensure that we were all on target during the process.
Interacting with PCAOB Staff
Our interaction with the PCAOB staff has been fairly steady throughout the project, and often by email, with the goal of making sure our team understood some of the key issues of interest at the PCAOB. In addition, I had a couple of calls with Gary Holstrum and other PCAOB staff to clarify the scope of our project. The PCAOB staff has been extremely supportive of the project and has provided important feedback to help us to more effectively communicate the message of the underlying academic research. Gary also has been very helpful in prompting us to think about and enumerate areas for future research on auditor risk assessment. In summary, our interaction with the PCAOB staff has been very positive.
Informal Review Process
Our first draft of the paper (in late June) was sent to Gary Holstrum at the PCAOB, as well as to several prominent researchers in the area who previously had agreed to give us feedback (Mark Beasley, Jean Bedard, Rich Houston, Ed O'Donnell, and Arnie Wright). Each of these individuals provided us with valuable feedback in a very quick, two-week turnaround. The feedback included identifying several working papers and a few published papers that we added to the synthesis. After we incorporated these suggestions into the paper, we had Joe Carcello review the revised paper as an additional check, and we have given Gary Holstrum the opportunity to request any additional clarifications from the PCAOB staff's perspective.
Working Toward Completion and Publication
Our project is targeted toward publication in Accounting Horizons. The President of the Auditing Section (Linda McDaniel at that time) approached Editor Bob Lipe as to his interest in publishing reports prepared by the teams. He expressed interest, and he asked that each team submit a two-page synopsis of their project, which would be reviewed by members of his editorial staff. In our case, the editorial staff concluded the topic and the committee's approach were a good fit with Horizons, and Bob informed us our project would be a commissioned commentary. Also, the editorial staff provided feedback on the proposal. We will submit a full version of the paper to Accounting Horizons by late October. As long as the final document conforms to the proposal, Bob will publish it, subject to a final round of expositional editing. He and the incoming editors of Horizons expressed interest in treating other Auditing Section synthesis projects in this manner.
Personal Benefits
I have experienced three major benefits from participating in this project. First, it has been a pleasure to interact with the PCAOB staff and be in a position to contribute to their process of developing auditing standards. Often, academics may believe that no one is reading their work — or at least that no decision-makers are reading it. With these PCAOB projects, you can be assured that your work will be read and appreciated by the auditing profession's key regulator.
Second, I had the opportunity to work on a project with three people with whom I had not previously written a paper. Each of three, Bob Allen, Tom Kozloski, and Bob Ramsay, is dedicated, talented, and extremely conscientious. I fully expect that this project will lead to additional research projects.
Finally, I learned a lot from doing the project. Of the four of us, I think I had the most to learn about auditor risk assessment literature, but I believe that we all gained a much more complete perspective on the research in this area. This perspective will help me in the classroom (I am teaching undergraduate Auditing for the first time in my career this semester), as well as in future research projects.
Conclusion
I hope that these observations are helpful to others who are considering a PCAOB research synthesis project in the future. While the deadline for indicating an interest to serve on the seven current PCAOB projects has passed (September 15th), there will ongoing opportunities to serve on future research teams. I strongly encourage Section members to get involved. To me, this is what research is all about — providing insights to the decision-makers. In the process, I think you'll learn a lot, too.
|
