|
A Model for Assessment Planning
This chapter summarizes the model for assessment planning and implementation described in the remaining chapters. The model is represented visually in the icon used throughout the document. Examples of assessment plans are presented in Tables 3.1 and 3.2.Assessment planning can be conceptualized in terms of eight essential components. A brief description and illustration of each follows.
PROCESS
ILLUSTRATION Establish Assessment Purposes and Priorities: The assessment committee must specify the planning processes and decisions the assessment will support. The primary purpose of assessment is to improve learning outcomes. Assessment also serves multiple purposes, such as increasing program focus, marketing the program and ensuring public accountability.
Provide information to support curricular and instructional planning related to student learning outcomes in three areas: communications skills, teamwork, and professional ethics.
Establish budget and Obtain Needed Resources: Allocation of adequate resources is crucial to realize benefits of assessment. The assessment committee should identify needed resources and work with the administration to obtain them. The committee should seek to minimize costs and maximize benefits by capitalizing on existing data sources. The budget should answer the question: What will the program cost?
Costs for assessment include faculty participation, staff support, training and materials. Specific cost sources include planning, preparation and distribution of materials, data collection and analysis, and dissemination of results for discussion and use.
Clarify Curricular Goals: Assessment compares actual outcomes to the goals of the curriculum to determine program effectiveness. The assessment committee must therefore review curricular goals to clarify intended outcomes of the academic program. The goals should reflect the institutional and departmental mission, and the needs of the profession. They should answer the question: What is the program intended to accomplish?
Focus on communication skills): "Program graduates will be able to communicate effectively in professional settings, both orally and writing."
Translate Goals into Curricular Objectives: The committee must translate educational goals into tangible outcomes or objectives so that appropriate measures can be designed or selected. Learning objectives at the program level can be identified through a review of program documents, curricular materials, course syllabi and exams, and actual student work. The objectives answer the question: What will program graduates be able to do as a consequence of their education?
Graduating seniors will able to write or present a management report with a clear main point, using relevant examples and graphic illustrations to support their position.
Develop Research Plan and Methodology: The assessment research plan specifies questions to be investigated, key variables and measures to asses them, data collection procedures, groups to be studied, sample sizes, and analytical methods. The plan should yield credible information of practical significance and interest to faculty. The research plan answers the questions: What do we want to find out, and how will we go about it? What should we measure? What methodology should be used?
Research question #1: Do students' communication skills improve from sophomore to senior year? What factors influence change? Method: Communication skills and related learning experiences will be assessed in sophomore, junior, and senior years; trend analysis and causal modeling will be used to monitor changes and determine contributing factors.
Research question #2: Do graduating seniors display communication skills to meet the demands of practice? Method: Practicing professionals will review a sample of videotaped senior presentations, then participate in a focus group to discuss students' readiness to communicate effectively in professional practice, identify additional needs, and offer feedback to individual students.
Develop Measures to Assess Learning Outcomes: This part of the assessment plan describes the measures that will be used to evaluate learning outcomes. Cognitive outcomes such as communication skills and ethical reasoning can often be measured using "authentic" methods (such as oral presentations, simulations, and portfolios of student work) embedded in classroom instruction. A single measure can often be used to assess several different learning outcomes. Multiple measures should be used to validate results, especially for high-priority outcomes.
Values and attitudes are measured using questionnaires, surveys, interviews, focus groups, unobtrusive observations, simulations, and rating forms. The measures chosen should help to answer the question: What student behavior will we examine to determine whether the outcomes have been attained?
Videotaped presentations by students will be rated by faculty and practicing professionals on selection and application of accounting principles, complex problem solving, and communication skills.
Develop Measures to Assess Contributions of the Educational Environment: The educational environment can be measured by observation, student and alumni perceptions and satisfaction ratings, review of program documents and instructional materials, and review of institutional records on matters such as persistence, attrition, and course-taking patterns. The measures selected should help to answer the question: What factors in the educational environment should we examine to understand influences on learning outcomes?
To assess factors in the educational environment related to communication and other skills, a faculty subcommittee will construct a matrix indicating which courses address these skills through a) instruction and b) evaluation. They will review instructional materials and student and alumni perception of instruction related to these skills.
Use Results for Program Improvement: Prior to implementing its program, the assessment committee should specify the format, timing, and context for reporting results to stakeholders as well as how the data will be used for improvement of learning outcomes and processes. The plan for reporting results should be based on the original statement of purposes and priorities for assessment. The plan should answer the question: How will results influence the accounting curriculum and teaching methodology?
At a department meeting or retreat, the assessment committee will present results of three assessment initiatives described above (focus groups or videotaped student presentations, review of instructional materials, and student and alumni evaluations). The faculty will interpret results and recommend changes if warranted.
Identify Process Improvements: The assessment committee and department chair should review the assessment program or assign responsibility elsewhere. Review criteria should include accuracy, relevance, usefulness of the process and results, cost-effectiveness, and benefit to students. The committee should identify ways to improve and simplify the process and to increase the value and use of results. The process should answer the question: How will what we have learned from this cycle affect the curriculum and what have we learned to improve cycles in the future?
The department chair, dean, and assessment committee will analyze costs and benefits of the program at the end of each year of implementation, and make recommendations to the faculty at a department meeting
Table 3.1
SAMPLE DEPARTMENT ASSESSMENT PLAN AThe following assessment activities will be phased-in over a 5-year period.
Goal
Objective: Students will be able to:
Assessment Measures
Time Frame
Use of Results
Professional knowledge as foundation for lifelong learning.
Apply and adapt accounting concepts and principles in a variety of contexts and circumstances (Objectives, p. 6)
Portfolio analysis: faculty judgments of students' use of accounting knowledge based on portfolio of major assignments and Senior Capstone Project.
Junior and senior year
Determine patterns of strength and weakness in junior year; use to advise individual students, identify curricular needs for senior year and/or future sophomore-junior instruction.
Information Management Skills
Access, organize and synthesize information from print and electronic sources.
Faculty judgment of information skills, included as a rating scale on relevant class projects.
Junior and senior year
Same as above
Critical Thinking Skills
Use correct inference and deduction, recognize assumptions, interpret evidence, and evaluate arguments.
Diagnostic critical skills test.
Faculty judgments of students' reasoning, using holistic rating scale when grading targeted projects.
Sophomore year
Junior year
Annually compile holistic ratings for each class of students; review for trends, identify needed improvements if indicated.
Communication Skills
Use verbal and graphic displays effectively to communicate financial information and recommendations.
Samples drawn from coursework.
Annual Review of representative sample of student presentations.
Sophomore-senior comparison to determine growth; senior year review by practicing professionals.