Senior management meets periodically to review the allocation of effort between ongoing evaluations and separate evaluations used to conduct monitoring activities.

The mix of planned monitoring activities over internal control of external financial reporting may depend on senior management's assessment of:

Senior management may also increase the frequency of separate evaluations from the initial plan in processes where:

Hunter Manufacturing has thirteen different plant locations, six of which are considered significant. The management team of Hunter Manufacturing has been monitoring the internal control in the seven smaller, less significant plants, primarily through ongoing evaluations. However, management has now determined that some separate evaluations have become necessary. This decision has been made due to the increase in risk factors at these plants, including frequent errors in monthly and quarterly reconciliation activities and turnover among plant-level controllers and supervisory personnel. Accordingly, management now has both ongoing and separate evaluations in place as they have implemented random plant audits to periodically evaluate controls.

Viliam Financial Services is a publicly held global company. Recently the industry has experienced a significant rate of change because of increasing regulatory focus and complexity of the company's financial products. In response to these changes, Viliam's management and board of directors have reprioritized the activities conducted by its internal audit department, including:

Senior management develops a baseline understanding of the design and current state of the entity's system of internal control by:

Management then leverages the established baseline to:

Senior management may use the baseline information to establish which ongoing and separate evaluations are most appropriate.

The senior management of Judd Co., a beverage manufacturer and distributor, focuses the organization's monitoring efforts by risk priority. In areas of high risk the entity conducts and documents a thorough review of the design and operation of controls to establish a baseline. The documentation includes a written description, flowchart, and walkthrough narrative of how each control within the high-risk area operates. Past and current control performance must also be documented with any anomalies or significant variations noted and evaluated. With risks prioritized and the baseline established, management identifies monitoring activities that can evaluate changes to the system of internal control in a reasonable period of time. The baseline aids Judd Co. in selecting more efficient monitoring activities, such as self-assessments coupled with supervisory review. Then, at intervals appropriate to the level of risk, internal audit performs periodic separate evaluations to reconfirm the system of internal control against the baseline and the effectiveness of the ongoing monitoring procedures.

Management identifies metrics that correlate to the completeness and accuracy of financial transactions to provide ongoing evaluations of established control activities. When identifying metrics, management considers the processes and sub-processes that should be monitored, and develops the appropriate measure and frequency for the evaluation.

The metrics may use the following information:

Some metrics have clearly defined allowable tolerances that have been calculated for current performance data, which may be used to highlight anomalies. Other metrics have less defined thresholds and are reviewed by knowledgeable employees for reasonableness and unusual items.

Approximately 90% of Mynarski Manufacturing employees are located at company plant sites. To monitor whether the payroll processing control activities are working, Henrik Saunders, the corporate payroll manager, reviews the plant payroll metrics. Payroll metrics include:

In his review, Mr. Saunders looks for any unusual fluctuations, such as increases and decreases in the number of employees and excessive overtime. His review is done in the context of current plant productivity and target thresholds based on historical data and planned productivity, which varies by season. If Mr. Saunders identifies any fluctuations, he investigates the underlying reasons and adjusts the process or control activities as needed.

Tony Rosco is the controller of Still Craft Foods. He uses operating measures and key performance indicators (KPIs) for major accounting and financial processes, including accounts receivable, payroll, accounts payable, and financial statement preparation. Accounts payable KPIs, for example, focus on the accuracy, timeliness, completeness, and compliance of documents received for vouching and checks prepared, with performance tracked to established targets.

Mr. Rosco leverages his knowledge of changes in the business when developing his expectations on how performance is likely to be consistent with, or vary from, established targets. In the case of accounts payable KPIs, those variances from the established targets could result from known factors, such as significant new vendors, changes in payment terms, and cash flow goals. Where results do not meet expectations, Mr. Rosco evaluates them for potential underlying issues in established control activities. Additionally he uses the KPIs to identify trends that could indicate some fraudulent activity (e.g., he sees a concentration of payments to a vendor that is new or for which he would not expect that volume).

He shares his findings with the management team, which uses the information in performance appraisals and related development programs.

As part of its ongoing evaluations, management develops and implements dashboards for reviewers to use in the ordinary course of business. Reviewers are usually supervisors of those employees with first-level knowledge and who are accountable for processes, activities, and their controls. Dashboards may include:

Langdale Manufacturing, a manufacturer of industrial machinery parts, uses a set of operating dashboards by business process, with each dashboard containing a series of tasks assigned to the appropriate managers for action. The dashboard for the production inventory process, for example, includes costs associated with tooling: where the warehouse manager checks the usage of tools during production noting how often they are needed, who requested them, and where they are purchased from.

Management then considers this information when reviewing tooling costs included in inventory. In the monthly management meetings, these dashboards are reviewed. Each of the managers responsible for specific tasks discusses recent progress and expected changes over the coming month. To the extent that an increase in tool usage was noted, management would expect that costs related to tooling would be up for the period.

Management uses technology to support the monitoring of the system of internal controls in the ordinary course of business through automated monitoring applications. Management uses the automated monitoring application to efficiently and continuously review large volumes of data at a low cost with a high standard of objectivity (once programmed and tested). Automated monitoring activities may include:

Gentoo Financial Services employs a continuous monitoring tool to perform a simple regression analysis of nonperforming loans by branch and by loan officers as one form of monitoring control over loan origination. The output from the tool allows Gentoo to look for outliers across multiple dimensions (e.g., policy, industry standards, and statistical standard deviations) and provides input for Gentoo's allowance for loan losses. Further, the report can be repopulated in either real-time or batch mode. This analysis helps Gentoo identify loan officers and/or branches that may not be following loan origination policies.

Penguin Ice, a manufacturer of ice cream, uses an automated computer application as part of its ongoing monitoring activities. One of the application's activities identifies any trends in the processing of journal entries of personnel who consistently approve entries just below their authorization limit. Management then considers this information in monthly meetings to determine if any fraud is occurring or if journal entry control activities for authorization limits need to be changed.

Management may conduct separate evaluations of internal controls over external financial reporting by:

Annually, the board of Generation Now engages an independent third party to evaluate the effectiveness of its whistle-blower program. The purpose of the evaluation is to ascertain that (1) the general counsel has reviewed the logs of all calls received and reported all calls in the quarterly progress reports to the board; (2) the internal auditor (or other independent individual) carried out the investigations into allegations, as necessary, and made recommendations to address any shortcomings in the whistle-blower program; and, (3) all parties complied with the company's policies and procedures in resolving all whistle-blower calls on a timely basis.

Annually, Bio-Adaptive's chief data officer reviews a system generated report that identifies employees who have access to sensitive financial data and information. For these employees, the chief data officer evaluates the suitability of assigned restricted access and their adherence to the standard operating policies and procedures. Based on the assessment, the chief data officer recommends modifications to existing restricted access, standard operating policies and procedures, and control activities relating to identifying and protecting sensitive financial data and information.

Gregson Grenville is a publicly held consumer products company with multiple manufacturing facilities throughout the world. Every year, the company's senior financial officers for each division visit each subsidiary's headquarters, manufacturing site, and/or sales office to gain an understanding of significant business processes at those locations. During these visits, the senior financial officer discusses procedures and controls for all relevant processes impacting financial reporting with those performing the control activities and their supervisors. In addition, a mini-audit of select control activities is conducted, the findings are documented, and the local team develops management action plans for all pertinent recommendations. In addition, findings are shared broadly throughout the organization to facilitate control enhancements at other locations, and areas of concern impact the focus of future senior officer visits at this and other locations.

Jaron and Associates provides Internet-based securities brokerage and financial services. Recently the company instituted a formal internal control assessment program (ICAP). Under this program, managers of each business unit perform a quarterly control self-assessment and certify the effectiveness of certain controls for which they are responsible.

The senior management of Jaron recognizes that self-assessment, while not completely objective, is an effective first line of defense against internal control failure. Internal audit helps compensate for the lack of objectivity in the control self-assessments by performing periodic audits and comparing the results to the self-assessments.

ICAP allows management to concentrate its ongoing evaluation efforts on several issues:

Now Jaron and Associates is better able to focus its separate evaluation efforts on a prioritized risk basis and modify ongoing evaluations where necessary.

Management uses an appropriately staffed and adequately trained internal audit function to provide an objective perspective on key elements of the internal control over external financial reporting. Internal audit reports are distributed to senior management, the board of directors, and others who are positioned to act on the report's recommendations. Internal audit's separate evaluations may be influenced by:

Maxwell's internal audit considers management's assessments of the likelihood of the risks of material omission and misstatement due to fraud, its planned responses, and the control activities to mitigate these risks when planning its audit projects. Internal audit selects and develops its monitoring activities including the scope, nature, and timing of its evaluations based on its views of the assessed fraud risks and management's planned responses. Internal audit reports these identified fraud risks, along with management's responses and its planned approach, to the chief audit executive and audit committee. Internal audit also discusses the results of its fraud procedures with the external auditor. As part of its approach, internal audit compares any noted fraud incidents to business unit management's fraud risk assessment to identify and evaluate any shortcomings within management's risk assessment process.

Lee-Basker Parts designs, manufacturers, and distributes precision components and assemblies for aerospace applications. From time to time the board directs the company's internal audit department to perform separate evaluations of specified high-risk business processes that impact the entity's financial statements. The scope and frequency of these evaluations depend primarily on the significance of the related risks and importance of the controls in reducing risks to an acceptable level.

Subsequent to management's input, it is up to the chief audit executive, Maria Geide, to determine whether the internal audit department adequately understands the process, the overall internal control structure, and the objectives of the review.

Once the review is complete, Ms. Geide submits a report on the process controls to senior management and the board covering the scope of the work (including identification of the controls evaluated), a description of the major risks and the appropriateness of the controls, a list of identified deficiencies, and management's response and proposed remediation.

Management obtains and reviews periodic information from outsourced service providers to detect any changes in activities that impact the entity's system of internal control over external financial reporting. Information obtained may include:

To determine what impact any identified changes may have on the entity's system of internal control over external financial reporting, the following may also be assessed:

Based on management's review and findings, it may be necessary to reassess the separate evaluation activities over the outsourced service provider.

Finlayson Home Works supplies materials used in residential construction. This public entity has outsourced its payroll activities for a number of years to a reputable payroll services provider. The chief audit executive, Rolf Brunner, obtains an annual service auditor's report detailing the internal controls at the service provider. Mr. Brunner then compares the current report to past reports to determine whether there have been any changes in relevant controls that could impact the judgments made on planned monitoring activities over the payroll process. The current report indicates some key changes in the payroll service provider's software and several negative test results in priority risk areas. As a result, Mr. Brunner has the internal audit department of Finlayson Home Works perform a reconciliation of the payroll service provider's processing results to evaluate if additional separate evaluations of the payroll service provider may be necessary.