Basics: The Case for the Deep Dive

Articles Robert McKinney Aug 08, 2022

Auditors should stress depth and thoroughness in assessing process controls.

Internal audit should avoid the pitfalls in testing that can prevent them from correctly assessing process controls.

Auditors, like most businesspeople, are frequently under pressure to complete their work expeditiously and adhere to a budget. Sometimes that can lead to testing decisions that consume fewer audit resources and seem to satisfy assurance standards but don’t go deep enough to uncover control failures. Internal audit should avoid the pitfalls in testing that can prevent them from correctly assessing process controls.

Adequate Sampling

Sample testing, one of the most common fieldwork approaches to control testing, consumes a great deal of audit resources. Auditors frequently adopt testing guidelines that shorten the testing required to evaluate control effectiveness. Standard sampling guidelines such as “sample 25 items for a daily control” or “sample 30 items for large populations” are really sampling shortcuts that mask the level of assurance and can lead to sample sizes that don’t provide a true picture of how a control is performing.

For example, when testing a control that was performed 1,000 times over the course of an audit period, testing a sample of 25 outcomes implies a confidence interval of 68% and a 10% margin of error. If the auditor has reason to believe the probability or the consequence of control failure is low and it’s enough testing and assurance, then this can be perfectly suitable.

But what about cases where the risk is high? Perhaps there was a recent operational loss event or significant changes to systems or process controls and the auditor is skeptical about control effectiveness. In those cases, using standard sampling guidelines may result in sample sizes that are not large enough. For example, for a control assessed to be high in control risk, the auditor may adjust sampling parameters for a higher confidence interval, say 90% with a 10% margin of error. This implies a sample size of 64, which is more than double.

Clearly there is a cost associated with larger sample sizes, and most auditors cannot afford a high level of assurance in every control test. So, internal auditors should use a risk-based approach to choosing assurance levels and sample size selection. Auditors should be deliberate about the level of assurance they are providing so they can adequately protect the organization.

Not every control test lends itself to statistical sampling, and there is a role for judgmental sampling in audit fieldwork. Process outcomes that are not normally distributed, such as small populations or samples used in analytical reviews, are situations where judgmental sampling may be appropriate. Even if auditors cannot always afford to test extensively, they should be deliberate in providing an assurance level. Use a sample-size calculator to get a statistically meaningful sample size calibrated to the level of control risk and audit budget.

Audit Automation

Automated auditing can make this more efficient. Although some investment is required to set up an automated audit routine, the payoff can be quite pronounced. Automation permits auditors to examine 100% of the data under review, so they can move from sample testing to population testing. In addition to a thorough assessment made possible by larger sample sizes, dealing with large data sets also offers the opportunity to better understand process behavior and controls performance through data analysis.

Understanding Audit Evidence

Auditors must understand the audit evidence to evaluate it. Misunderstood client explanations, unclear terminology, and uncertainty as to how reports were generated can all lead to incorrectly interpreting audit evidence. This can be particularly acute for external auditors, where there is less organization-specific knowledge. Internal auditors should keep this in mind when overseeing external audits.

Pressure to complete audits increases the probability of misinterpreting audit evidence. Internal auditors are sometimes predisposed to believe controls are functioning effectively, and getting through the evidence quickly allows them to stay on budget.

To avoid this pitfall, internal auditors must maintain their professional skepticism, question the meaning of unclear terms, and understand how evidence is generated. They should not take the client’s explanation at face value, but rather square it with other information they have collected and their understanding of the processes and systems under audit. Auditors need to take the necessary time to understand what they are looking at.

Challenging Control
Input Assumptions

Closely related to understanding audit evidence is understanding control assumptions, particularly when it comes to sources and completed information. For example, when evaluating an automated reconciliation of one system’s data to another, it is not enough to examine the reconciliation output and resolution of reconciling items. Auditors must also be certain of the sources of the data being reconciled. The auditor may ask:

What is the method of data transference to the reconciliation tool from each source system, and is it functioning with the latest data?
Is the data pulled directly from the system being reconciled, or does it pass through an intermediary data store where the original data can be transformed or modified?
How can internal audit be certain it is looking at all of the data?

It may not always be practical to verify the scope and authenticity of the inputs to each control with 100% certainty. The working assumption may be that controls function with the intended inputs, but auditors need to challenge the control input assumptions especially when it comes to systematized controls.

If the audit budget does not enable extensive testing to verify sources and completeness, auditors should gain assurance through examination of system configuration, automated scripts, one-off manual reconciliations to source systems, or at minimum, inquiry of technology support staff.

Scope Limitations

Because other audits cover tangential processes, limited-scope audits are necessary to focus on a particular area of risk to best apply scarce audit resources. Limiting audit scope requires judgment, however, as it is possible to eliminate the controls testing that is key to the performance process. Not testing enough controls is a common planning error and can result in the awkward situation where operating problems reveal themselves soon after an audit is completed. It is not enough to “touch” a process through a limited-scope audit; it is critical to audit the right controls that are key to process outcomes.

To avoid this pitfall, internal audit should conduct a thorough process risk assessment when planning the audit. Auditors should take the time to gain a solid understanding of which controls are the key controls and make sure they are included within the audit scope. A good test to determine key controls is considering if process key performance indicators or desired process outcomes can be achieved without the control.

Control Design

Testing to determine whether controls function effectively only provides positive assurance to the extent that the controls are designed well in the first place. Without an appropriate assessment of the controls design, some process risks may not be mitigated.

Auditors should do a thorough design assessment before testing. They should begin by identifying process risks and work toward related controls, rather than the other way around. Creating a risk and controls matrix is a good way to ensure every identified risk has associated controls. Additionally, to be designed appropriately, controls must address all of the risk elements. For example, if process completeness and timeliness is important, then controlling only population completeness is not a well-designed control.

Assessing controls design requires thoughtful contemplation and analysis. Internal audit should not shortcut this step by jumping directly into controls testing. Auditors should spend the planning time and effort to consider design.

Go Deep

For auditors, the concept of providing reasonable assurance sounds, well, reasonable. But what audit clients are really looking for is absolute assurance, which is typically not practical. However, auditors should strive to come as close to that standard as possible. Internal audit clients, whether internal or external, are engaging internal audit to learn if processes work as intended and to know if outcomes are reasonably predictable and in control. Auditors should give clients what they need by stressing depth and thoroughness in their work practices.

IIA Resources

ROOT CAUSE ANALYSIS FOR ENHANCING INTERNAL AUDIT EFFECTIVENESS

ANALYZING & IMPROVING BUSINESS PROCESSES (THEIIA.ORG)

Robert McKinney

Robert McKinney, CISA, is head of Internal Audit at Point72, L.P. in New York.

Learn more with our other resources

May 23, 2024

Access the Digital Edition

Read Now