Building a Better Auditor: What AI Can’t Do

At AuditBoard’s Connected Risk summit in London in May, Richard Chambers said, “Generative AI will be positioned to do assurance better than we can really soon.”

When it comes to AI, science fiction future is fast becoming science fact. It looks like we are at an evolve-or-become-extinct moment. What will that look like?

A thought experiment — imagine a future justice system where there is:

• No judge.
• No defense.
• No prosecution.
• No jury of your peers.

There’s just push-button justice delivered by AI.

“You’ve been accused of a crime, citizen. But don’t worry, the verdict will be delivered in an instant. Just push the Justice Button, and you’ll be found guilty or not guilty.”

Would you like to live in a future like that? Neither would I!

The Good News

As Aristotle noted, “Mankind is a social animal.” The technology we create often is designed to facilitate one essential drive in all of us: the desire to connect. AI will almost certainly be positioned to provide assurance better than we can soon. But that will leave internal auditors with more time to do what really makes a difference: Building audit client relationships on a foundation of empathy, trust, and a desire to create value.

When AI listens to you, it is because that is what it was created to do. When another human listens to you, it’s because that is what they freely chose to do. And they could have freely chosen not to, leaving you talking to yourself. You need to talk in a way that makes your audit clients want to listen, or you will be in big trouble in the AI-assurance future.

The Obstacle

David Hume wrote, “When men are most sure and arrogant, they are commonly most mistaken.” Many internal auditors don’t know how to disagree well. Ask what makes them most stressed about their audit fieldwork conversations, and many will tell you it’s when their audit client disagrees with them.

My friend Jon Taber experienced a similar situation firsthand. After completing a payroll audit for a centralized function that oversaw more than 30 countries, he identified an observation related to overtime monitoring, which was ultimately deemed a low-level issue. Two months later, while conducting an audit in one of those countries, payroll was again within scope, and the concern about overtime came up again. However, management disagreed with the observation, claiming inconsistent treatment of issues across reviews. And they had a valid point.

If you are a disagreement-averse internal auditor, you need an urgent mindset change. Because disagreement is something to be celebrated. Think about it.

If your audit clients agreed with everything you were saying to them — if there was no discomfort, no doubts, and no objections — would you really be telling them anything of any value? Disagreement happens when we are challenged in ways we don’t like. If you are not challenging your audit clients about their approaches to governance, controls, and managing the risks in the way of their business objectives, you really would be obsolete. So, when it comes to being valuable in the AI-assurance future, two key skills will be:

1. Seeking out disagreement with your audit client on your findings.
2. Disagreeing well when you do find it.

But when you ask internal auditors how they prepare for a finding update meeting with an audit client they know will disagree, many say this: “I check my facts and figures.” This is not preparation for disagreeing well — this is preparation for defensiveness.

The Solution

In 1859, John Stuart Mill wrote On Liberty, his timeless philosophical treatise on the nature and limits of power. In it, he gives us the idea of “epistemic humility.” This is the ability to be humble about what it’s possible for you to know as a human being experiencing life from your own personal perspective. It is a recognition and acknowledgement that your knowledge and understanding has limits. And it is a rallying cry for open-mindedness, intellectual growth, and disagreeing well.

Mill provides three tenets for internal auditors to follow:

1. All human judgment is fallible. You can make mistakes, even when it comes to your audit findings.

Take, for example, the scenario in which Jon identified a correlation of system downtime with a specific internet service provider. When sharing that with the business, he was met with the following statement: “That said, this is still not enough evidence to infer that this service provider or even the internet connection is the root cause of the issues.”

So true. Correlation is not causation. Recognizing your fallibility will make you open to hearing your audit client’s opposing viewpoints because they may reveal errors in your thinking. The result? You will be closer to the truth! 

2. There are often partial truths in opposing viewpoints. Even if your audit client’s opinion is partially false, it often contains some truth, too.

In one case, Jon and his team were working on a project that included the recovery of specific business costs. A stakeholder questioned the review’s purpose, arguing it was not something that would result in a high return on investment. While cost recovery was within the project’s scope because of associated risks, the stakeholder raised a valid point: Shouldn’t return on investment be a consideration for certain projects?

By welcoming and listening to opposing views, you can gain a more comprehensive understanding. The result? You will be closer to the truth!

3. Free discussion is a necessity, not a nicety. Free and open discussion is indispensable for the pursuit of truth.

Jon experienced this firsthand when a stakeholder from the business let him know that the information being provided during a cash handling project was not useful. Initially taken aback, Jon took the time to understand the stakeholder’s perspective and later successfully convinced the director of the need to review and improve that area.

When your opinions clash with the views of your audit clients, you get to test the robustness of your ideas and strengthen your understanding. The result? You will be closer to the truth!

The Call to Action

Mill’s three ingredients for epistemic humility, when put together, make up the chambers, valves, and conduction system of the beating heart of disagreeing well. AI will soon be positioned to provide assurance better than we can. But if you master epistemic humility, your value as an internal audit agent of change will be equally indispensable.

Jon Taber, CIA, CPA, CFE, CFF, contributed to this blog post.