Building a Better Auditor: Questioning Test Results

A blood pressure cuff was then applied to my upper arm. As it barely inflated, I didn’t have much confidence in the accuracy of the results. The medical assistant and physician assistant nodded affirmingly as they jotted down 120/75, a normal measurement. But I knew it shouldn’t be normal. For several months, I had experienced high blood pressure and had been taking daily readings at home.

I finally spoke up. “That doesn’t sound right,” I reasoned. “My pulse and blood pressure have been high lately.” The physician assistant shot back an instant rebuttal. “Your blood pressure measurement is in a normal range. There’s no reason to think it’s not accurate.”

“Yes, but my blood pressure tends to not be in a normal range. So a normal result is surprising,” I gently pushed. “At my visit two months ago, it was really high.”

The physician assistant started going through my chart, seeking support for her position. “Your blood pressure was 120/80 last August,” the physician assistant replied. “This is in line with that.”

Auditors pride themselves on being objective, a critical trait to meet stakeholder needs. And since audit results are usually based on test results, or data, it’s hard to imagine how a lack of objectivity could impact findings. After all, data is data, right? Unfortunately, it can be easy for even the most seasoned auditor to fall prey to bias, even when taking a data-based approach.

As humans, we tend to see what we expect to see. Sometimes this means not questioning results that are in a “normal” range — such as my blood pressure readings. As auditors, giving in to this line of thinking can limit the value we provide to our clients.

For example, an auditor might perform a standard variance analysis and find that results are comparable to the prior year. However, perhaps that account balance should have changed significantly from the prior year. The lack of change should be a red flag, but an auditor not thinking critically might miss this sign and instead perceive it as an indication that no further testing is needed.

In other instances, auditors may receive inaccurate or incomplete data. And — as in the case of my blood pressure reading — that data might provide reasonable enough results that the auditor might fail to perceive that they are missing part of the puzzle.

There are several techniques auditors can use to avoid falling into these traps:

• Listen to your clients, recognizing that they are the experts in their business. If the test results don’t align with the client’s expectations, a critically thinking auditor might need to dig deeper or even design additional tests. If the medical assistant and physician assistant had listened to me, they would have quickly realized that the results were inaccurate.
• Consider internal and external factors that could impact results before performing data analytics. For example, during times of short staffing, it would be likely for customer service ratings to decrease. If the customer service ratings stayed the same or even increased, a skeptical auditor would identify a red flag indicating further review, but a biased auditor might see those same results as indicative of a reasonable metric and move on.
• Verify the accuracy and completeness of data. This can be challenging to do as we are often relying on humans to pull data for us, which creates a risk of error. However, auditors should take all reasonable steps to validate data, comparing it to other data or systems where possible.
• Stay humble. Recognize that you are capable of error. Constantly challenging our thinking allows us to pivot when needed and take a more holistic view of our analytical and other testing procedures.

In the case of my medical visit, thankfully my doctor applied the techniques above. He retook my blood pressure and found it to be high. The initial reading had been due to an inexperienced medical assistant not performing the test correctly. As for the pulse, it turns out the medical assistant had been reading the pulse oximeter upside down and misread 99 as 66. The data was correct, but the interpretation was flawed!

Even though it was corrected in the end, my experience didn’t leave me with the utmost trust in my medical team. Similarly, failure to think critically, validate data, and set reasonable expectations for data analytics results could damage the trust our audit clients have in us. We must be constantly vigilant of our biases so they don’t impact the value we provide as trusted advisors.