Mind of Jacka: Too Much/Too Little Information

Of the following, who was a world boxing champion?

1. Lyndon Johnson
2. Ronald Reagan
3. Jimmy Carter
4. Franklin Delano Roosevelt
5. Abraham Lincoln
6. Pat Paulsen
7. None of the above

Here’s the answer in a format that will make it harder for you to cheat. (Not that any internal auditor would do that.)

˙ɹǝʇɹɐƆ ʇɐɥʇ ʇou ʇsnſ ˙(Ɛ) sᴉ ɹǝʍsuɐ ǝɥ┴ ˙(ㄥ) pǝʞɔᴉd ǝldoǝd ʇsoW

Internal auditors love information. It is our lifeblood. And that means we love data. But how much data is enough?

Yeah, the eternal question. And the honest answer is… well, it depends. It depends on why we need it. It depends on how we want to use it. And it depends on how much of that data we can really turn into information.

But, no matter what, we always seem to want more.

Here’s another question/quiz/story. (And I may have told you this one before, but it is worth repeating.)

In 1974, a study was conducted using horse race handicappers. Eight professional handicappers were studied to determine how well they predicted the winner of horse races. The information they received was based on real races, but the names of the horses were deleted so the handicappers would not be swayed by that information. They were told they would handicap forty races in four rounds of ten.

For the first ten races, the handicappers were allowed any five pieces of information they desired. This could be the weight of the jockey, the conditions of the track, prior race results, whatever five pieces they wanted. They made their picks and were given the results.

With this information they were 17% accurate. There were 10 horses in each race; blind chance would have resulted in a 10% success rate. So, 17% was pretty good. In addition, they were 19% confident with their predictions.

This was repeated with 10 pieces of information, then 20 pieces of information, and finally, 40 pieces of information.

Here’s the question. How much better were they at their predictions and what was their new confidence level?

[The author takes a quick time out while the readers ponder the question.]

Pencils down.

Their confidence went from 19% to 31%. A great improvement. Their accuracy went from 17% to (wait for it) 17%. They were no more or less accurate with up to eight times the information.

Why do we want/feel like we need more information?

Two important lessons here. First, when it comes to data, internal auditors always want more. (That is why we love analytics and always promise… promise… we’ll start using them more in the future. Another sermon for another day.) But do we really need more/all? Sure, data is now easier to get. But, even if it is easy to get, do we really need it?

Do we require additional data to ensure audit assurance or because we just want self-assurance?

Lesson number two comes from the question at the start of this post. Most of us answer that question incorrectly because, while we all know about U.S. presidents, few of us know a lot about world boxing champs.

Internal auditors can be experts in a lot of things. And there are specific areas where we should be experts. (For example, controls, risk, and processes.) But we can’t know everything. And we have to be sure that the expertise we bring to the table matches the expertise that is needed. If we need to know about the presidents, then the common knowledge we have about presidents may be sufficient. But, if we also need to know about boxing champions, then we’d better learn about boxing champions. Or we have to be willing to admit what we don’t know and find the experts who do.

Otherwise, no matter how much data we gather, it will lead to false conclusions, simply because we refuse to admit we don’t know what we are talking about.

(As so often happens, thanks to Seth Godin for the concepts that got me started down this road.)