Postmortem Blues​

In coming up with a list of potential excuses, as well as discussing how those excuses result in our not being able to get to the real reasons things may not be going well, I think I made an interesting (and potentially erroneous) assumption — that we actually spend time after audits determining what went right or wrong. In fact, there is a dandy chance that most of us do not really go back to see if things actually went right or wrong in the first place. Sure, we may have a gut feel, but do we actually know, shy of various review notes and a superior's comments — sometimes at the end of the audit; sometimes at the end of the year —  that the work for any one audit was or was not up to par?

There is an incredibly effective tool at our disposal that we ignore because we are so focused on moving forward to the next project — the postmortem meeting.

When I got the job as manager over Farmers Insurance's audit department in Phoenix, I inherited a very good staff that had been run by a not-so-very-good manager. (The gods of understatement currently smile down upon that sentence.) One of the first things I needed to do was figure out if my initial perceptions were correct.

Unfortunately, time and circumstance took control of destiny, and I was not allowed to leisurely stroll among them interjecting my brilliant pearls of internal audit wisdom as they diligently performed their all-important work. No, we got hit with a large number of fraud investigations. And so, in relatively short order, all five of us were spread across the three-state region of Arizona, New Mexico, and Nevada, pursuing the potential malfeasances that happen in cash-based economies.

Fast forward a couple of weeks.

It was Friday. I set up a meeting after lunch. The meeting had a single purpose: Each person (myself included) was to talk about the investigations they had performed. I laid out the ground rules: This was not about finding fault with anything anyone had done, no one was going to get in trouble — unless they did something like taking a bribe from the potential fraudster — and the ultimate purpose was to provide a learning opportunity for everyone. How had the fraud investigation been conducted? What problems were encountered? What snafus happened? What went well? What innovative approaches were used?

We didn't really have a name for it, but we were effectively holding a postmortem over the work we had been doing for the last few weeks.

It went really well. It was a good group that, even though they had been in a really bad situation, had gelled as a team. They talked openly about what went wrong and right, provided input to each other on how to do better next time, made suggestions for how to move forward on anything that was still open, and, in general, showed the power that can exist when a group trusts each other and wants to make things better.

As we left at the end of the meeting, the supervisor pulled me aside and said, "I knew we needed to do something like that; I just never felt like we had the time."

I'm not going to pull out the old chestnut about never having time to do things right, but having time to do it over (oops, looks like I just did), but that is kind of what was going on here.

We never have time to go back and see what we have done wrong or right, but we always have time to make the same mistakes over and over again.

The last couple of posts, I've been quoting from Annie Duke's book Thinking in Bets. I've been focusing on her descriptions of the postmortems that occurred after games, and how they led her to being a better player. These after-the-game analyses were not about how luck and circumstance had turned for or against her, but open discussions with fellow players — players she could trust to provide honest feedback — to gain a better understanding of the choices and strategies that were involved, all geared toward making the group better poker players.

If we truly want to make our audit work better, we need to take a similar approach, using postmortems and open discussion to learn what went right, what went wrong, what should be changed, what should stay the same, and just figuring out how to make internal audit better.

And let me hastily note that I'm not talking about the manager or the supervisor or the lead auditor or whomever-is-in-charge pulling together a quick compilation of review notes and other feedback; calling the auditor into his or her office or cubicle and providing a synopsis of things that popped up; and finishing up with a "Keep up the good work," "Do better next time," or "You may want to start brushing up your resume." I'm talking about a thorough open investigation of the strategies and approaches that were used in the audit. And I'm talking about a meeting that includes more than just two people — that brings together the whole group with the intent to ensure everyone grows.

There are two major impediments to pulling off this kind of postmortem — not having time and not having an open and honest atmosphere. Time you can find. How much work is really getting done on Friday afternoons? How about just establishing that every Friday afternoon there will be a meeting to talk about the work that is being done?

But that second thing — openness and honesty — is a bigger and much more significant problem. If the team is not willing to be open about what is happening, then conducting postmortems is the least of your issues. The audit department that is not built on honesty, trust, and respect may as well grab the audit charter, fold it up, and crawl away right now.

I'd love to tell you that the meeting we had in Phoenix became a common occurrence. Unfortunately, it didn't happen. Yes, we were able to pull it off occasionally. But it was much more common for us to succumb to time constraints, fatigue, and (to be quite honest) laziness. But whenever we did get the gumption to hold such meetings, they were always a good thing.

Invest in the time to have the entire group talk about what has gone right and wrong with the audit work that is being conducted with the idea of increasing the odds of things being done right the next time. You will not be sorry.

In the above, there's a lot of discussion about the importance of an audit team that is willing to talk to each other. But best practices in this area actually require more. We'll get there next time and eventually talk about what gets talked about, as well as get to the point that really got me started on all this.