Stop Auditing What They Tell You

Pattern Chaser: Audit Analytics for Internal Audit

PATTERN CHASER

Audit Analytics for Internal Audit in Financial Services

Process owners describe reality their way. Data tells it another. The gap between the two is where your real findings live.

Hey Reader 👋

When nobody’s watching, do people behave the same way they said they would?

Economist Paul Feldman tested this. In the early 1980s, he started delivering bagels to offices around Washington D.C. on an honour system: take a bagel, leave money in the box. When he was physically present, payment rates hit 95%. The moment he left the room, they dropped.

Observation changes outcomes. Remove the observer, and the gap between what people say they do and what they actually do widens.

That gap has a name in economics: information asymmetry. And for auditors, it’s the default condition of every engagement.

In today’s issue:

↳

Why walkthroughs hand narrative control to the wrong people

↳

A 3-step framework for building your own picture before the first interview

↳

Plus: how to apply this even when you don’t have automated data access

Examples in this newsletter are composites drawn from industry experience, peer conversations and published research. Not attributable to any single organisation.

DEEP DIVE

Why Walkthroughs Start You in the Wrong Place

The standard audit approach begins with interviews. You sit down with process owners, walk through documented procedures, ask follow-up questions, and build your understanding of how things work.

This approach has a structural weakness: it starts from management’s narrative.

The people you’re auditing live inside their processes. They know the workarounds, the informal approvals, the spreadsheet on someone’s desktop that never made it into the official workflow. Some of this is deliberate. Most of it is simply unconscious. They don’t describe what they’ve stopped noticing.

When your first act is to ask “How does this process work?”, you’ve handed narrative control to the party with the most to gain from a favourable description. Every subsequent question is anchored to their framing.

The fix isn’t to stop interviewing. It’s to change the sequence. Build your own picture of reality first, then use management’s narrative to identify the gaps.

The 3-step framework

Step 1

Baseline the reality

Before your first detailed interview, request data extracts from relevant systems. If you don’t have live access (most audit functions don’t), send the data request at the start of scoping, not during fieldwork. Give IT or the business enough lead time to extract it. The point is to have the data in hand before management’s narrative shapes your understanding.

For transaction-heavy processes, pull identifiers, timestamps, users, and status fields. For policy or governance audits where transaction data doesn’t exist, pull the artefacts: approval logs, committee minutes, email trails, training records, exception registers. The principle is the same: evidence of what actually happened, not someone’s summary of what should have happened.

Run exploratory analysis: who actually touched these items? When? How frequently? What patterns emerge?

Step 2

Flag the four gaps

Look for patterns that signal incomplete descriptions:

↳

Time gaps: long delays between stages reveal manual steps or workarounds

↳

Dead ends: items that enter but never complete the described path

↳

Unexpected participants: names or departments not mentioned in the walkthrough

↳

Volume anomalies: counts or frequencies that don’t match what was described

For non-transactional audits, the same logic applies: committee meetings that didn’t happen when they should have, policies reviewed by people outside the stated approval chain, exception approvals with no documented rationale. These aren’t findings yet. They’re red flags that tell you where to dig.

Step 3

Question with evidence

Now go back to management with specific, evidence-driven questions.

Not: “Are there any other steps in this process?”

Instead: “20% of transactions enter the approval system but never reach ‘Paid’ status. Can you walk me through what happens to those?”

Or for a governance audit: “The policy says the committee meets monthly, but the minutes show six meetings in the last twelve months. Two of those had only two attendees. What happened in the other months?”

The evidence gives you both the confidence to ask and the specificity they can’t easily dismiss. You’re no longer asking management to volunteer information. You’re asking them to explain something you’ve already observed.

Feldman’s bagel experiment ran for years. He collected data on over 7,000 office deliveries. What he found: payment rates dropped in bad weather, around holidays, and when office morale was low. The data told the story that interviews never would have.

You didn’t become an auditor to document what someone tells you.

You became one to find what they’re not telling you.

The walkthrough is their story.

The data is yours. Build it first.

WHAT’S POSSIBLE

The Pre-Interview Evidence Request

Most audit teams treat data requests as something that happens during fieldwork, after the process has been explained to them.

Flip the sequence.

You don’t need live system access or automated feeds to do this. Most functions don’t have that. What you need is to move your data request earlier in the audit cycle. At the start of scoping (not during fieldwork), ask IT or the business to extract a minimum viable dataset: transaction or activity identifiers, timestamps, users, and status fields. Give them lead time. The data needs to arrive before your first interview, not after.

For non-transactional audits, request the artefacts that show what actually happened: meeting logs, approval records, exception registers, training completion dates, policy review histories.

Then run a 30-minute exploratory review before your first interview. Plot the timeline. Look for the four gaps. In my experience, this step alone surfaces surprises in the majority of audits. Departments that shouldn’t be involved show up in the data. Time gaps that management would describe as “a day or two” turn out to be three weeks.

The information asymmetry hasn’t disappeared. But you’ve closed the gap enough to have a conversation grounded in evidence rather than narration.

BEST LINKS

Worth your time.

🔍 THE PATTERN

Writing “The Market for Lemons”. Akerlof’s Nobel Prize lecture on information asymmetry. The seller knows the car is a lemon; the buyer doesn’t. The same dynamic plays out in every walkthrough where the auditee controls what you see. His solution: find signals that reveal true quality. For auditors, that signal is data.

🧠 THE MIND

What Users Say vs What They Do. Nielsen Norman Group on the gap between self-reported behaviour and actual behaviour. UX researchers learned this decades ago: watch what people do, not what they say they do. The same principle applies to every process walkthrough you’ll ever sit through.

📊 THE PROOF

ACFE Report to the Nations 2024. 43% of occupational fraud was detected by tips. External auditors detected 3%. Internal audit detected 14%. The people closest to the process know what’s happening. The formal audit process often doesn’t. That’s information asymmetry in the data.

🗞️ WHERE IT’S HEADING

AI Went From Assistant to Autonomous Actor. 49% of organisations are blind to machine-to-machine traffic. AI agents now outnumber humans 82:1 in some environments. If your audit function can’t see what the agents are doing, you’re facing information asymmetry at a scale the profession has never dealt with before.

THAT’S A WRAP