Journal · 18 September 2025
Session replay will not save a dishonest funnel
Watching a thumb hesitate on a button is instructive. It is not an audit of whether checkout_completed fired twice.
Session replay tools arrived as a kindness to product managers who were tired of guessing. Used narrowly — a dozen sessions around a known drop-off — they can show a keyboard covering a primary action, or a legal checkbox that arrived after the scroll. Used as a substitute for event integrity, they become expensive television.
Funnel Integrity exists because the count and the film disagree. A replay may show one person completing a purchase. The event log may show three completions: a retry after a 500, a duplicate listener, and a server echo. If you quote the funnel, you are quoting the log. The film cannot vote.
There is also a privacy cost that the house will not shrug at. Replays in a United Kingdom consumer app can capture names, messages, and the occasional card fragment a masking rule missed. If you do not have a purpose, a retention period, and a masking review, you should not keep the film. Qualitative research can be done with recruited interviews and far less residue.
Our advice is unfashionable. Repair the event contract first. Rebuild one conversion path from raw events with identity and retries in view. Then, if a remaining drop-off is still mysterious, watch a small number of sessions with consent and masking you can defend. Do not invert the order because the replay product has a prettier homepage than your warehouse.