What our import gets wrong
PocketVault Finance reads your bank's text messages to fill in your spending. Sometimes it reads one incorrectly, and sometimes the message never arrives. Here is how often each happened the last time we counted, and how we counted.
Parsed wrong
4.3%
5 of 116 95% CI 1.9–9.7%
- What counts as a failure
- A message that states a payment but is not recorded, is recorded with the wrong amount, currency or direction, or a payment recorded from a message that never described one.
- How it was measured
- 500 messages drawn at random from the inbox with a fixed seed, then read and labelled by hand before the importer was allowed anywhere near them. 116 of them turned out to describe a payment.
No message arrived
21.6%
8 of 37 95% CI 11–37%
- What counts as a failure
- A line on a bank statement for which no text message ever arrived, so there was nothing for the importer to read.
- How it was measured
- Three of my own statements — one Axis savings export and two ICICI histories — checked line by line against the same inbox. A line counts as covered when a message from that bank, dated within three days, states the same amount.
Both figures come from one real Android inbox, 9,109 bank messages received between March 2022 and August 2026, covering both an Indian and a US bank.
What it found
A measurement is only worth publishing if it is allowed to fail. This one did.
- Three Google card authorisation holds were recorded as $1.00 payments. A hold of that kind is released, not charged, so nothing should have been recorded at all. The shape appears 23 more times in the same inbox.
- Two Axis fee notices were recorded at ₹25 when the body said the charge excluded GST and the statement showed ₹29.50. Both are now read with the tax added.
- All eight missing statement lines were bank-initiated postings under ₹500 — fees and small interest credits — while the payments the account holder made themselves did come through. From 1 January 2027 the RBI alert mandate covers transactions above ₹500, so a ledger built only from texts will thin out at the bottom by design.
Before you quote these numbers
- This is one inbox, mine. Your banks, your alert settings and your phone will give a different number.
- The inbox is not published, and never will be — it is a real person's bank messages. What is published is the method, the counts and the fixes.
- The two figures answer different questions and do not add up. The first is about reading a message correctly; the second is about a message existing at all.
- The fixes above came out of this very sample, so re-running it now scores better than it should. The next figure needs a fresh draw to mean anything.
Why we publish this at all
Every finance app claims its import is accurate, and none of those claims can be checked. Ours can: the definitions above tell you what we counted, and the app shows you every message it could not read instead of hiding it. If a figure here only ever improves, stop believing it.