I replaced the budgeting app that runs my household finances. The part that decides whether such a project is a success is not the interface — it is the importer: the code that reads a bank's export and decides which rows are new, which are already there, and which are a duplicate wearing a different hat.
Unit tests for that code pass easily, because you write the tests from the same understanding that wrote the code. So before trusting it with anything, I ran every bank export I had ever archived — two years of them, across nine accounts — through the new engine, into a scratch database, and compared the result against the system it replaces.
What the exercise actually is
Three inputs: the folder of archived exports, a read-only snapshot of the old system's balances and row identifiers, and the old pipeline's mapping from filename to account. The test imports every file in the order the files arrived, then compares, per account: the balance, the set of bank row ids, and the bank's own running balance where the export carries one.
None of this lives in the repository. The files are real statements, so the test is skipped unless you hand it paths at run time. That single decision — inputs as parameters, not fixtures — is what made it possible to have a test this valuable at all.
What it found
Identical twins collapsing. Two charges on the same day, same amount, same description, are a single hash. The importer kept one. The bank's balance column proved there were two. The fix is a numbered suffix on the second occurrence, counted only across rows that have posted.
Rows that post late. A transaction pending at export time can post earlier than the newest row already imported. A strict cutoff strands it forever. One account sat $1.61 off the bank for a month because of this. The fix is a short look-back window behind the cutoff, with a content guard so a row already present by date and amount is not added twice.
A loan's balance compared with the wrong sign. Loans report the amount owed as a positive number. Compared unflipped, every loan looked off by exactly twice its balance.
A balance check run against history that does not exist. Comparing a file older than the account's opening balance produces a difference every time and teaches you to ignore the check.
A guard that counted its own successes. The duplicate guard counted rows the file's own ids had already matched, which made a genuine second same-day payment look like it was already there.
A 500 where a 413 belonged. Not the importer: the shared error handler turned "file too large" into "internal server error", which sends you looking in the wrong place entirely.
Why the differences that remained are the interesting part
After the fixes, every account with a balance column matched the bank exactly, or was off by a constant that was present from its very first file. Each constant had an explanation: rows dated before the old system's starting balance was really taken; one row the old system had received from outside the archive; an account with no export at all, kept by hand.
That is the outcome to want. Not "zero differences" — differences you can name. A reconciliation that comes out clean on the first attempt usually means the comparison is not looking hard enough.
What I would do again
Write the comparison before the confidence. It cost an afternoon and it moved six defects from "discovered in production, with real money" to "discovered on a Tuesday, in a scratch database".