Reconciliation: A game designed to frustrate the player

We have previously covered bank transfers as a payment method. Stripe (where I work) recently released support for bank transfers in Japan (where I live). It is theoretically part of my job to be able to explain why bank transfers are different from most payment methods. It is not generally speaking part of my job to create manga-inspired video games to make that explanation interactive, but I ended up doing that anyway; go on over to KeshikomiSimulator.com to play it.

What follows is the long-form written explanation, in case you don’t want to viscerally experience excruciatingly work-like pseudo-productivity for an hour.

(My usual disclaimer applies here: this is directly relevant to my professional interests, obviously, but both Bits about Money and Keshikomi Simulator are personal projects rather than endorsed by my employer. I’d like to acknowledge JT and Kay for helping me build Keshikomi Simulator; it would be a lot worse without their contributions or those of our illustrious designer.)

Reconciliation: your money has insufficient bits in it

How much human attention should be paid to money? Quite a bit, say most societies. How much human attention should be paid to individual money movements? Asymptotically approaching zero. Money is, after all, just a message between parties about their desires for the future, and while the content of that message might have involved the work of ages, technology has driven down the marginal human effort require to move a message from point A to point B to a number so low it is undetectable compared to historical scales.

An interesting habit I have when reading sweeping sketches like the above is thinking of counterexamples. Which sorts of messages still have material delivery costs? Why do they? Will those continue into the future?

Japan manages to spend about $500 million a year on telegrams (telegrams!), which is a surprising fact about the world until you realize that functionally all of them are part of wedding rituals. All cultures agree that setting money on fire is a major function of weddings; specifics differ. You might reasonably think that, far into the future, aunts of the bride will still pay NTT about $20 to have an RSVP thanking them for the invitation but begging off due to unavoidable necessity delivered to the ceremony hall; $40 or so extra if you want the telegram accompanied by decorative Mickey and Minnie statuettes.

One place human effort is currently required and glaringly should not be: B2B payments. For the last several hundred years, these have followed a very well-understood dance. A deal is struck. The exact amount of compensation is decided upon and memorialized by the sender in an invoice. The purchaser receives the invoice, reads it, then wraps money around a metaphorical brick and throws it through a metaphorical window. Someone on the other side of the window then applies forensic science to the question of what caused this particular brick to arrive. This process is called reconciliation; "keshikomi" in Japanese.

Reconciliation is a business process which arises almost entirely because of a lack of structured data in the pipelines that convey money between businesses. Because of the lack of structured data, to a greater or lesser degree worldwide, many people must spend a great deal of time every month reconciling incoming payments to invoices. Because of technological and organizational barriers, the people who do this work (in Accounts Payable or Operations or office management or any other seat at the company) often do not have direct visibility into the customer relationships which gave rise to the invoice or the instrumentalities of money movement. They’re only allowed to see the epiphenomena of the work.

How reconciliation works in Japan

As we covered previously, Japanese bank transfers hold effectively four pieces of metadata: receiving account number (bank code, branch code, 7 digit number), receiving account name (effectively guaranteed to be correct by the banking system given that the account number is correct), amount, and sender’s name.

The task of reconciliation is taking these four pieces of information, of which two are generally useless because they’re about you rather than the transaction, and associating the transfer with the previously agreed-upon debt between businesses it satisfies.

Why do this? The biggest reason is accounting; you can’t know how much money entering your bank account represents actually earned revenue and how much represents Something Else unless you’re able to trace payments back to someone agreeing to buy goods or services for money. Tying payments to an invoice is an important piece of that puzzle. Reconciliation also helps answer the questions “Which of my customers haven’t paid yet?” and “How much earned-but-uncollected money do I have on my balance sheet?”

(An important question! Uncollected revenue is not quite cash, but it is an asset, and you’re likely to collect the supermajority of it in the very near future! Your company is in the constant state of being owed money and that money being the difference between life and death, just like you yourself are in the constant state of being the air in your lungs and about three minutes away from asphyxiation. It’s really important to know your next three minutes of air are in the bag.)

Any unstructured data field will be abused by users

96% or so of Japanese business transactions, many trillions of dollars worth, go over bank transfers and are only reconcilable because of a hack of those two remaining data fields. The dominant form of the hack is “Please override your name as the sender and replace or supplement it with the following alphanumeric invoice ID.” The recipient can then read that ID and quickly look up the corresponding invoice, rather than looking through all of their outstanding invoices.

This feature is built into most payment systems that are used at scale. Credit card payments, for example, are broadly designed so that a point-of-sales system or online portal knows specifically which transaction they’re charging you for when you charge them. Operations done on that transaction in the future trivially flow over to the associated movement of money. If you go to McDonalds and your hamburger is not to your liking, when the clerk voids the sale, you get your money back so transparently that nobody needed to consider that voiding the sale and giving you your money back are in fact two different operations on two different parallel events.

This is possible with cards, and most payment systems, because of a carefully coordinated dance between the parties at the time the transaction is made. Mostly for historical reasons, bank transfers assume that this dance will be coordinated not by software but rather by people.

There is no intrinsic reason this is so! Humans don’t actually add value via e.g. a review process to most transactions! There is no one sitting at the Tokyo Stock Exchange refereeing in real time between a foreign hedge fund and domestic marketmaker saying “Yep, I’ll certainly allow this multi-million dollar transaction to go through.” Everyone has precommitted to a system which, through a great deal of ongoing effort but with virtually no marginal human attention, balances books, tidies up entries, and catches problems.

But for business payments, the lifeblood of the economy, we still rely on many millions of hours of human labor every month.

The trouble with people and unstructured data

By far the most common problem with giving people the instruction “Put the following reference number in your bank transfer” is that they simply don’t follow that instruction. Much of that behavior isn’t stupid or malicious; there just physically isn’t a pathway between the person in receipt of the invoice, the accounts payable department, and the bank which would allow overriding the company’s name on the outgoing transfers. (Surprising, but accurate.)

The company’s perspective on this problem is that if you want to keep getting their money then you’ll deal with it the same way they deal with it. You will employ a floor of people who do basically nothing other than this a week every month. You will tolerate 15 business day average confirmation times on a ~500 ms latency payment method.

Once the invoice ID is omitted from the transfer, all bets are off on attempting to reconcile it, and the sleuthing must begin. People do not have unique names. Companies don’t necessarily pay from the entity that the receiver was expecting; this could be a subsidiary or parent company (in the more sophisticated case) or from the owner’s personal account to save a few hundred yen in fees (in the less sophisticated case).

Your bank does not have any knowledge of your suppliers’ invoice IDs or expected formats, but will expect you to enter them all by hand, leading to a truly disheartening number of typoes. Customers will never cease to amaze in finding other numbers on their invoice or previous correspondence to use in place of the invoice ID. Dates, phone numbers, the Giants’ current box score, etc are all useful information to someone but probably less than optimal for your Accounts Payable or operations team.

How reconciliation could work

Recall that, of the four pieces of metadata that bank transfers carry, two are broadly useless. How about the third one, the amount?

Many Japanese businesses do factually override the amount of the invoice to help their reconciliation process. This is a thing that really happens. The mechanism is sneaking a line-item onto all of your invoices for an unspecified arbitrary discount for the customer, such that the final two digits of your invoice are unique among all invoices you expect to send that month. Then, to find the correct invoice to attach NoUsefulNameHere’s payment of 234,235 yen to, you just scan down your list to find the only one which ended in 35 yen.

This, of course, breaks down at scale, but it is a common trick in the arsenal of small business professionals throughout Japan.

The thing which actually works at scale, but is barely adopted yet, is treating the receiving bank account as being useful information rather than being the same for literally every payment into your company.

This is generally done via a mechanism called Virtual Bank Accounts (VBAs), which are a product available from the banking industry in Japan, the U.S., Mexico, Brazil, and many other countries. You contract with your financial institution of choice to reserve a block of bank account numbers corresponding to a far smaller number of actual bank accounts. You give out those numbers to your customers rather than giving out your “real” bank account number. You then take action based on which account number your customers use.

Due to technical and social issues within the financial industry, the banks offering VBAs generally expect you to bring your own implementation work at this point. Should you e.g. re-use VBAs within your block? They probably don’t have a straightforward answer to that question; up to you. Should you treat them as secrets? Up to you. Should you share them between customers? Up to you. What should you do once you know one of your VBAs has received a transfer? The bank will give you your money, what you do with the data is up to you.

Stripe does basically the simplest thing that works: give each customer/business pairing a unique VBA, shared across all invoices for that pairing (to avoid e.g. a customer not updating their supplier management system with the new bank account number on the second month’s invoice). Use ability to introspect invoices (and their open/closed/etc state) and inferences to tie incoming payments to the invoice they’re most likely associated with. Kick all the exceptions to a human or computer system, whichever the user specifies.

This works really well. Which shouldn’t be surprising: virtualized addressing is essentially to how your email, phone calls, and web visits get delivered. Billions served an hour, error rates far below any human-mediated process, cost almost too cheap to meter. The only thing preventing its use in payments is not deciding to do it. (One could similarly use virtual addressing for postal mail, by the way, and Japan has started doing so.)

The implications are straightforward: this lowers the total cost of payments by eliminating human effort, which is expensive, and human errors, which are expensive and numerous. An underappreciated consequence is that freeing people from drudgery gives them the ability to do more important, meaningful work.

Story time

I was once a peon in the bit mines of a systems integrator, and had to implement a system quite like Keshikomi Simulator, except much more serious. It was designed so that operators at a particular Japanese university could reconcile tuition payments against tuition invoices. It did not automate their paper-based process; it moved their paper-based process onto a web interface, for (real, but modest) gains in observability and recordkeeping efficiency.

For two weeks of every year, from the founding of the university through the present day, they had endured an absolutely hellish crunch quickly reconciling 80-90% of incoming payments and then dealing with edge cases.

Don’t dun Hanako; her tuition was paid by her dear uncle, who doesn’t share her last name, but did send in tuition in a timely fashion. Don’t dun Yusuke; his father paid through the account of his care dealership rather than from his personal account. Don’t dun Esmerelda; her parish took up a collection to pay her tuition and you can find it under the name of the parish treasurer d/b/a the small town church.

These facts, and hundreds more like them, were discovered every year by painstaking work on phones, faxes, and gossip trees. Any uncaught mistake in the process would result in a student being unable to register for classes, often thrown into substantial doubt as to whether they had a future in college or whether a substantial portion of their family’s life savings had been irretrievably lost. The people who performed that work (and who were in close proximity to mistakes that were made) were highly educated professionals who the university and society would have preferred do absolutely anything else. They were reconciling invoices because there was simply no other alternative; the invoices had to be reconciled or the university would cease to function.

That is a silly constraint on the world, like phone calls needing to be manually switched between circuits was a silly constraint on the world. It should be done away with. The optimal number of staff hours to spend on that process was zero. Zero dinners with family missed during the crunch weeks. Zero students dunned unnecessarily. Zero registration holds.

Zero is a bold number, but it is an achievable one. We can do it; we have the technology.

Beyond reconciliation

As payments start to carry more useful data and be more natively operated on by computers, they get far more powerful. The blockchain enthusiasts are pretty convinced that programmable money is going to win the future. I entirely agree with them on this score; I just think that the most programmed money is going to be the dollar, yen, or similar.

A big question in network economics is where in the network the value accrues. The Internet stitches together a lot of disparate data siloes but relatively little value accrues to the network itself and almost zero to the protocols. Banks and companies are siloes with some (fairly primitive, by modern standards) wires connecting them; it remains to be seen to what degree and in what fields intermediaries between them succeed. The market opportunity, though, certainly isn’t a small one.