A Brief History of Crypto Audits

In 2014, a former Bloomberg programmer named Changpeng “CZ” Zhao became chief technology officer at OKCoin, a Chinese start-up token exchange.

Being OKCoin's community cheerleader was a big part of the role. The price of Bitcoin had plummeted following the collapse of the Mt. Gox exchange, Beijing's hostility toward crypto intensified, and miners worldwide had shut down.

OKCoin played a pivotal role in reviving interest in crypto as a gambling token by adding derivatives and staking, while CZ used its social media profile to boost trust among ardent believers. He was often found on Reddit, like in this now-deleted but archived post denying that OKCoin used bots to fake volume. And he's been a regular on podcasts like this one, where CZ explains (at about 25 minutes) that wash trades on the stock market were due to Chinese traders trying to win a car.

CZ left OKCoin in February 2015 after just eight months.

At first everyone seemed to be in agreement, with CZ saying the exit was "a difference in direction." Then came a Reddit post (deleted but archived) in which CZ makes dozens of accusations against OKCoin and its founder Mingxing “Star” Xu, including the alleged use of bots to inflate volume.

OKCoin responded on Reddit to deny what it called CZ’s “lies and desperate nonsense,” adding numerous accusations of its own.

It's easy to dismiss public mudslinging as a relic from crypto's frontier age. The trade in insults — which allegedly stemmed from a contract dispute between OKCoin and Roger Ver, an early evangelist known as Bitcoin Jesus — was fought in typical messageboard bluster using jargon that can often seem deliberately impenetrable. Nevertheless, the battle over the issue of auditing is relevant today.

In August 2014, a few months before CZ's departure, OKCoin released what it described as the industry's "first proof-of-reserves audit." The report was written by Stefan Thomas, who at the time was CTO of Crypto Payments Group Ripple and was well known to the community. He concluded (with numerous caveats) that his Bitcoin balances appeared to cover user funds by more than 104 percent.

Calling this an audit wasn't helpful. There are many good explainers of the mechanism used; The very simple version is that Thomas performed integrity tests on databases provided by OKCoin that listed his Bitcoin assets and liabilities. These tests generated a type of asset checksum known as a Merkle root, which depositors could use to verify that their own holdings were present on-chain. It offered a bare minimum of transparency, as Thomas warned:

Note that this type of testing has limitations. It does not examine an exchange's fiat assets and liabilities or other aspects of its balance sheet. It is also difficult to definitively prove that the Bitcoins in question are actually owned by the exchange rather than, for example, on loan.

In an interview with CoinTelegraph, CZ dismissed criticism of the “audit” and disputed suggestions that OKCoin may not have given Thomas all the relevant data.

Then the story changed. CZ's Reddit post called the audit "fake" and claimed OKCoin understated liabilities by hiding its own bot accounts. “Essentially, these bots trade fractional (or notional) reserves,” he said. "Thomas was lied to during the audit. This is an unfortunate limitation to the method of proving reserves."

In response, OKCoin claimed that hiding the bots was to avoid double counting of borrowed coins. CZ “doesn’t even understand what an exam is,” she added.

Eight years later, CZ wants to emphasize that he knows what an audit is:

Verified proof of reserve. Transparency. #Binanz https://t.co/IClZxTYaWp

— CZ 🔶 Binance (@cz_binance) December 7, 2022

Binance, now by far the largest crypto trading venue in the world, has been pushed into transparency by the collapse of FTX. But what it delivered last week can be seen as no more of a test than OKCoin's efforts in 2014.

Binance appeared to be 101 percent collateralized at an underlying level on November 22, according to a letter from the South African subsidiary of accounting group Mazars to “Binance Capital Management Co. Ltd” in the British Virgin Islands. The term audit is not used in the letter. Instead, Mazars says it followed an agreed upon procedure (AUP), meaning its employees could only make factual determinations within Binance's predefined parameters. The accounting firm conducted no additional research, formed no opinions, and offered no assurances—including the validity of the exercise as a whole, as the introduction to its five-page letter attempts to make clear:

Binance management acknowledges that the AUP is fit for purpose and is responsible for the subject matter for which the AUP is being executed. [ . . . ] We make no representation regarding the adequacy of the AUP.

This AUP engagement is not a business audit. Accordingly, we do not express an opinion or an audit conclusion. Had we conducted additional procedures, we may have identified other matters that would have been reported.

There is no information about which of Binance's Nomadic business units was tested. It only covers self-reported Bitcoin assets and liabilities and excludes US operations, according to a Wall Street Journal article. Improvements to the 2014 Merkle Tree methodology, such as: Some changes, such as using dummy transfers to verify an indefinite number of wallets, are incremental at best.

And even then, Binance failed.

“We found that Binance is 97 percent collateralized,” writes Mazars partner Wiehann Olivier. Required passport marks “taking into account the out-of-scope assets pledged by customers as security for the in-scope assets lent through the margin and credit service offering, resulting in negative balances in the customer liability report.” In other words, the headline figure of 101 percent was based on excluding Bitcoin from the liability side, Binance said, which had been lent.

That's a plausible excuse. Collateral held for Bitcoin loans would be held in currencies or tokens other than Bitcoin and would therefore be beyond the scope of Mazars' report. Still, it's difficult to place much trust in an audit that revolves around such narrow parameterssilentcan arrive at no good answer other than asking for more blind faith.

Binance says it will provide information about tokens other than Bitcoin in the coming weeks - the most important of which are the native BNB token and the Binance USD and Tether stablecoins. Shortly after FTX's implosion, Binance reported that it held approximately $69 billion worth of coins as of November 10, and that these three tokens accounted for more than 70 percent of reserves by value. Diagram below from Mike Alfred:

FTX’s failure sparked a race among crypto exchanges to avoid “not your key, not your coins” becoming an existential threat. Star Zu's OK Group, which includes exchanges OKCoin and OKX, is one of many promoting transparent proof of reserves through auditor-backed token validation, although it is based on the same basic methodology as its 2014 audit.

But even compared to the data of its smaller competitors, Binance's efforts at transparency have not convinced many commentators. Research group Mysten Labs (which was funded by Binance) published a report last month identifying “potentially exploitable vulnerabilities” that could mean liabilities were being undervalued.

The closest thing to a public sentiment gauge on Binance, its BNB token, came under pressure on Monday after a Reuters report said the U.S. Department of Justice could face money laundering charges as part of a long-running criminal investigation into the exchange. Binance posted a lengthy response on its website, and CZ responded on social media in his signature style - with a dig at an opponent and an appeal to the community.

smh, some media still work for...

— CZ 🔶 Binance (@cz_binance) December 12, 2022

Ignore FUD. Keep building!

— CZ 🔶 Binance (@cz_binance) December 12, 2022

But after nearly a decade of CZ noise, it looks like the community wants some new material: