Dan Guido

Before I begin, I think the MAIAN analysis tool is a useful advance to the field and we plan to incorporate some of their features into Manticore in the coming months. My main issue is with their evaluation of results.

In their paper, a “greedy contract” is any contract that can lock ethers. However, ALL CONTRACTS compiled with solc <= 0.3.6 without a ‘withdraw’ function are “greedy contracts” and are included in their totals.

This is because solc < 0.4.0 allowed any function to receive ethers. This behavior has since changed and current versions of solc use the “payable” attribute to denote functions that can receive ether.

To really drive this home, even their example “true positive” issue in section 5.2 is a greedy contract due to this behavior of solc. This vastly inflates their numbers beyond what reasonably should get counted.

Next issue: the authors indicate they found only 1,524 _distinct_ greedy contracts. ~30k comes from instances of the issues. There is no discussion of the distribution. Are there a few contracts that repeat the vast majority of these issues? We don’t know!

Even further skewing their results, they neglect to discuss that _many_ contracts deployed to public chains are there only for testing. Better benchmarks would account for usage.

In our own testing at @trailofbits, we have tools (e.g., Slither and others) that have detected _thousands_ of vulnerabilities on the public Ethereum blockchain, but most of the contracts affected are not used or are useless.

We think our work contributes to the field and have found many issues for our clients with them, but we have resisted fantastical headlines like the ones in this paper because we are aware of the caveats.