Smart Contract security and vulnerability detection:
A fundamental framework for retrieving and execution
Keywords: Smart Contract, Vulnerability, Smartbugs
Highlights: Research, Innovation, Data Descriptor
Disclaimer: “This article is a final deliverable from Prof. Luyao Zhang’s project entitled “Trust Mechanism Design: Blockchain for Social Good,” supported by the 2022 Summer Research Scholarship (SRS) program at Duke Kunshan University. We give many thanks to DKU 2022 SRS program and the SciEcon Insight editorial board.”
Overview
Over the past few years, the smart contract (SC) application has been deployed substantially in document preservation and accessibility, administrative payment and billing, etc. At the same time, SCs’ code is made available to the public when used in permissionless blockchains, which means anyone with access to the blockchain can therefore examine the SC’s code. This enables external actors to learn more about the processes and actions that SCs carry out. An attacker may take advantage of this information to choose a prime target to maximize the benefits of a potential attack. An attacker can take money from the account linked to the victim’s SC by utilizing vulnerabilities or loopholes that cause recurring SC activities. Therefore, researchers have developed different detection tools or mechanisms to detect existing vulnerabilities of smart contracts in terms of reentrancy (Qian et al.), time manipulation (Dave), and so forth The Smartbugs supports and categorizes 8 typical SC vulnerabilities as table 1 shows.
Table 1 Type of SC vulnerability
However, as no such tool is universal in detecting all types of SC vulnerability, it is not trivial for users to reproduce or compare those tools. This article will introduce an easier framework called SmartBugs that simplifies the execution of detection tools and the retrieval of related datasets. The work is constructed as the following structure: 1. Trial results that could be derived from SmartBugs 2. What datasets do we use, and how to import data to this framework 3. Guideline for using SmartBugs and retrieving data The structure of the work is shown in Figure 1.
Figure 1 Mindmap of this article
Part I Introduction
SmartBugs support ten tools and two datasets of Solidity contracts at the moment.
For the supported tools, the summarized information is as below (“ — — ”denotes not applicable, and Smartcheck was deprecated since 2020) :
Table 2 Supported SC vulnerability detection tools by Smartbugs
For the two datasets, the first dataset, which contains 143 annotated susceptible contracts with 208 labeled vulnerabilities, can be used to assess the accuracy of analytic algorithms. The second dataset includes 47,518 different contracts that were acquired using Etherscan. As we will further introduce them in the coming sections, this part uses figures to strengthen the comparison results between the detection tools supported. Once we run the command to ask all the 10 tools to work simultaneously, we will get the detection outcome stored in a folder called results by default.
Table 3 shows the vulnerabilities identified per category by each tool. When a tool finds a vulnerability of a particular category at a particular line and matches the vulnerability noted in the dataset, it is said to have been discovered. In this table, each row represents a category of vulnerabilities, and each cell shows the number of vulnerabilities for which the tool finds a vulnerability in this category. Table 3 has some cells with numbers in brackets, meaning these numbers represent the number of vulnerabilities found by a single tool. This table concludes the strengths and weaknesses of the existing detection tools. As we can see, the tools can relatively accurately detect vulnerabilities in Arithmetic, Reentrancy, Time manipulation, and Unchecked Low Calls. However, for the rest of the categories, the tools failed to detect the vulnerabilities on a lower-than benchmark basis. It shows that there is still room for researchers to enhance the tools' ability.
Table 3 Vulnerabilities identified per category by each tool
Table 3 also shows the variance of detection accuracy of each tool. As we can see, Mythril has the highest accuracy rate with 27%. Moreover, Mythril, Slither, Smartcheck, and Manticore detected the largest number of different categories of vulnerabilities (more than five). However, as no tool shows the comprehensive ability to replace other tools, researchers need to make combination usage according to their different features.
Part II Datasets
Two datasets distribute SmartBugs: SB curated and SB wild. Users can assess the correctness of analytic tools by using the SB curated dataset, which has 143 annotated contracts and 208 vulnerabilities that have been tagged. And SB wild is a dataset with 47,398 unique contracts from the Ethereum network. The first dataset is either real data or created purposely for researchers to examine their detection tools. The second dataset is collected from the real Ethereum blockchain information. This article will emphasize the second dataset and introduce how to retrieve data with the help of Google BigQuery and Etherscan API.
Table 4 Dataset Variable Explanation
For detailed information about Google BigQuery and Etherscan, please see https://cloud.google.com/bigquery and https://etherscan.io/.
The steps for retrieving data from the Ethereum blockchain and creating the dataset SB wild are as below:
- Select all the contracts that have at least one transaction using Google BigQuery. The collection was presented on August 8th, 2019. We used the following request as figure 3 shows:
Figure 3 Bigquery command
Figure 4 presents an overall summary of the data we produce.
Figure 4 Brief summary of the data retrieved from Google BigQuery
2. Download the source code associated with the contract addresses. Here, SmartBugs use Etherscan API with python to access the data. As we will introduce in the next step when we run SmartBugs on the user’s terminal, it will automatically direct you to the SB wild and there is no need for users to worry about the Python coding. Figure 5 presents the sample code for accessing Etherscan API and retrieving data. For detailed codes, see here.
Figure 5 sample code of retrieving data with Etherscan API
Table 5 shows the sample data we produced so far.
Table 5 Sample data retrieved from Etherscan
3. Filter the set of contracts to remove duplicates.
Part III Step-by-step guidelines for SmartBugs
Introduction of the structure
The system requirements, tools and datasets that are readily available, techniques for adding tools and filtering datasets, and the available interfaces are all highlighted in this section’s description of SmartBugs. The five main components of SmartBugs are the command-line interface, tool configurations, tool Docker images, datasets for smart contracts, and the SmartBugs Runner, which combines all of the components to run the analytic tools.
Figure 6 Framework of SmartBugs
Installation
Before we start to run SmartBugs, we need to first make sure we have already got Docker and Python in our environment. You can check the detailed information on Docker here.
Once you finish the installation, follow the steps below in your terminal.
- Clone SmartBugs’s repository:
2. Install all the Python requirements:
pip install -r requirements.txt
Command-line Interface
SmartBugs provides a command-line interface that allows users to run different analysis tools on the available datasets of contracts. SmartBugs command-line interface can be invoked as:
Figure 7 Command-line Interface Sample
For example, if we want to use the tool oyente to detect the type of vulnerability reentrancy, we can type python3 smartBugs.py — tool oyente — dataset reentrancy in your terminal. Figure 8 illustrates the outcome of your command here.
Figure 8 Sample command example
It will take nearly 10 minutes to run the outcome. Afterward, all the detected outcomes will be automatically uploaded to the result folder in your SmartBugs repository.
WUI Dashboard
For use in interacting with SmartBugs, a web-based dashboard user interface is also available. The user has easy access to a list of tools, named datasets, and vulnerabilities found by each tool, all of which are linked to DASP 10 categories, through this dashboard. A screenshot of the dashboard is shown in Figure 8. There are three ways to examine smart contracts: The user can import a smart contract by uploading a file, paste or construct a smart contract directly in the browser, and run the offered tools on pre-defined datasets.
Figure 9 Illustration of SmarBugs WUI Dashboard
To access the dashboard, the first two steps, as we have introduced before, are copying the repository and installing all the requirements. Then, run the code below.
npm install
npm run smartbugs
Part IV Potentials
SmartBugs contains ten mainstream Ethereum Blockchain vulnerability detection tools and two datasets. For the second dataset, SB wild, which we introduced before, can be utilized to analyze real-world smart contract vulnerability attack cases. For researchers, SmartBugs could also be used to evaluate the performance of the detection tools. As illustrated in the Results section, the tool shows different features and abilities. In this case, it is suggested that researchers are expected to apply tools combination on detection application since none shows a total detection power. For more information and research detail on evaluating different detection tools, please see Empirical Review of Automated Analysis Tools on 47,587 Ethereum Smart Contracts. All in all, SmartBugs is a powerful simplified framework to retrieve data and access the tool. More interactions and applications can be found here.
About the Author
Yihang Fu
Yihang Fu is a junior undergraduate student at Duke Kunshan University majoring in Data Science. He received a quarter-admission scholarship and was supported by the 2022 Summer Research Scholarship. He is passionate about interdisciplinary Study and conducts pieces of research in different fields such as Blockchain security, natural language analysis, environment study, etc. He is keen on exploring and looks forward to converting the in-book knowledge to real-life applications. Yihang is also a member of SciEcon CIC.
Acknowledgments
Executive Editor: Yufan Zhang
Associate Editor: Xinyu Tian
Chief Editor: Prof. Luyao Zhang