By Ray, IOSG Ventures

As a large-scale computer system, the current system complexity of blockchain has far exceeded the level of 5 years ago, the degree of infrastructure modularization is more refined, the logic of smart contracts at the application layer is becoming more and more abundant, and the interaction between contracts is very frequent , more importantly, the number of assets managed by the blockchain system is already very large, so there have been more discussions about the security cycle in the blockchain security community recently (the situation is the same as in 2017, when people mention security, they only think of developers It is very different to write the contract and throw it to the friends of the Ethereum Foundation to take a look and do some basic tests).

Throughout the security life cycle of blockchain programs (from testing, inviting third-party audits to post-event monitoring, update audits), the bug bounty community is like a safety cushion to attract white hats through game theory and cluster work They conduct the last review of the code of the project party, and some smart contract security workers feel that the bug bounty is more like the last man on the line of defense, but I think the bug bounty and audit competition have the potential to play a greater role in the future, acting as a The roles throughout the entire security life cycle improve the security of the system as a whole. **

Of course, there are also bug bounty programs (Bug Bounty or Vulnerabilty Rewards) in the field of traditional network security. First, major technology companies such as Facebook, Google, Microsoft, etc. will deploy bounty programs for their own inhouse security teams and their own product lines. Secondly, Bug bounty third-party platforms represented by HackerOne and Bugcrowd have emerged since around 2015. Currently, these two leading security companies rely on bounty draws as their main income, and their annual income can reach nearly 50 million U.S. dollars and 20 million U.S. dollars respectively. dollars. In the blockchain world, bounty is a more interesting topic that is often discussed in the security circle. The main reason is that the open source of blockchain code actually makes the cost of hacking and improving attack strategies cheaper. In addition, the crypto world advocates clustering Work, Creator and Ownership Economies open to contribution models that make a more open white hat economy even more valuable.

**What are bug bounties and audit contests? Why do we need them? **

Security is a dynamic game between the attacker and the defender, just as computer security expert and cryptographer Bruce Schneier said, "Security is a process, not a product. It is a way of thinking that must run through the software development process Every aspect.” In the blockchain world, a dark forest where all codes are open source and transparent, a blockchain project that wants to survive for a long time must have eternal needs for the security of its products/contracts. Chain products all have more or less financial attributes. The most important asset in finance is trust, and the user's trust is only once.

Where are the shortcomings and problems of traditional auditing? What advantages can community-driven bug bounties and audit competitions have to make up for these problems?

Developers using auditing services often find that:

• Even after purchasing the services of a third-party auditing company, there are still problems with the code after auditing. Although the reasons for these problems are different (technical and non-technical), it does not seem to be completely reliable to rely on an auditing company in the end. However, the quality of code audit still depends on the level of auditors, and customers often lack the ability to discern "who is better".
• While the bounty platform and audit competition are more open "sandboxes", the project code can be reviewed by white hats at will, regardless of background (there may be personnel from professional audit companies, and there may be freelance security analysts), the arsenal is unlimited, and all customers have to do is set a reasonable bounty and pay their contribution when the white hat finds a problem.
• Usually customers will first submit their code that needs to be reviewed by the white hat, define the security level of the vulnerability (usually related to the possible economic loss, the easier the vulnerability that directly causes economic loss, the higher the severity level), bounty budget, testing code scope and even test steps.

How big is the market?

The business model of bounty platforms and audit competitions is usually to draw a portion of the bounty paid by customers or the total bonus pool set up as the service fee of the platform. Customers (project parties) who need code security audits will announce their plans on the bounty platform according to their own needs (which codes need to be covered by the audit, how to define the severity of vulnerabilities, and how much reward they are willing to pay), and the white hats Vulnerabilities will be found according to the needs of the project side. Once the loopholes are found by the white hats and meet the needs of the project side, the bounty will be distributed to the white hats, and the bounty platform will draw a commission from it as a service fee.

In the field of Web2 traditional network security, the bug bounty platform is also a relatively young direction (appeared after 2012), and currently the largest bug bounty platforms are HackerOne and Bugcrowd. In 2022, HackerOne's annual revenue will reach 58 million US dollars, the company's valuation will reach about 500 million US dollars, and the cumulative bounty paid in history will be 230 million US dollars (2021 and 2022 will pay 150 million US dollars in rewards), and more than 65,000 softwares will be discovered. Vulnerabilities, with more than 1 million registered hackers, and more than 1,000 customers using HackerOne services every month. Its competitor, Bugcrowd, is expected to generate more than $20 million in revenue in 2022.

In the field of Web3 security, in 2022, all web3 bug bounty and audit competition platforms will distribute a total of 50 million US dollars in bounties to white hat hackers, and the average fee level of such platforms is around 10% to 30%, so it is conservatively estimated The current market size is around $5m~$15m, and it is still a very emerging market.

Another interesting thing is that more and more customers are willing to directly use the code auditing services provided by this decentralized security community. The most famous example is that Opensea did not directly find the second-tier auditing service before launching their new platform Seaport. The three-party audit company chose Code4Rena, the largest decentralized audit competition platform at present, and set up a prize pool of 1 million US dollars. Today, the traditional security audit market is increasingly involved (volume human resources, volume technology tools, volume market BD ), will decentralized security services be an important growth in this market? (Currently there are 56 auditing companies in the market, and the revenue of the leading companies in the past year was US$10 million to US$40 million. I think there is a lot of room for imagination in the decentralized security market).

Bug Bounty Platform vs Audit Contest Platform

Although the bug bounty platform has a ten-year development history in web2, the audit competition platform is a new thing in web3 native. The object of the audit competition service is those project parties who are about to launch products or some new functions, and use the power of the decentralized community to help them complete the audit service within a specific time (more than 2 weeks). From this perspective, the audit competition will It will bring no small business threat to traditional audit companies.

Below I will show the differences between the two platforms in terms of participation methods, reward structure, and test coverage:

way of participation

Bug bounty platforms such as Immunefi are usually open projects where anyone can participate at any time. Participants typically independently explore and report vulnerabilities in exchange for rewards. If two people find the same repeated vulnerability, the first-come-first-served principle will be followed, and whoever submits the report first will get the reward first.

Community-driven audit competition platforms (eg Code4rena, Sherlock) are often time-limited, competing with participants to find and report vulnerabilities within a certain time frame. Compared with the bounty platform, there will be some teamwork (for example, each project will have a clear assignment of Lead Senior Auditor and Lead Judge, and finally review and summarize all audit results into an audit report to the customer, and these two leaders also follow The principle of decentralization of community elections and competitions). In addition, if two audit competitors find repeated loopholes within the specified time, both of them can get rewards.

Reward Structure

The actual rewards issued by both will mainly consider the severity of the discovered vulnerability.

The only difference is that a community-driven audit competition platform like Code4Rena will have a fixed portion (5%~10%) of the bonus pool for each project allocated to Lead Senior Auditor and Lead Judge, because they actually assume the role of project leaders of traditional audit companies. character of.

Another interesting point is that the project party on the bug bounty platform sometimes places project tokens as rewards, but I also see that some white-hat hackers in the community prefer to get USDC, USDT stable coins rather than price fluctuations Project tokens.

Scope and Focus

Bug bounty platform projects usually have a broad scope, while projects on audit competitions usually have a more focused scope, targeting a specific function or aspect of the software, while requiring white hats to focus on completing the work in a shorter period of time.

Projects focused on auditing competitions

Code4Rena - An esports-like community-driven audit competition platform

Code4Rena has three character types:

1 Auditor (Wardens) review code. Anyone from a professional security engineer to a novice developer trying to gain more experience can register as an auditor to participate in the public auditing competition.

2 Judges (Judges) are usually the best engineers in the C4 community. They determine the severity, effectiveness, and quality of vulnerabilities and evaluate audit performance.

3 Sponsors (Sponsors) are project parties, such as Opensea, Blur, ENS, Chainlink, etc. They create bonus pools to attract auditors to audit the code of their projects. Sponsors also have the option to host private, invitation-only competitions for added privacy.

One of the most interesting points is the culture that Code4Rena is building: collaboration and teamwork are encouraged. Unlike traditional bug bounty programs, Code4Rena pays all auditors who report a valid vulnerability even if the vulnerability has already been reported. This encourages healthy competition among auditors as they are motivated to find high-severity and common vulnerabilities. On this platform, some auditors will form temporary teams to find loopholes together.

business model:

Any project can go to Code4rena to start an audit competition program and provide USDC or ETH to set up a basic prize pool (usually the size of the prize pool is $40,000 ~ $100,000), and Code4rena will charge 20% from the basic prize pool as a platform to organize competitions, provide reviews, and organize audit outputs Service revenue for reporting results. The project party can also provide project tokens on top of the basic prize pool to set up an additional prize pool, and Code4rena will charge 40% of this additional prize pool.

Sherlock - Community driven auditing with smart contract insurance

Similar to Code4rena, Sherlock also has roles such as auditors, sponsors, and judges. The uniqueness of Sherlock lies in the insurance services provided by the platform. Anyone can invest in the insurance pool on the Sherlock platform. Investors deposit USDC into the insurance pool, and agreement customers can purchase services to hedge the risk of smart contracts being hacked. The sources of income for insurance investors include: premiums paid by agreement customers + interest earned by depositing insurance pool funds in other DeFi pools (Aave, Compound, etc.) + Sherlock token incentives. But the investor bears the risk of repaying the policy while reaping the benefits.

Another point that is different from Code4rena is the distribution mechanism of audit service income provided by the platform. Compared with Code4rena, Sherlock has rules that allow the chief senior security auditor and the chief judge to obtain a fixed amount (5%~10%) from the bonus pool to properly compensate and motivate full-time senior auditors. In addition, there are systems of selection and competition for the selection of leadership roles.

**How to build a hacker community? What is the biggest concern of Web3 white hats? **

After we observed different decentralized security communities (ImmuneFi, Hats Finance, Code4Rena, Sherlock, etc.) and chatted with some security entrepreneurs, we think that what all decentralized platforms are dedicated to doing is to build a healthier, An efficient communication and collaboration platform, the bounty platform is like a marketplace between Hackers and projects, they must consider their needs from the perspective of hackers (as shown in the table below), and at the same time consider what the project party cares most from the perspective of the project (Audit Quality).

Source: 《Bug Hunters’ Perspectives on the Challenges and Benefits of the Bug Bounty Eco》

In addition to some common needs, I also saw some interesting topics in the Immunefi white hat community (the most lively white hat discord community I saw).

for example:

There is a white hat named Rappie who wants to disclose some project loopholes he/she has discovered before, and asks what community rules need to be followed. (1. Only publish bugs that have been fixed. 2. Make sure that any public information has no negative impact on the protocol or its users. Keep confidential information e.g. after they fix your SQL injection vulnerability, don't publish information about their full database. 3. Make sure that you need to send a private message to the project team before making it public).

A white hat named Noam Yakov has doubts about the definition of a bounty project (this often happens, because usually only serious security vulnerabilities can be rewarded. How does the project define the security level of the vulnerability? Something that the white hats care deeply about, and the community hears about such disputes a lot). In the Uniwhales bounty project, he had doubts about their definition of MEV impact as a serious security vulnerability. In the end, everyone discussed that this type of description does not apply to all MEV situations. For example, for some toxic order flows, the protocol pool The situation of asset draining is definitely a serious security incident (so it is often not enough to define a set of security level frameworks, and usually a similar role of arbitrator in the platform is required to intervene in different actual cases).

And for a very interesting topic, "What are your demands and expectations for a bounty platform like Immunefi?" A white hat named ckksec gave his answer: 1) Help these anonymous encryption white hats earn their labor income Do some legal clarifications like invoicing. 2) The platform should not only have a scoring system for white hats, but also score the quality of the project because white hats often need to spend time distinguishing the quality of the project. 3) For white hats who are willing to open their proflie, the platform can show their workflow. At the same time, it is better for the platform to more transparently display the security analysis report information received by the project party.

What tools can help white hats?

With the fire of LLMs GPT, I have recently heard people frequently discussing whether security audits can also be replaced by AI. The experienced security practitioners I have talked with generally believe that GPT is difficult to directly replace human intelligence. Some low hanging fruit (problems that are easy to find) may be detected by language models, but those problems with medium and high risks still require expert participation. For example, according to a senior security expert’s feedback, for similar data analysis and dynamic analysis, these more complex tests need to be artificially combined with the actual business logic of the protocol to conduct security analysis tests in advance and define the expected target attributes of the test in advance. The most difficult part is to write a good properties and define the correct test field. According to their experiments on GPT, they believe that GPT cannot completely replace humans at this stage.

Of course, there are currently more optimistic results showing that LLM can greatly improve the analysis efficiency of security analysis tools and reduce the false positive rate:

Let's think about this topic from another interesting non-technical perspective. It is a dynamic game between security attackers and defenders. The magic height is one foot higher and the height is higher. Will AI bring security challenges to security attackers? help?

Safety is people-oriented

People will habitually think that software is a cold, mechanical, and logical thing, and improving system security only needs to improve analysis technology and system defense level. However, people lack of thinking about security issues from the perspective of economic incentives and human nature. In the dark forest of open source code, we need a distribution system that is more in line with the assumptions of rational people. People who contribute wisdom to system security join.

The current traditional security audit market structure is stable, and brand reputation is the most important intangible asset of companies in this field. Over time, the influence of top security brands and the trust of customers have steadily increased, but traditional security audits also have their own problems (the business model relies solely on manpower and it is difficult to grow in scale, and leading companies need to balance growth and audit quality. Some companies have encountered such a bottleneck and even affected the value of the brand).

**Community-driven security audit competition is an innovative business model. **Currently, the number of customers of the two platforms has exceeded 300 and gradually found PMF. The bounty platform is a good supplement to the security life cycle. Although these decentralized platforms have not yet found a particularly effective token model, we are very optimistic about the large-scale growth of this market in the future (because the wisdom of the crowd is very suitable for the offensive and defensive game scenarios in the security market).

** Will community-driven auditing platforms pose a threat to centralized auditing companies? We think they will have a benign mutual competition and complementary relationship. In the short term, when a platform like Code4rena forms a certain network effect and has a good track record (the proportion of audited projects being hacked is low), it may indeed give Some centralized companies in the middle and tail will bring certain competitive pressure, but in the long run, this may also force the centralized audit platform to form some commercial cooperation with the community-driven platform, because this can also broaden the customers of the centralized security audit platform Group and improve audit quality (a bit like the original security bounty project that was independently operated by a major web2 company and later formed a cooperation logic with third-party platforms such as HackerOne).

Although the direction of the community-driven security platform is to be more DAO-oriented (Forta can actually be included in this category), in the actual operation of the current project, there are still problems such as: how to make the workflow and economic distribution process more transparent and open , How to weigh the privacy and security considerations of the project party, how to more clearly define the relationship between teamwork and personal contribution, how to solve the problem in a more fair and professional way when conflicts of interest arise, etc. These are the things that security DAOs need to face Right challenge.