Every year, our Application Security team takes the time to step back and reflect on Shopify’s bug bounty program and share what we’ve learned with the community.
Spoiler alert: 2021 was a busy year that saw thousands of reports submitted to our program and bounty payouts totalling $1 million (including a maximum bounty that can be paid). This year, we're doubling that maximum amount and moving a few key services into our highest severity level.
Read on for a deeper look at insights from the past year, as well as details on new changes, and what’s in store for 2022!
2022 Program Improvements
Following a successful (and busy) 2021, we have some exciting new announcements.
Maximum Bounty Rises to $100K
We are doubling our maximum bounty, effective immediately. Reporting a CVSS 10.0 issue now awards $100,000, with increases specifically to the High and Critical brackets.
This is a demonstration of our ongoing commitment to reward researchers appropriately for their hard work, and ensure that it is worth their time to dig in and identify very impactful, difficult to find issues. We recognize that quality security research takes time and dedication, and are ensuring that our bounty structure reflects that commitment.
Shopify Plus, Shop and Shop Pay are Core Services
In addition to bounty increases, we are also moving several services into our highest severity bracket and scoring them as Core assets. This means that High and Critical bugs in these services will be paid according to the new scale detailed above. The following assets are moving into that category, effective immediately:
- shopify.plus
- arrive-server.shopifycloud.com
- shop.app *
* Note that issues specific to the mobile client itself will still be considered Non-Core, under “Shopify Mobile Applications”.
More Resources!
We know that there is a ton to learn when getting started hacking on Shopify, and are working on ways to reduce the barrier to entry and flatten that learning curve. Keep an eye on the Bug Bounty Resources repo for upcoming resources, guides, and even blog posts! We have just released a CVSS scoring guide there which will help in addressing some of the common questions we get about bounty decisions. Transparency is important to us, and we welcome any questions or feedback about the scoring process.
Bug Bounty Program Highlights from 2021
$1 Million in Bounties Paid Last Year
We hit a huge milestone last year in terms of overall bounties paid. Combining our main program with events and private programs, our payouts exceeded $1 million in bounties and bonuses for the first time. This is more than double our bounty payouts of approximately $460,000 in 2020. We also awarded our maximum bounty of $50,000 for a CVSS 10.0 issue for the first time in January of 2021, and then were able to do so again in December.
It’s clear that hackers are learning to dig deeper and are finding impactful bugs, and we’re thrilled to have awarded all of these bounties for their hard work. By doubling our maximum bounty, we are committing to going even bigger in 2022.
First Full Year of CVSS-Based Bounties
In fall 2020, we began paying bounties via the Common Vulnerability Scoring System, and launched our own calculator to determine bounty amounts. Having now had our first full year of bounties via CVSS, we can thoroughly reflect the impact of these changes compared with our previous bounty table.
Overall we’ve had more thorough discussions among the team and made more consistent decisions. Each and every bounty is discussed in full and scored by the team, and we now provide those explanations for hackers on the scoring metrics with each bounty message. For this reason, follow-up discussions with hackers after a bounty have been very productive, as we’re able to dig in on particular metrics to analyze a decision. The aforementioned CVSS scoring guide is a great way to learn about how we score bounties.
A Big Focus on Open-Source Security
In 2021, Shopify became a sponsor of the Internet Bug Bounty program in order to help fund open-source bounties and better support maintainers. We’re also making a further commitment to the Rails program, sponsoring an additional $500 bonus on all valid issues that are accompanied by an accepted patch. You can learn more about this on the Rails program page. Leveraging our bug bounty program to better support open-source projects is very important to our team, and is an area we’ll continue to focus on.
2021 Bug Bounty Statistics
Hackers kept us very busy again over 2021, submitting over 3,000 reports to our programs. This amount is on par with the volume we saw in 2020, which was a large surge compared with previous years.
Let’s dig into those reports, and highlight some interesting trends from last year.
Approximately 23% of incoming reports were valid issues, up from 19% in 2020. We are also seeing that Not Applicable issues were down to 26.75% in 2021 compared to 41.92% in the previous year. These are both strong indicators that our feedback to hackers, transparent CVSS scoring, and policy updates are helping to successfully guide researchers to more impactful issues.
Last year was by far our biggest year for bounties—we paid out more than double the bounties from 2020. This was due to the success of our H1-2102 live event in January of 2021, and the fact that we have also seen higher bounties on average throughout the year. The average bounty in 2021 was ~$3,000, compared to ~$2,070 in 2020. This upward trend is due to a couple of factors: hackers are digging in to find more impactful issues, and our CVSS-based system led to more competitive bounties across the board.
Another key focus for Shopify’s bug bounty program is response time, and we strive to do our absolute best to provide timely and consistent feedback to all hackers. For this reason, we track our response times on a monthly basis. You can follow the @ShopifyEng Twitter account to see these updates.
In addition to these monthly updates, it’s important to step back and examine our response time trends year over year. In 2021, we saw a slight increase in time to triage, but a significant decrease in time to bounty (137 hours compared with 249 in 2020). We are constantly striving to better these numbers, to get feedback and bounties to researchers as soon as possible. Though we have improved our numbers throughout the year, we’re also actively working on automation using the HackerOne API that will help shorten response times even further.
Overall, the past year was an exciting one, and we can’t wait to see what hackers dig up in the rest of 2022! If you’re interested in helping to make commerce more secure, visit our program page to start hacking, or head to our careers page to check out our open Trust and Security positions.
Happy Hacking!
- Shopify Trust and Security
Jenn Newton is a Manager on the Application Security team and currently leads the Bug Bounty team at Shopify. She's been on the Application Security team since 2016, working directly with engineering teams and bug bounty researchers to secure a wide variety of Shopify-developed apps. Visit Jenn on Twitter!
Wherever you are, your next journey starts here! If building systems from the ground up to solve real-world problems interests you, our Engineering blog has stories about other challenges we have encountered. Intrigued? Visit our Engineering career page to find out about our open positions and learn about Digital by Design.