Let’s Encrypt x Shopify: Securing the Web 4.5 Million Domains at a Time

  • 3 minute read

On June 30, 2021 Shipit!, our monthly event series, presented Let’s Encrypt and Shopify: Securing Shopify’s 4.5 Million Domains. Learn about how we secure over 4.5M Shopify domains and team up to foster a safer Internet for everyone. The video is now available.

It’s already been six years since Shopify became a sponsor of Let’s Encrypt.

In 2016, the SSL team started transitioning all of our merchants' stores to HTTPS. When we started exploring the concept a few years earlier, it was a daunting task. There were few providers that could let us integrate a certificate authority programmatically. The few that did had names like “Reseller API.” The idea that you would give away certificates for free and no human would be involved was completely alien in this market. Everything was designed with the idea that a user would be purchasing the certificate, downloading it, and installing it somehow. It’s a lot more problematic than you might think. For example, a lot of those API return human readable error messages instead of having a defined error code. Normally, they would expect the implentor to send back the message to the user trying to purchase a certificate, but in a fully automated system there is no user to read anything. For Shopify, all 650,000 domains would get a certificate, and they would be provisioned and renewed without any interactions from our merchants.

I first heard about Let’s Encrypt in 2014. A lot of the chatter online was around the fact that they would become a certificate authority providing free certificates (they were pretty expensive until now), but a bit less about the other part of the project, the ACME protocol. The idea was to fully automate the certificate authorities using standardized APIs.

In the summer of 2015 they still hadn’t launched, but I started to write a Ruby implementation of the ACME client protocol on the weekend to get a feel for it. I’d already been through this exercise a few times with other providers. Working from a specification was pretty refreshing. They’re boring documents, but when trying to automate hundreds of thousands of domains that you don’t really control, you want to know that you have all your exceptions accounted for. That’s when we reached out to them to figure out how Shopify could help and agreed on a sponsorship. We didn’t intend to make use of their service, at least not in the immediate future, but we share value around the open web and the importance of removing barriers of entry using technology.

Interacting with a small organization that does their work fully open was also quite refreshing. My experience dealing with certificate authority would be to work with an account manager who forwards my question to a technical team. The software they run is usually not implemented by them, so there is a limit to how much they can answer questions. Let’s Encrypt being fully open changes the dynamic. I asked questions on IRC and they answered me with github links that point at the actual implementation. I reported bugs or inconsistencies in the specification, and they tagged me in the pull request that fixed it.

In late November, we started rolling out our shiny new automated provisioning system. We immediately ran into some scalability issues with our initial providers. We did some napkin math with the throttling they were imposing on us, we would need about 100 days to provision every domain. We let it run over the holidays and launched in February 2016.

The team was already engaged in its next mission but in the back of our mind we knew we needed to revisit this. Now that the bulk of the domains were done, new domains would come at a slower pace and eventually renewal, but that would be good for a while at our current growth projection. Our main concern was emergency rotation. If for some reason we had to rotate our private keys or the certificate chain was compromised somehow, we’d be in trouble. A 100 days is too slow to react to an incident.

We needed to be more responsive for our merchants, and that’s why we decided to add Let’s Encrypt as a backup option. We were able to roll Let’s Encrypt out in a few hours compared to months with our original providers. The errors we ran into were predictable because of their specification and server implementation being open source, so we could refer directly to it to debug unexpected behaviour. It was so reliable that we decided to make them our main certificate authority.

Let's Encrypt is a game changer for the industry. For a big software-as-a-service company like Shopify, it saves time because their implementation is built around an open specification. You can even change or add a new certificate authority that supports the ACME protocol without redesigning or having to change your entire infrastructure if you wanted to. It's more reliable than the API from the past because it's designed to be fully automated from the beginning.

Shipit! Presents Let’s Encrypt and Shopify: Securing Shopify’s 4.5 Million Domains

Shipit! welcomes Josh Aas, co-founder and Executive Director of Let’s Encrypt and Shopify’s Charles Barbier, Application Security Development Manager, to talk about securing over 4.5 million Shopify domains and teaming up to foster a safer Internet for everyone.

Additional Information

Charles Barbier is a Developer Lead for the Application security team. You can connect with him on Twitter.

We're planning to DOUBLE our engineering team in 2021 by hiring 2,021 new technical roles (see what we did there?). Our platform handled record-breaking sales over BFCM and commerce isn't slowing down. Help us scale & make commerce better for everyone.