Let's Encrypt is revoking digital certificates after discovering CAA bug

Let's Encrypt has discovered a bug in its Certificate Authority Authorization (CAA) code and will have to revoke millions of certificates today unless customers force a renewal of their certificates.

Any site that fails to renew its certificate will display security warnings to visitors until the problem is rectified. While no specific sites have been mentioned, with up to three million certificates involved, there is a chance that some high-profile sites could be affected.

The bug means that there is a problem in checking whether a Let's Encrypt subscriber has a valid security certificate for all their domains.  A mere two hours after the bug was discovered, a fix was pushed out, but certificates need to be renewed for this to have any effect.

Let's Encrypt posted a security advisory about the bug discovery:

On 2020-02-29 UTC, Let's Encrypt found a bug in our CAA code. Our CA software, Boulder, checks for CAA records at the same time it validates a subscriber's control of a domain name. Most subscribers issue a certificate immediately after domain control validation, but we consider a validation good for 30 days. That means in some cases we need to check CAA records a second time, just before issuance. Specifically, we have to check CAA within 8 hours prior to issuance (per BRs §3.2.2.8), so any domain name that was validated more than 8 hours ago requires rechecking.

The bug: when a certificate request contained N domain names that needed CAA rechecking, Boulder would pick one domain name and check it N times. What this means in practice is that if a subscriber validated a domain name at time X, and the CAA records for that domain at time X allowed Let's Encrypt issuance, that subscriber would be able to issue a certificate containing that domain name until X+30 days, even if someone later installed CAA records on that domain name that prohibit issuance by Let's Encrypt.

We confirmed the bug at 2020-02-29 03:08 UTC, and halted issuance at 03:10. We deployed a fix at 05:22 UTC and then re-enabled issuance.

Our preliminary investigation suggests the bug was introduced on 2019-07-25. We will conduct a more detailed investigation and provide a postmortem when it is complete.

In a separate post, Let's Encrypt shares details of affected serial numbers and a link to a hostname checking utility for anyone who wants to check whether their domains could be hit.

Image credit: deepadesigns / Shutterstock