Cloudflare, a major CDN service, has accidentally leaked customers’ sensitive information for the past few months. On February 17th, Tavis Ormandy, from Google’s Project Zero, notified the CloudFlare security team of leaked data that he came across while working on a corpus distillation project. According to Ormandy, “This information included private messages from major dating sites, full messages from a well-known chat service, online password manager data, and hotel bookings. We're talking full https requests, client IP addresses, full responses, cookies, passwords, keys, data, everything.”
Cloudflare responded swiftly to Ormandy’s notification and within hours they had disabled new features of their service including: email obfuscation, server-side excludes, and automatic HTTPS rewrites – that had been found to make this problem surface. Cloudflare had to work directly with top search engines such as, Google, Yahoo, and Bing because their web crawlers’ caches had inadvertently stored the leaked data.
This type of leak is known as a buffer overrun and started in September when CloudFlare added a new HTML parser into its system. This program wasn’t the flaw, according to the CloudFlare team, the issue came up with its introduction. This introduction created a separate coding error that created the massive leak.
In the official CloudFlare incident report, John Graham-Cumming, explained why this issue took place in detail, “Unfortunately, it was the ancient piece of software that contained a latent security problem and that problem only showed up as we were in the process of migrating away from it. Our internal InfoSec team is now undertaking a project to fuzz older software looking for potential other security problems.”
The incident report shared that between February 13th and February 18th when 0.00003% of every page request through its network potentially allowed private information slip. On Y Combinator’s Hacker News Forum, Graham-Cumming clarified that his team found data leaked across 3,438 unique domains.
This leak has already been compared to Heartbleed – the 2014 computer bug that allowed sensitive data to be leaked in HTTPS sessions. Ormandy has already coined the nickname, “CloudBleed” for this 2017 leak.
The buffer overrun has been stopped and CloudFlare has gone on to scrub their code for any other leaks or issues. However, that does not mean that you are in the clear. Ormandy, after reading the incident report said that, “It contains an excellent postmortem, but severely downplays the risk to customers.” Today, downloading and caching content is common for many websites, not only the aforementioned search engines. There are plenty of crawlers out there that may have some of the leaked data from this incident without realizing it.
At this time, we are unsure if this leak has been used for harm. CloudFlare has made no recommendations for their customers at this time. We would advise that you take the time to change passwords and authentication credentials. It may also be of use to speak with an IT security expert to ensure your company is protected in case of another leak of this magnitude.
For more information on how to protect your business contact us.