Think of CloudBleed as sitting down at a restaurant, supposedly at a clean table, and in addition to being handed a menu, you’re also handed the contents of the previous diner’s wallet.
Content Delivery Networks (CDNs) are like enormous ball bearings for the Internet – without edge caches to speed the flow of pictures and other static content, the whole monstrous collection of tubes would grind to a halt. Yet the CloudBleed bug involving the CloudFlare CDN illustrates that there are security trade-offs in using CDNs to improve performance.
A CDN is a type of overlay network that improves end user experience by moving certain web site operations closer to the end user. Common services offered by internet overlay networks include edge caching, SSL offloading and edge routing. Overlay network providers include Cloudflare, Akamai, Amazon CloudFront and Teridion (where I am CEO).
Internet overlay networks allow web site providers to leverage third party infrastructure to improve performance and security. Rather than having to build a large number of regional data centers, a web site provider can “rent” overlay network infrastructure at a fraction of the cost.
This document describes potential security concerns for overlay networks and CDNs, along with alternatives for reducing those vulnerabilities while still accelerating internet performance.
Security Considerations for CDNs and Overlay Networks
There are three key architectural issues that web and mobile application providers should consider when assessing security issues for overlay networks:
- Stateful vs Stateless Overlay Networks: the type of edge service performed by the overlay network (e.g., caching, routing, SSL offload) matters greatly, as storing sensitive content across many edge nodes raises security risks.
- SSL key required vs Keyless Overlay Network: requiring SSL keys can provide performance enhancement but at the cost of opening new security vulnerabilities, particularly if content is being cached within edge nodes.
- Shared vs Shared Nothing Overlay Networks: shared infrastructure means other people’s problems can become your problem. Isolation can reduce this risk.
Stateful vs Stateless Overlay Networks
Most CDNs cache content at many geographically distributed locations. In contrast, some overlay networks are “stateless”, with no sensitive content stored in the edge nodes. Caching static and publically available content in a CDN has very low risk. For example, public images, videos and fonts are commonly and safely cached in CDNs.
To address these security issues, there are two types of stateless overlay networks:
- SSL offload: an edge node can handle the SSL handshake with the end user on behalf of the origin server. This can greatly decrease the overhead of supporting an SSL connection, but at the cost of requiring the origin web site to share their SSL keys.
- Edge routing: an edge node can handle TCP termination and optimize routing back to the origin server. This can improve performance without requiring SSL certificates.
SSL Key Required vs Keyless Overlay Networks
An overlay network can perform SSL handshaking with end users through an edge node, but this requires an SSL certificate from the origin web site. Although traffic between the end user and the edge node is encrypted, content within the cache is generally not encrypted, making these caches a potential target for hacking attempts.
Overlay networks that can accelerate traffic without requiring SSL certificates reduce security risks for web and mobile app providers. CloudFlare is one of the leaders in promoting keyless SSL as a more secure network overlay architecture.
Shared Everything vs Shared Nothing Overlay Networks
Overlay network vendors typically make significant investments in security infrastructure. Yet their very size and success attracts attack. In short, CDN Points of Presence (POPs) increase the attack surface for potential hackers. This risk to web sites grows even more when they share SSL certificates with an overlay network provider.
The benefit of a CDN is to limit network congestion in serving up static content. However the CDN security risk is that any user with root-like permissions on a CDN server can access and replace content. This in turn requires that CDN customers trust the security for every CDN POP.
In addition, having many web sites share the same overlay network infrastructure creates additional security challenges. For example, Cloudflare operates a large, shared infrastructure, which meant that even though only about 3,000 of CloudFlare’s web sites had malformed HTML tags, the resulting bug leaked private data from over 7 million web sites.
A different approach is for overlay networks to operate separate and isolated virtual networks on a per customer or even per URL basis. This “shared nothing” overlay network helps ensure that even if one virtual network is compromised, no other networks will be affected. With resource isolation between customers, a bug like CloudBleed poses a far lower risk.
Towards More Secure Overlay Networks
While CDNs will undoubtedly remain popular for caching static, public files, other approaches should be considered for more sensitive or dynamic content. The following table summarizes how different overlay networks compare for managing highly secure content.
|Overlay Network Security Requirement||Edge Routing||SSL Offload||Edge Caching|
|Stateless – no data cached at edge||Yes||Yes||No|
|Dynamic – accelerates all content||Yes||Yes||No|
|Keyless – no SSL key required||Yes||No||No|
|Isolated – no shared infrastructure||Yes||No||No|
One example of an overlay network that works based on edge routing is Teridion. The Teridion solution is focused on accelerating both dynamic and static content across the internet without compromising security. The following chart shows the elements of the Teridion solution.
In summary, the CloudBleed bug has raised awareness of the potential security issues associated with distributing content and SSL keys via CDN. In the future, web and mobile application providers should look at a variety of factors to determine the optimal overlay network solution that meets their requirements.
“This is the tiniest compromise of Cloudflare. A moderate compromise of Cloudflare could be an internet-threatening incident.” – Ryan Lackey, former Cloudflare employee