Upgrading CDN infrastructure | Deliver modern CDN & WAF scaling capabilities

This article explains how BlueGrid modernized a legacy CDN and WAF stack using virtualization, Lua-powered WAF rules, and full SSL chaining, achieving a 40% performance boost and seamless scaling across PoPs.

Speed is table stakes, but flexibility and scale win the long game. That’s what our client – an undisclosed player in edge compute – had in mind when they asked us to help rethink their PoP (Point of Presence) strategy. The ask: replace aging edge infrastructure with something faster, lighter, and more secure.

In fact, we didn’t just upgrade a few boxes – we rebuilt the entire flow, down to how the frontend talks to the WAF. Here’s how.

Modernizing Legacy CDN Infrastructure

However, the existing setup was a patchwork of Version 1, Push boxes, and a delivery pipeline that had grown too rigid. It needed a refresh:

Broke monoliths into discrete, virtualized layers
Integrated filtering (WAF) into the content delivery (CDN) infrastructure
Delivered high performance under load
Made future scaling and security updates easy

Our Approach: Segment, Virtualize, Automate

Therefore, we designed a PoP architecture where form follows function. In practice, five physical servers per site:

3 for CDN – each running 4 VMs with Varnish and NGINX.
2 for WAF – 5 VMs total, with NGINX + Redis + Lua for dynamic filtering.

All running on KVM, provisioned via virsh. The result? A lean, redundant, high-performance setup that plays nicely with DDoS protection and is future-proofed for growth.

Inside the CDN Layer

Each frontend (FE) runs Varnish, caching requests with a smart 30s TTL that collapses duplicates and eases backend strain.

Backend nodes (BE) run NGINX reverse proxies, routing to either the origin or the WAF depending on the configuration. Cache keys are carefully structured to optimize reuse.

We used a hybrid 3FE/1BE or 2/2 split on each server, controlled by director objects that dynamically map FEs to BEs for balanced load and failover.

Integrating WAF Directly into the CDN Layer

On the WAF side, traffic filtering happens in Lua-loaded NGINX phases – rewrite, access, headers, body, and log.

Redis stores the rules in JSON, organized by category (XSS, SQLi, misconfig). Moreover, rules are updated in real time from the control panel – no need for deploys.

Filtering is surgical and smart, powered by handcrafted regexes and dynamically applied logic.

Example? Here’s a rule that blocks classic XSS vectors:

{
  "category": "xss",
  "regex": {
    "patterns": ["<\\s*object\\s+(classid|type)\\s*=\\s*[\\s\\S]*<\\s*/\\s*object\\s*>"]
  },
  "action": "Block",
  "name": "XSS Attack Prevention"
}

SSL All the Way Down

External traffic terminates SSL at the CDN frontend
Internal traffic (CDN → WAF) also runs over SSL using private subnets
Certificates are defined in NGINX, even internally, for full-chain encryption

Traffic Flow: Fast and Redundant

BGP Multipath via Bird keeps traffic flowing across all routes
Juniper switches anchor the routing core
Health probes dynamically prune dead nodes from routing tables
Redis and NGINX logs provide deep observability across the board

Automating Edge Scaling and Failover

To ensure resilience, each PoP pairs with a geographic sibling (e.g., JFK-WDC, LAX-SJC) for zero-downtime maintenance. We manage failover and rollouts through DB-controlled configs and Ansible playbooks.

Results

✅  Collapsed requests reduced origin load significantly
✅  Virtualization boosted hardware utilization
✅  Lua-powered WAF allows near-instant rule updates
✅  PoPs are modular, secure, and easy to scale