Looking at your specs, I personally would chose OpenBSD (http://openbsd.org) and commodity i386 or amd64 rack servers (check the hardware compatibility list to avoid nasty surprises).
Besides a stateful packet filter "pf" functionality, OpenBSD comes with many tools to cluster those servers, such as "pfsync", CARP (VRRP replacement), rapid spanning tree (STP), NIC trunking/teaming, etc. Also a powerful layer-7 reverse proxy/load-balancer/SSL accelerator "relayd" is readily integrated. You may also want to run RIP, OSPF or BGP daemons if some support for dynamic routing should be required.
In an enterprise security application, I have implemented a large two-stage clustered firewall & Internet services gateway using exclusively OpenBSD on SunFire X4100M2 rack servers. The outer cluster pair is an "invisible" filtering bridge, the inner cluster pair is operating as a filtering router which also includes the reverse proxy, resilient OpenVPN SSL gateways, and other critical infrastructure services such as NTP, DHCP, named, etc.
I am also aware of similar OpenBSD setups in very large commercial Web / e-commerce server farms.
At home, I run nearly the identical setup on small embedded i386 machines from PCEngines.ch at leass than 40 Watt total power consumption. Such a setup is also ideal for pre-production tests in the lab, before implementing changes on the "heavy irons".
Prior to that, I had extensive exposure to CheckPoint clusters on Solaris, as well as to Linux/iptables based systems, such as Astaro. In my opinion, OpenBSD beats them all hands down, at least in my setups, in terms of security, stability, life-cycle, innovation, scalability, and price-performance ratio.
Unless you want/need graphical user interfaces for administration. Then my second choice would be m0n0wall or pfSense, both based on FreeBSD.
Regards, Rolf