PBwiki loves Core2Duo

7 Jan

We’re working behind-the-scenes to seriously ramp up our server fleet. When David started PBwiki the core servers were a bunch of 2001-vintage VA Linux boxes from Craigslist — Essentially “San Carlos: 400 pounds of 2U servers $10 OBO, you pick them up tonight”. Since those heady days of off-brand ramen and Mountain Dew we’ve accumulated a few generations of machines, with a handful of 0.5U Rackable units still providing low-load services. Today the bulk of PBwiki’s heavy lifting is done by a few white label 1U P4-3Ghz machines with twin SATA drives. This piecemeal approach has produced two interesting outcomes: a very capital-efficient company and a very diverse server stack. We’ve got hardware from several generations of chipsets — bogomips ratings span 1×1300 through 4×6100 and RAM from 512Mb through 4Gb. We’ve got IDE, SCSI, SCSI RAID, and SATA, plus 8 different ethernet controller chipsets.

Normally this kind of diversity isn’t such a big deal — just install whatever OS and let the drivers fight for survival on every boot. We add to the challenge with our security policy, which has us using a monolithic kernel, patched with grsecurity and identical across all the production machines. We use Debian which makes this a bit easier but it’s still a lot of things to keep straight.

We’ve recently embarked on a project to build One Heck Of A Server Cluster with more or less modern hardware bought new. PBwiki is the baby sister and she’s about to get her own prom dress. The funny part of all of this is of course the orders of magnitude we’re talking about versus the first dot-com boom circa 1999 — We’ll spend less for ten of these new machines than a single Sun E420, and each one is easily five or six times faster than the one E420.

So we ordered up a sample machine which had chips we were familiar with and specs and a price that made sense. The machine was a dual core P4 3Ghz with a single 500Gb SATA drive in your average white label 0.5U case. One great thing about modern hardware is the monitoring you get — temperatures, voltages, and fan speeds are all measurable if you’ve got the right driver code compiled into your kernel (see above). After some yelling at ssh terminals I got lm-sensors working on this new machine and we got some numbers out. As an aside, we’ve set up syslog so every machine in our system sends all of its health, security, and operational log events into a pair of machines with big hard drives and we have an ongoing record of everything (logins, temperature changes, hard drive health measurements, etc etc) in one place. Very handy.

Since it’s a dual-core machine running one thread of “while(1) {}” will occupy one CPU, and two threads of the same will run both CPUs at full throttle.

Intel Dual Pentium 4 @ 3Ghz
Ambient air temp 65F
Idle CPU temp 131F
One CPU running 176F
Two CPUs running 186F

So — those numbers are terrifying. The machine at idle raised the ambient temp by almost 70F, and running full steam the delta was a full 120F. We looked around for a T-max-ever spec for these parts and the general consensus was that our numbers were a bit high but ‘yep those chips run hot, don’t they’. Unbelievable.

So we decided to give the newer Core2Duo chips a try. They’re more expensive and require a slightly more modern motherboard and so on which means added cost there as well. They’re clocked a bit slower in general but between deeper caches and some other differences the real performance is actually equivalent or better. This new motherboard required a bit more yelling at the ssh terminal to get lm-sensors working (spot the trend?) All in all the extra few hundred bucks will be worth it — check out the new test results:

Intel Core2Duo @ 2.4Ghz
Ambient air temp 66F
Idle CPU temp 73F
One CPU running 87F
Two CPUs running 98F

Yeah – with both cores of the Core2Duo running full-throttle the chip surface is 90F cooler than the equivalent Pentium 4. Sold.

The heat output is really important for us because the Bay Area is out of power. We’ve got enough fiber optic cable running up and down the peninsula that you can trip over it if you’re not careful, but the limiting factor for adding machines to our server cages will be how much power they consume and how much heat they put out — since any heat burned off by the computers needs to be cooled by big expensive air conditioners eating up more power than your server burned through to keep the whole thing from melting down. It’s really the difference between being able to rack 5 of the Pentium 4 boxes or 10 of the Core2Duo ones.

In hindsight this makes a good deal of sense given the data I’ve seen from the Mac crowd (myself included) — the previous generation of MacBook Pro machines were almost painfully hot in general use. I’ve got a current-generation unit (with basically the same chip as in the second test box) and it’s never more than warm, even running several virtual machines at once and doing all sorts of divx decoding in the background. Very impressive work from Intel getting these chips to run more efficiently.

-Nathan

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: