[Seattle-SAGE] Server room build questions

Ski Kacoroski kacoroski at comcast.net
Thu Apr 28 17:44:24 PDT 2005

Paul English wrote:
> I finally have a chance to set up a server room properly and I'd love to 
> get some tips and feedback on what, how and who. 

First get an architect/electrician/hvac group that has done server rooms 
before and then visit some of the rooms they have done.  This is the 
most important thing you can do and they can answer many of the 
questions you list.  Server rooms are a speciality item and folks who 
have not done them before will make lots of mistakes.  No matter how 
much you specify things, you will still lose out if the vendor does not 
believe you and tries to do things the "normal" way.

Second, get a visit to the Westin or Fischer buildings in Seattle.  If 
nothing else call a colo vendor and ask to see their server room.  Ask 
them questions about what they like and do not like about it.  This will 
provide you background for interviewing the folks above.

> We are looking at signing a 5 year lease and as part of the lease having 
> the landlord build the server room space for us. I'm keen to be very 
> specific about what we want to ensure that we don't end up getting stuck. 
> We currently have 4.5 racks that we've built up over the last 3 years, and 
> I think we will be adding 1 more rack in the next 6 months. We are looking 
> at getting enough space for approx 20 racks which should be more than 
> enough.
> We have machines in 3 categories:  
> 3/4 rack of 24/7 critical customer-facing systems, network switches, T1 
> CSU/DSU, etc. This probably won't be growing much.
> 1 rack of "operational" systems which can be down for 24 hours max. These 
> are growing slowly to maybe 2 racks by the end of 5 years. 
> All the rest which are growing the fastest are research style computational 
> nodes which can be shut down for a day or two if necessary, and individual 
> nodes can be up and down at various times with no issue. 

Treat all the machines the same when you are putting in the server room. 
  Unless you are omniscient, the ratios above will change several times 
so if you try to optimize parts of the server room for certain systems, 
you will quickly find that you need more room (power/HVAC) for the 24/7 
systems than what you put in.

> So on to the questions:
> Given a commercial office space, does a roof-mounted AC make the most 
> sense?

I would go with roof A/C as the in plenum A/C's will not be big enough 
and they have a nasty habit of leaking onto equipment.  I do not like in 
  room A/C's except for emergencies as they take up valuable space.

> What are people's feelings on raised floors vs. tiles/anti-static carpet? 
> I'm very opposed to running network/kvm cables under floors (give me a 
> ladder rack any day), but raised floors can be handy for routing AC to the 
> front of the racks and running fat trip-causing power cables. 

Skip raised floors unless you have lots of $$$.  We compromised by 
making raised racks (e.g. we built a 1 foot high wall and put the racks 
on it.  Then we blow cool air into this space and the return is through 
a cap on top of the racks into the ceiling plenum.  The lower space also 
allows us to run the power in it.  Finally, it raises the lower machine 
a foot off the floor which I like better.

> How much AC should I spec - get the maximum capacity 20 rack's worth now, 
> or get a lesser amount now and plan to upgrade?

The more the better.  Double whatever you think you need.  I have been 
in several places that ran out of A/C and it is not pretty.

> Should I spec out a backup AC? Or just plan on shutting down all of the 
> non-critical systems in the event of an AC failure. 

We have several monitors so if the A/C fails we get notified.  The 
machines shut down if the Temp gets too high (hopefully we will get 
there first).

> How much power should I spec? On this one I'm fairly sure that I should 
> probably spec the full 20 rack's worth (200A?) and just get the breaker 
> panel put into the room with circuits sufficient for 6 racks. 

Again, the more the better.  If you cannot get a good 
architect/electrician I would spec the racks by taking the power 
requirements for something like a blade server (high power density) and 
use that to calculate the power for each rack (assume they are filled 
with these).

> What are the considerations for a backup generator vs large (Liebert?) 
> UPS? I'm leaning towards an auto-start generator right now, with small 
> rack-mount UPSen on the 24/7 critical systems. 

If possible, get a large UPS and generator combo.  The vendor can help 
with this.  A whole bunch of small UPS's will be a nightmare to manage 
and then to replace all the batteries, etc. after 5 years.

> Do we bother with a fancy dry fire supression system? Seems like a major 
> PITA and would take a big chunk out of the budget for everything else. 

I would spend the $$$ on a hot remote site for the critical servers instead.

"When we try to pick out anything by itself, we find it
  connected to the entire universe"            John Muir

Chris "Ski" Kacoroski, kacoroski at comcast.net, 425-681-0092

More information about the Members mailing list