[SASAG] New Data Facility in Lynnwood

Michael T. Halligan michael at halligan.org
Fri Mar 30 17:02:33 PDT 2007

> Lots of fun to stop by yesterday, and when it's done, I'm sure this  
> will
> shape up into being a great facility. My observations:

Sadly, I missed the party. Too bad, since I seem to owe quite a few  
LOPSA members
drinks, and I was hoping to do it on the vendor's dime :)  I'm  
chatting with their sales
rep now, so I figured I'd ask some of your questions.

> - Facility is a former REI building which was built using lots of
> (combustible) wood. Pleasant to look at, but I worry a lot more about
> fire control in a building like this.
> - Corollary: fire control is a sprinkler system (water). Yikes! Not  
> sure
> whether there are plans to replace this? Hope so...

Water-based sprinklers are fairly common in datacenters.  Equinix,  
365Main, and at least 1/2 dozen other facilities I've put gear into  
in the past 7 years have all had all had water based sprinklers. My  
real concerns lay in the architecture of the fire supression. Is it a  
dry-pipe system? How fine-grained are their zones? Is the system  
monitoring for smoke via sensors at the CRAC intakes?

I've verified NetRiver's supression system is a pre-action, dry-pipe  
system (which I just verified verbally. I'm in talks with them about  
doing a rack as an off-site storage location).  That being said,   
there are disadvantages to the various gas-based fire suppression
units including, well, death to anybody stuck inside the datacenter.

I do understand the business reasons to use water versus  gas. I was  
speaking to someone at DRT last year, who was telling me how a genius  
@ 200 Paul in SF decided to clean up some construction dust with a  
gas operated airblower, setting off the FM2, causing an $100k loss  
between cleanup, refill, and re-certification. With the gas based  
systems, my understanding is that the release is all or nothing. You  
need to release enough into the room's atmosphere to suppress flames,  
no matter how small the triggering incident is.

> - Not sure how much fuel is on hand or storage capacity/run time.
> NetRiver was pretty confident in their fuel service contract, but  
> in an
> emergency, this won't be worth the paper it's written on. Just ask the
> DirectNIC guys!

 From the tour I took the other week, it looked like 1200 gallons,  
with a planned increase before their 2MW generators get in. Speaking  
to their architect, and looking at the blinkies on their UPS, they  
appeared to have capacity to spare for the near future (depending on  
their growth rate), and
I think they will do the "Right Thing" as capacity needs increase.  
This is certainly something that needs
to be specified contractually.

That being said, if Seattle has the same problem that the DirectNIC  
guys ran into,  well most of us will
probably have little care about our jobs, and more care about how  
loaded our shotguns are.

> - Currently connected to Snohomish PUD, but only on one grid.  
> Redundant
> grid connection is in the works, but not installed yet, leaving the
> facility vulnerable to localized outages in the interim.
> - Fiber to Westin Building via Global Crossing. 30Gbps theoretical
> capacity, but only one strand is lit. I understand (although was
> skeptical to hear this--is it really this bad?) that there is no
> redundant connectivity of any sort into the building. Being one  
> backhoe
> incident away from disaster definitely isn't somewhere I'd want to put
> anything critical.

30GBS is an odd number.. I thought you could split a strand of fiber  
into 30 lambdas, each of which can do 10gb/s? I'm afraid I didn't  
peak in at their routers while I was there, since I was more   
concerned with the space.

Their hard date for their second grid is May 15th.  My big concern  
was that currently their primary link (like every other datacenter in  
Seattle) goes soley through the Westin. Their next pipe will actually  
take them to their other datacenter in Edmonds, which I believe has  
connectivity to 1000 denny and somewhere in Eastern Wash.  A strong  
part of their pitch is that they're trying to make their network  
tolerant to Westin problems, both from a DR point of view, and the  
fact that the Westin is by far the busiest shop in the area, and is  
prone to construction issues.

> - Access controls are pretty relaxed, and collocation isn't in locked
> cages. This isn't necessarily a problem, as long as all visitors to  
> the
> facility are escorted and monitored throughout their visit.  
> However, it
> might be hard to scale this security approach if it gets busy.

Granted, their model does not allow for cage space, they do maintain  
of their cabinets, and have told me they'd gladly do custom upgrades  
for keypad
or biometric security devices.  That being said, two f/t security  
guards in their lobby
would go a long way towards alleviating any human bottlenecks for  

> - Emergency exit door does not appear to be alarmed, and is a  
> vulnerable
> spot.
> - Facility supports the use of seismic base isolation platforms.
> Although NetRiver sells this gear, they don't appear to use it on  
> their
> own racks. A tepid endorsement at best. :)
> - Shared hosting blade server is connected to a small HP SAN (can  
> scale
> to 30TB, built out to 1TB now). The SAN was installed by an HP
> consultant so I assume it was configured properly, but I couldn't tell
> whether there were redundant HBAs on the blade server (I couldn't see
> from outside, and didn't want to open up their rack and start poking
> around). Prior to purchasing a VPS or blade from NetRiver, informed
> consumers should probably ask for more details.

Quick point of correction.   SAN, and it's currently built out to  
5TBs, scaleable to 85TB.  Also, they
do have multiple HBAs on it.

> The CEO of NetRiver announced that, having successfully completed this
> facility (although "completed" is perhaps debatable based on the above
> noted concerns), he's already planning to build another one. Clearly,
> NetRiver wants to grow rapidly; potential customers should be  
> cognizant
> of the growing pains this can entail.

As someone currently spending $250k moving out of his old datacenter  
(n SF)
and into a new datacenter (South Seattle), it's nice to know that the  
is looking to build more capacity in the area.

> Overall, the facility is about 90% complete at this point, and I think
> it'll be really nice when it's done and fully staffed (they're still
> short a couple of NOC technicians).
> -TProphet

A note here.. NeRriver will gladly let you bring your own local loop  
in, which I'd likely do from MFN
and ATT, so I'm less concerned about the fact that theyir NOC is  not  
staffed 24/7, since (I believe)
their NOC doesn't actually handle the power systems. In this model,  
NetRiver would just be providing
Space, Power, and Cooling.

I'd actually place NetRiver above some of the larger facilities in  
the area like Switch & Data (who makes it
clear that they don't want your business if you need less than 50  
racks worth of space), and some of the
smaller players I've toured as well.

> Josh Fulleton wrote:
>> Thanks for all the hype!  It will be a great event, and I am looking
>> forward to meeting some of you.
>> Don't hesitate to give me a call in the future if you can't make  
>> it, I
>> will be more than happy to give private tours.
>> Josh Fulleton - Account Manager
>> NetRiver Corporation
>> 425-741-7014 Office
>> 425-967-0066 Direct Desk
>> www.netriver.net
>> -----Original Message-----
>> From: members-bounces at lists.sasag.org
>> [mailto:members-bounces at lists.sasag.org] On Behalf Of Jim Hogan
>> Sent: Monday, March 26, 2007 5:46 PM
>> To: Josh Fulleton
>> Cc: members at lists.sasag.org
>> Subject: Re: [SASAG] New Data Facility in Lynnwood
>> Josh,
>> I am going to do my best to attend.
>> I just wanted to help out a bit -- do what I could to raise  
>> expectations
>> :)
>> Thanks,
>> Jim
>> On 3/26/07, Josh Fulleton <josh.fulleton at netriver.net> wrote:
>>> Jim,
>>> I am not exactly sure what APC is going to be giving away as of yet.
>>> Once I find out I will be sure to post exactly what.  I know  
>>> there are
>>> going to be some nice items that will be raffled off.  Overall it is
>>> going to be a great event to network with people, get some free  
>>> food,
>>> and also learn more about what we are up to over here in Lynnwood.
>>> Josh Fulleton
>>> NetRiver Corporation
>>> 425-741-7014 Office
>>> 425-967-0066 Direct Desk
>>> www.netriver.net
>>> -----Original Message-----
>>> From: members-bounces at lists.sasag.org
>>> [mailto:members-bounces at lists.sasag.org] On Behalf Of Jim Hogan
>>> Sent: Saturday, March 24, 2007 11:15 AM
>>> To: Josh Fulleton
>>> Cc: members at lists.sasag.org
>>> Subject: Re: [SASAG] New Data Facility in Lynnwood
>>> Josh,
>>> Awesome.  I love coffee mugs.  Now did you say that you are having a
>>> drawing for a custom SUA1500 with included AP9617 and a custom
>>> silk-screened front panel with blue LED status lights?  I am
>>> *definitely* driving to Lynnwood if that rumor is correct.
>>> Jim
>>> On 3/23/07, Josh Fulleton <josh.fulleton at netriver.net> wrote:
>>>> Hello Everyone,
>>>>                 I am new to this list and want to first start by
>>> saying
>>>> hello.  So Hello!  Anyways, I want to let everyone know that the
>>> company I
>>>> work for is having an Grand Opening next week, and guess what, you
>> are
>>> all
>>>> invited.   This is an APC sponsored event so there will be lots  
>>>> free
>>> food
>>>> and beverages, not to mention tours of our brand new High Density
>> Data
>>>> Center.
>>>> We will be doing live presentations, Tours, Q&A Sessions, and
>>> Giveaways.
>>>> The first 600 people will receive an APC/Netriver Mug so come early
>> to
>>> make
>>>> sure you get one!
>>>>  We will are RSVPing so if you want to come fill out our online  
>>>> form
>>> at:
>>>> http://www.netriver.net/rsvp.html
>>>> if not don't worry about it just show up, see below for the  
>>>> details:
>>>> Thursday, March 29th , 2007 1:00pm - 8:30pm
>>>> Ribbon cutting at 3pm, Seminar presented at 6pm
>>>> Where:
>>>> The NetRiver Building
>>>> 4200 194th Street SW
>>>> Lynnwood, WA 98036
>>>> Ground Floor
>>>> Josh Fulleton
>>>> NetRiver, Inc.
>>>> 425-741-7014 Office
>>>> www.netriver.net
>>>> _______________________________________________
>>>> Members mailing list
>>>> Members at lists.sasag.org
>>>> http://lists.sasag.org/mailman/listinfo/members
>>> --
>>> -*-  Jim Hogan
>>>      Seattle, WA
>>> _______________________________________________
>>> Members mailing list
>>> Members at lists.sasag.org
>>> http://lists.sasag.org/mailman/listinfo/members
> _______________________________________________
> Members mailing list
> Members at lists.sasag.org
> http://lists.sasag.org/mailman/listinfo/members

More information about the Members mailing list