Tips For Picking A Data Center Part 1: Lease vs Build, Tiers, and Redundancy

As many of you may know, in my past life I was a CTO of a company that built high security mission critical data center facilities as well as owing and operating data center facilities.

I often talk with clients who are struggling with choosing data center space, so I wanted to give you a few tips when you are on the market looking. I have seen companies make a lot of mistakes over the years.

Lease vs. Build

First, I have seen many companies want to build a data center in their office building. In this day and age this is almost always a mistake. Many company’s real estate groups will try to roll the build out of the data center into the TI, or tenant improvement, budget and amortize the costs over a 10-15 year or longer lease and say that is cheaper than leasing colo space. This fails to capture the true costs and compromises you will encounter when owning and operating your own facilities, and it is very unlikely you can ever achieve the efficiencies and economies of scale that a dedicated facility can.

People talk about tier ratings, and it is very difficult to build a Tier 3 and, for all intents and purposes, impossible to build a tier 4 equivalent facility. First problem is that having your office space with your data center facility is not a best practice, and can introduce many problems. 

An enormously important part of building a facility is on-site power generation, typically diesel generators. There are a lot of problems putting generator of sufficient capacity for a data center in commercial facilities. The size and weight of generators becomes a problem, the noise is a problem, the smoke and particulates that they emit are problematic, and don’t forget, you need to test generators at least monthly probably more often in cold climates. You are likely going to need at least two generators for redundancy, so double all of that. 

Perhaps the biggest challenge is fuel storage. You need enough fuel onsite to weather an extended outage. You need to ability to deliver the fuel, so you need space for the truck. In an emergency situation, getting fuel delivered to a busy downtown may be impossible. Many generators end up on roofs where you need to pump the fuel to day tanks. You can’t store too much fuel on the roof, because it can create a serious fire hazard. Usually your fuel storage tanks are underground. This storage comes with EPA regulations and inspections. Also, don’t forget that Diesel fuel can be difficult to maintain over time, if you aren’t burning it regularly, you can have problems with fungus and bacteria. You really have to know what you are doing when you have a diesel generator, service contracts alone just don’t cut it.

Let’s talk about some of the terms and concepts you will hear frequently discussed when looking at facilities.

Tiers

You will likely hear the quality of data centers expressed in terms of Tier ratings. Most of the dedicated collocation facilities will say they Tier 3 at a minimum, rarely do you encounter true Tier 4 facilities in the wild, because they often don’t make economic sense.

The Tiering system was popularized by Uptime Institute, and this is typically what people are talking about when they talk about tiers. However, to actually have an Uptime Institute Tier rating requires an audit performed by the Uptime Institute, and the vast majority of facilities never go through this process, and are technically not eligible to advertise themselves as such.

The tier levels are:

Tier I - Basic site infrastructure (non-redundant)
This is basically your repurposed office space turned into a data center, offering little or no redundancy. They feature 99.671% Uptime which is about 28.8 Hours of downtime per year.

Tier II - Redundant capacity components site infrastructure (redundant)
These facilities have partial redundancy in their cooling and electrical systems and feature 99.749% Uptime, which translates to about 22 hours of downtime per year

Tier III - Concurrently maintainable site infrastructure
These facilities have the ability to take systems offline for maintenance activities. This is where you start getting into the 3 nines and above range with a 99.982% uptime rating or about 1.6 hours of downtime annually and they feature at least a N+1 architecture and are rated for at least 72 hours of onsite generation protection.

Tier IV - Fault tolerant site infrastructure
These are your minimum 4 nines facilities, with 2N+1 redundancy across all their infrastructure. There are a ton of very costly requirements for 4 nines in terms of not only equipment but staffing, capabilities and training. These are not very common on the market as the economics of them are hard to justify. The money spent going from a tier 3 to a tier 4 facility is usually better spent on other areas to improve operations and capabilities.

Tier 3+
Another term you will hear a lot is Tier 3+. While this is technically not a thing, there is no Tier 3+ Uptime Institute certification, what they are describing is a facility that meets a Tier 3 standard, but has some of the enhanced features of a Tier 4 facility. Typically this comes in the form of 2N and 2N+1  features.

Much of the Uptime Institute standard grew out of work the Telecommunications Industry Association TIA 942-A standard which is now the TIA 942-B standard. These are good guidelines for best practices in building facilities. All of these standards are out there and available for you to review.

Redundancy

I have mentioned N, N+, 2N and 2N+1, so let me define these terms a bit. N is your basic non-redundant system. I need 2 generators, I have two generators. N+ refers to a situation like I need 5 generators and I have 6 so if 1 fails I have another one to bring online. This would be N+1, if you needed 5 and you had 7 generators it would be N+2 and so on. 2N would be I need 5 generators and I have 10. 5 are in production and 5 are in standby waiting to take over if a system goes down. This is your A/B sides, where you can take the A side down for maintenance and run on B. 2N+1 is where you have a fully redundant set of infrastructure plus extra to run as a still redundant system even when you are in maintenance mode.

2N systems are often what you see in a Tier 3+ scenario. When we talk about redundancy like this one must consider the complete delivery path and whether it is 2N. Power systems may be 2N in generation, but the complete delivery path may lack separate independent delivery paths. Typically in cooling systems you may have 2N on the CRAC (Computer Room Air Conditioner) or CRAH (Computer Room Air Handler), but the delivery ductwork is non-redundant due to the difficulties of building redundant cooling delivery systems.

I have built facilities with ducted or plenum cooling as one side with in-row cooling systems to provide a diverse, independent and redundant 2N cooling path for the other side, but this is not common.

Understand your Architecture

The most important thing is to understand your needs and your budget. With modern software architectures, high availability facilities may not be as important. This is the micro services vs mainframe perspective. If you have large monolithic systems, you probably need highly engineered and redundant systems, but if you have a distributed, stateless micro services architecture, availability is probably much lower on your priority list and you should take that into consideration as you could likely save some money.

This is going to be a multi-part series since there is so much to talk about here. The next videos are going to cover what is generally referred to as MEP systems in the business, or Mechanical, Electrical, and Plumbing which would cover your electrical and cooling systems. I will cover company finances, facilities, fire suppressions and detection systems, security systems and practices, Networks, additional services and ultimately, some tips for negotiating a contract.

If there are any topics you would like to have me talk about, let me know in the comments.