5 Lessons Learned from June 29 2012 AWS Outage

Discussing a difficult situation is never fun, and I have been wrestling with how to start this post. It’s about revealing unpleasant cloud truths. And not necessarily the truths you might be expecting to hear. I am not here to preach, but my message to you is important. For the past five years I have been working on a project that uses the cloud to it’s fullest potential, celebrating the victories and learning from the defeats.

I’m speaking to my fellow Amazon cloud citizens. My co-tenants, if you will, in the “Big House of Amazon.” We’re all living together in this man-created universe with its own version of “Newtonian Laws” and “Adam Smith” economics. 99.99% of the time all is well… until out of the blue it’s not, and chaos upends polite cloud society.

If you lost data or sustained painful hours of application downtime during Amazon’s June 29 US-East outage, then you can only wag your finger in blame while looking in the mirror.

I know, I know, the cloud is supposed to be cheap AND reliable. We’ve been telling ourselves that since 2007. But this latest outage is an important wake up call: we’re living in a false cloud reality.

Lesson 1: Follow the Cloud Rules

Up front, you were told the “rules of the cloud”:

• Expect failure on every transaction
• Backup or replicate your data to other intra-cloud locations
• Buy an “insurance policy” for worst case scenarios

These rules fly against the popular notion that the cloud is “cheaper” than do-it-yourself hosting.

There is a silver lining to this dark cloud event. Everyone in the cloud will learn and improve so we don’t have to repeat this episode ever again.

Read more…

Part 2: FedRAMP, Inheritance and Key Controls

I am leading the FISMA project at Sonian, and we’re getting closer to achieving our first FISMA Moderate accreditation. For background on FISMA, read my first blog post on this subject.

With FISMA Moderate accreditation, Sonian will be able to manage non-defense government data. The accreditation is granted in the form of an “Authority to Operate (ATO)” bestowed upon a project by the government agency that will implement and utilize the product/service. A cyber security team within the government agency evaluates each project’s security documentation and gives the thumbs up or thumbs down. It’s an iterative process, that starts with extensive documentation, and audit, and government review and oversight. FISMA applies to both third party services purchased by the government, as well as internally developed and managed IT projects.

FedRAMP… Briefly

Currently, if a vendor wants to sell the same IT service to more than one government agency, FISMA requires an ATO from each agency, which adds time, complexity and cost to the procurement process. Historically, each agency has implemented and interpreted FISMA standards differently. The National Institute of Standards and Technology (NIST) devised the “FISMA Reference Architecture” for all agencies to follow, but in reality the local interpretation has varied. A “new and improved” accreditation standard is supposed to fix some of these issues. FedRAMP is a single umbrella guideline encompassing current FISMA rules, as well as updated rules that better align FISMA with technologies such as Software as a Service (SaaS) and cloud computing. When the legislation that created FISMA was drafted in 2002, SaaS and cloud computing were not on government technologist’s radar. FedRAMP is a modernization of FISMA, and also strives to streamline government IT purchasing, lower costs, and expedite project time lines. FedRAMP will benefit from FISMA’s first decade, so I am hopeful for an improved certification process when FedRAMP is officially ratified in about a year. There is already quite a bit known about FedRAMP and Sonian is working on a dual strategy to get FISMA Moderate for one agency, and then focus on FedRAMP for all other agencies.

Read more…