Archive for the ‘Amazon Web Services’ Category

Cloud Cost Savings In Action

This morning Amazon Web Services notified its cloud customers a new CPU configuration is available in all regions. This new virtual CPU type is hi1.4xlarge, and is significant in a number of ways. Amazon heard from customer a high I/O, low latency configuration would be ideal for applications like relational and NoSQL databases. It’s also the first EC2 instance type to use SSD storage. Netflix, like Sonian, a beacon of cloud success, has already shared a great benchmark study showing how this new instance will improve performance and lower costs.

Wow… more performance… and lower costs. This trend tracks back to a previous post I wrote about active and passive cloud cost savings. The introduction of this new instance type creates an “optimization opportunity.” If we cloud customers are willing to invest engineering resources to optimize our software around a new instance type, that is an example of “active savings.” We have to apply effort to realize a cost reduction. On the other hand, if AWS simply lowers the price of an existing instance type, that is an example of passive savings. Just happens automatically.

This is the cloud’s grand bargain. Cost efficiencies flow from infrastructure provider, through the application layer, to the end customer.

Posted on July 19th, 2012 in Amazon Web Services, Blog Series, Cloud Compute, Cloud Masters | No Comments »

The Cheap Cloud versus The Reliable Cloud

5 Lessons Learned from June 29 2012 AWS Outage

Discussing a difficult situation is never fun, and I have been wrestling with how to start this post. It’s about revealing unpleasant cloud truths. And not necessarily the truths you might be expecting to hear. I am not here to preach, but my message to you is important. For the past five years I have been working on a project that uses the cloud to it’s fullest potential, celebrating the victories and learning from the defeats.

I’m speaking to my fellow Amazon cloud citizens. My co-tenants, if you will, in the “Big House of Amazon.” We’re all living together in this man-created universe with its own version of “Newtonian Laws” and “Adam Smith” economics. 99.99% of the time all is well… until out of the blue it’s not, and chaos upends polite cloud society.

If you lost data or sustained painful hours of application downtime during Amazon’s June 29 US-East outage, then you can only wag your finger in blame while looking in the mirror.

I know, I know, the cloud is supposed to be cheap AND reliable. We’ve been telling ourselves that since 2007. But this latest outage is an important wake up call: we’re living in a false cloud reality.

Lesson 1: Follow the Cloud Rules

Up front, you were told the “rules of the cloud”:

  • Expect failure on every transaction
  • Backup or replicate your data to other intra-cloud locations
  • Buy an “insurance policy” for worst case scenarios

These rules fly against the popular notion that the cloud is “cheaper” than do-it-yourself hosting.

There is a silver lining to this dark cloud event. Everyone in the cloud will learn and improve so we don’t have to repeat this episode ever again.

Read more…

Posted on July 5th, 2012 in Amazon Web Services, Cloud Compute, Featured, FISMA Moderate | 3 Comments »

Reflecting on One Year of Cloud Cost Optimization

For the past year I held the unelected position of “Cloud Cost Czar.” I have written about the duties such a role entails in A Day in the Life of a Cloud Cost Czar. Recently I handed over the cost czar responsibility to a colleague who will carry on the daily routines and continue to improve our cloud cost management endeavors. In the handoff process, almost a year to the day of assuming the czar’s responsibilities,  I reflected on the previous twelve months and all the accomplishments the company made as a united team to “tame the cloud.”

I created a graph to visualize the dramatic change over one calendar year. To the right is an area graph that shows subscriber seats (in green) overlaid on subscriber costs (blue, orange and red; our principle costs are cloud compute and two types of cloud storage.)  As subscriber growth increased, costs went up, peaked, and then went down over the course of one year. The rise, peak, and subsequent decline all map to various cost cutting efforts initiated by Sonian engineering and support groups.

Throughout the year we got smarter on how to “purchase” compute time for less than retail, how to store more customer data while consuming less cloud storage, and how to process more customer data using fewer CPU hours. In the cloud, we re-affirmed with a high-five on each improvement, we were in control of our cost destiny. This is when the phrase “infrastructure as code” really means something.

Read more…

Posted on April 29th, 2012 in Amazon Web Services, Archiving, Blog Series, Cloud Compute, Cloud Masters, Featured, Sonian | No Comments »

Comparing 6 Cloud App Marketplaces

Enterprise application marketplaces are sprouting up like Spring-time Daffodils. The latest entrant is Amazon Web Services’ AWS Marketplace. Amazon the e-tailer is no stranger to broad e-commerce initiatives, having conquered books, home goods, electronics, digital media and most recently mobile. (Aside: All indications show Amazon’s new Android Marketplace is off to a great start after a somewhat lukewarm industry reception.)

Many of the newest cloud apps are launched in the AWS cloud. AWS has done a great job courting startups onto their cloud platform. With the AWS Marketplace, Amazon is helping its customers be more successful by giving visibility to both small and large companies who choose AWS for their cloud infrastructure. The AWS Marketplace will also further cement customers into the AWS cloud, since Marketplace participation requires an AWS account. You can’t sell a non-AWS hosted application in the AWS Marketplace. Recently AWS has been publicly advocating the idea of “take your data/app” with you, but in reality moving a complicated SaaS application with a large data footprint from one cloud to another is no small feat. The AWS Marketplace is one more glue point between ISV and AWS.

Apple’s extremely successful iOS App Store, along with iTunes, paved the way for the current marketplaces targeting enterprise customers. Salesforce.com is the poster child for business application marketplace success.

I found six “cloud” themed business oriented marketplaces which are described below in alphabetical order. Across these six marketplaces we do see a recurring theme: marketplaces are tied to their underlying technical platforms, and there are none that support a “cross platform” environment. Google, Box and Salesforce each allow the others to sell into their customer base, but all require a technical hook into an API or account.

  • AWS Marketplace
  • Box.com OneBox
  • Chrome Web Store
  • Google Apps Marketplace
  • Salesforce.com AppExchange
  • Zoho

1. AWS Marketplace

What is it?

The AWS Marketplace aggregates and curates thousands of applications powered by the AWS cloud.

Amazon has powerful e-commerce tools for subscription management, billing, shopping carts and customer ratings which AWS customers can use to get more third-party customer traction. The AWS Marketplace compliments DevPay and paid AMI’s with a robust e-tailer user experience.

Requirements?

  • AWS Account
  • Application must be running within the AWS cloud

Pricing Model?

Application publishers choose their own price. Currently ISV’s can sell a paid AMI in which case Amazon generates revenue from the EC2 costs when the application is running on an EC2 instance. For turnkey SaaS applications, the AWS Marketplace acts like a referral business, in which case the revenue to AWS is indirect.

Interesting note:

The AWS Marketplace and Amazon Partner Network both launched within days of each other. Amazon is accelerating innovation on multiple fronts for its juggernaut cloud platform. The startup community is pretty much a lock-in. Now the goal is to expand to the enterprise, and Partner Network and Marketplace are two steps toward that goal.

Read more…

Posted on April 21st, 2012 in Amazon Web Services, Blog Series, Cloud Compute, Cloud Masters | 1 Comment »

A Tale of Two Cloud Search Engines

Sonian Cloud Search and Amazon Cloud Search. Their names may sound the same, but they  couldn’t be further apart in terms of how much they cost to operate and their intended use cases.

Sonian is a veteran “Cloud Search” pioneer. In 2008 we launched the first version of search in the cloud, and today the service operates simultaneously across multiple public clouds using a single reference architecture.

Over the past 4 years we have perfected cloud search scaling and cost efficiencies. It’s been a steep learning curve, but well worth the effort. Today there are over seven billion documents indexed, with fifteen million new documents added each day. Daily index and retrieval volumes rise as new customers sign-up for the service.

The secret to Sonian Cloud Search mastery is a combination of open source and IP developed in-house and detailed metrics to show us information on cost and performance. Every few months improvements are deployed to lower costs and increase reliability. We’ve achieved per-document unit costs to fractions of a cent.

Read more…

Posted on April 18th, 2012 in Amazon Web Services, Archiving, Blog Series, Cloud Compute, Cloud Masters, Featured, Sonian | 1 Comment »

Kudos to Amazon On Their New Partner Network

I am a veteran of past enterprise IT vendor partner networks that reigned during their respective era’s of influence. Novell was the kingpin in the late 1980′s through the mid 1990′s. Microsoft, learning from Novell’s partner eco-system success and ultimately displacing Novell at the top, ruled up until a few years ago. Microsoft’s partner network is waning because a migration from on-premises to SaaS, powered by cloud computing, is quickly and dramatically changing the status quo relationships between customer, VAR partner and IT vendor.

The old three tiered hierarchy is undergoing change. The advent of the cloud will not render partner eco-systems irrelevant, but they will not be the same as in the past. Enter Amazon’s Partner Network. This is an example of a modest first step toward inserting cloud technologies into the existing IT VAR partner eco-system. The Amazon partner network has two tracks: Product and Consulting.

The Product track is designed to for ISV’s that develop a product on AWS with air cover support from Amazon to help make all AWS powered applications be successful. The Consulting track will support the more traditional VAR with tools to help enterprises utilize AWS cloud resources in what will probably be a decade long migration to the cloud. Hybrid operations will be commonplace, and that operating mode requires specialized consulting practices to ensure success.

 

 

 

 

 

 

Posted on April 18th, 2012 in Amazon Web Services, Cloud Compute | No Comments »

My “Automatic” 3 Layered Backup System

A recent computer theft, as well as last week’s World Backup Day, inspired me to get “my backup protection house” in order.

A positive difference between the Mac and Windows platforms (that is not often touted) is that full image Mac backups are much easier to manage than Windows PC backups. The Mac architecture enables easy disk cloning with “no Windows registry” annoyances. Couple this with the fact Mac full-system backup images are portable across all recent vintage Intel-based Macs, means a boot-able disk image backup of my Macbook Air can be attached to a Mac Mini and I will get hassle-free access to all my data and applications.

After much research and focusing on “frictionless automation,” here is my current strategy.

Goals:

  • Quick, complete “full system failure” recovery
  • Deleted or corrupted file recovery
  • “Set and forget” minimal maintenance

Requirements:

  • Protect 3 Macs (Macbook Air, iMac, Mac Mini) from hardware failure and data theft
  • Use a combination of local and cloud backup techniques
  • Automated processes after initial setup
  • Encryption for all data offsite

Solution:

  • Hourly Time Machine backups of all Macs to a 2TB Apple Time Capsule hidden in the house.
  • Hourly Arq 2 backups of work productivity files (documents, spreadsheets and presentations) to Amazon S3.
  • Daily image backups to a locally attached USB drive using SuperDuper! disk imaging software (the exclamation point in the product name is required… this utility is “that great!”). The daily image backups are rotated weekly to a fireproof safe.

Read more…

Posted on April 8th, 2012 in Amazon Web Services, Cloud Compute | 1 Comment »

FISMA Chronicles: FedRAMP, Inheritance and Key Controls

Part 2: FedRAMP, Inheritance and Key Controls

I am leading the FISMA project at Sonian, and we’re getting closer to achieving our first FISMA Moderate accreditation. For background on FISMA, read my first blog post on this subject.

With FISMA Moderate accreditation, Sonian will be able to manage non-defense government data. The accreditation is granted in the form of an “Authority to Operate (ATO)” bestowed upon a project by the government agency that will implement and utilize the product/service. A cyber security team within the government agency evaluates each project’s security documentation and gives the thumbs up or thumbs down. It’s an iterative process, that starts with extensive documentation, and audit, and government review and oversight. FISMA applies to both third party services purchased by the government, as well as internally developed and managed IT projects.

FedRAMP… Briefly

Currently, if a vendor wants to sell the same IT service to more than one government agency, FISMA requires an ATO from each agency, which adds time, complexity and cost to the procurement process. Historically, each agency has implemented and interpreted FISMA standards differently. The National Institute of Standards and Technology (NIST) devised the “FISMA Reference Architecture” for all agencies to follow, but in reality the local interpretation has varied. A “new and improved” accreditation standard is supposed to fix some of these issues. FedRAMP is a single umbrella guideline encompassing current FISMA rules, as well as updated rules that better align FISMA with technologies such as Software as a Service (SaaS) and cloud computing. When the legislation that created FISMA was drafted in 2002, SaaS and cloud computing were not on government technologist’s radar. FedRAMP is a modernization of FISMA, and also strives to streamline government IT purchasing, lower costs, and expedite project time lines. FedRAMP will benefit from FISMA’s first decade, so I am hopeful for an improved certification process when FedRAMP is officially ratified in about a year. There is already quite a bit known about FedRAMP and Sonian is working on a dual strategy to get FISMA Moderate for one agency, and then focus on FedRAMP for all other agencies.

Read more…

Posted on March 25th, 2012 in Amazon Web Services, Cloud Compute, Featured, FISMA Moderate | No Comments »

Amazon “Partnering” for Enterprise Cloud Success

GigaOM‘s Om Malik is reporting on a new business development partnership between Amazon Web Services and Eucalyptus Systems. Eucalyptus is the startup providing an open source implementation of the AWS cloud APIs. Eucalyptus allows customers to build their own “private” clouds with AWS API compatibility.

Smart move on Amazon’s part. Amazon’s amazing cloud success puts them in a unique position to maintain a commanding lead in public cloud infrastructure, and now with this partnership they have a great story to tell that bridges the gap between large-enterprise private clouds and their market-leading public cloud.

Enterprise cloud adoption success needs two crucial ingredients combined at the right inflection of market uptick. The first is applications and the second is a credible story how a “private cloud” can evolve to using public cloud resources.

Since Eucalyptus is the open source equivalent of the core AWS API’s, it seems natural and expected for Amazon to partner with the five year old Calif. firm. It’s also noteworthy that neither Amazon nor Eucalyptus want to characterize their partnership as a “hybrid cloud” play. Amazon probably feels that their ability to drive down costs will eventually attract every business to their cloud, over time. so partnering with the company that created the open source AWS API implementation is a great cloud on-ramp strategy.

As for applications, companies like Sonian are already proving that a public cloud is the best infrastructure to support an enterprise-focused SaaS service. Like Eucalyptus, Sonian is also a five year old cloud start-up.  The cloud makes it possible for Sonian to exist, while at the same time the cloud needs services like Sonian to solve a business pain point with an application built from the ground-up to use a public cloud.

It’s amazing what the “new” cloud-industry has accomplished in the past five years. Growth, innovation, and nothing less than a complete paradigm shift in Enterprise IT.

 

 

 

 

 

 

Posted on March 23rd, 2012 in Amazon Web Services, Cloud Compute, Featured | No Comments »

Only in the Cloud… Active and Passive Savings

File this one under “amazing but true.”

Today Amazon Web Services customers awoke to find their prices have been lowered for EC2, RDS and Elasticache.

All standard EC2 customers get a 10% discount. This is for doing absolutely nothing. Didn’t have to write more code, didn’t have to plea-with/strong-arm a sales rep, didn’t have to threaten to change vendors. This is the promise of the cloud. A system running on AWS yesterday now costs 10% less to run today.

For AWS customers who “meet Amazon in the middle” … i.e. “you do some work, Amazon does some work,” the savings are more dramatic. Reserved purchase reductions range from 37% to 41%. This is the other positive aspect of the cloud: As a cloud customer, if you are willing and capable to make changes in small increments, savings will add up. The cloud has a continuous history of price reductions in the form of new features and service derivatives. But in order to take advantage you have to write code. S3 Reduced Redundancy is a good example. It’s a flavor of S3 that has a lower price and lower durability. But it’s perfectly fine for storing objects that are less important. But you need to write code to take advantage of this storage class.

The cloud has the dual concepts of “passive savings” and “active savings.”

Posted on March 6th, 2012 in Amazon Web Services, Cloud Compute, Sonian | No Comments »

« Older Entries