This is a true story from the “Cloud Cost Czar Chronicles.”
Our S3 “penny per one thousand” API costs started to rise rapidly in the second half of the cloud infrastructure billing period. We have seen this behavior before, and knew this could be attributed to increased usage, a new defect, or a design flaw that rears its head at a scaling tipping point. My job as “cost czar” is to raise the alarm and work with the team to figure out what was going wrong. At the observed rate of increase, the excess charges would push the monthly bill beyond the budget. One thing we have learned in the cloud, is that costs can rise quickly, but take awhile to go down, since the deceleration effect can be out of proportion to the acceleration if trying to manage expense in a single billing period.
When we started using Amazon Web Services S3 (a PaaS object store) back in 2007, we were acutely aware of the three pricing vectors in effect; storage consumed, price for API calls to store and list data and price for API calls to read and delete data. We’ve been using S3 heavily for five years and we tried to model the “all-in” costs as accurately as possible. But “guestimating” costs beyond the raw storage was stretch. PaaS services have an intrinsic “social engineering” element. If you color outside the lines the financial penalty can be significant. But if you master the pricing game, the rewards are equally as significant. So five years ago we thought as long as we point in the right general direction, “we’ll figure it out as we go along.” Some assumptions proved a positive surprise. Raw storage costs went down. Some surprises not so pleasant; continually wrangling the API usage fees, especially the transactions that cost a penny per thousand, proved to be a constant challenge. But I still like my options with S3 compared to buying storage from a hardware vendor and having to incur the administrative overhead. With S3 we can lower our costs by smarter engineering. With storage hardware, the only way to lower costs is to wrangle a better deal from an EMC sales person. As one of the original “cloud pioneers,” Sonian is not alone in this effort, and it’s been a real eye-opener for software designers to have to think about how their code consumes cloud resources (and expense) at scale. Because whether a penny per thousand or penny per ten thousand, when processing hundreds of millions of transactions a month, any miscalculation suddenly brings a dark cloud raining over your project.
“Playing the Cloud” is the positive alternative to “Gaming the Cloud”
Both statements have the same good intention, except “end war” puts the emphasis on the relatively negative word “war,” while “make peace” puts the emphasis on a very positive thought. Words matter, just ask any political pollster how they craft their surveys & slogans, and you’ll learn the persuasive word science of Frank Luntz . Written words are the results of thought, thoughts are the results of “energy,” and putting energy out to the Universe creates results.
Internally at Sonian I coined the phrase “gaming the cloud” as a rally cry to describe how we manage the cloud to our benefit. We’re not doing anything unsavory or nefarious, just architecting software and creating processes that take advantage of all the positive attributes of cloud computing. We’re innovating exactly how cloud infrastructure vendors hoped would happen when they welcomed ISV’s to innovate on their platforms. It’s the best of all times to be creating cloud-powered software as a service.
So when I say “gaming the cloud,” what I really mean is seeking our best economic advantage. When we make the right software design decisions, we’re not only getting the best cost of goods, we’re also getting the best reliability. But I realize the phrase “gaming the cloud” carries an unintended negative pall. The mental image is not in the true spirit of our mission.
So what’s the positive alternative to “Gaming the Cloud?” …. Playing the Cloud.
“Playing the Cloud” to create superior economic advantage!”
Deconstructing the new phrase, the pivotal word is “playing” as the antidote to “gaming.” Gaming has a negative connotation, while “playing” is neutral at worst, and can be used in a variety of ways as a play on words. In an economic sense there is “playing the market,” or “playing the ponies,” and in a mission sense there is “playing to win.”
The cloud is a system of pricing rules. Prior to the cloud, system architects thought in terms of “servers” as their building blocks. In the cloud, the building blocks are compute units and API calls. Servers have costs, fairly easily understood since that was the reference standard for the past twenty years. Architects could determine overall system costs by knowing how many servers they need. In the cloud, compute units and API requests also have costs, but priced very differently than a piece of hardware. Cloud architects have a more difficult time figuring out total “infrastructure expense.”
The future looks great since the cloud is all about “infrastructure as code,” cloud-powered systems can be made self-aware of their own internal operating costs. That’s a dramatic paradigm shift from the old co-location days. And we’re witnessing this shift in real-time as cloud adoption rises.
So when you hear the phrase “gaming the cloud” don’t imagine the dark-side… think about the positive alternative “Playing the Cloud” for superior economic advantage.
Posted on December 18th, 2011 in Blog Series, Commentary FWIW, Gaming the Cloud | No Comments »