When all else fails, start-ups at least get mandatory media attention on their financing stages. Sonian closed a C round and below are a few articles from the mainstream tech media coverage:
Techcrunch - Sonian Picks Up $13.6M For Cloud Archiving And Search, OpenView New Investor And Strategic Partner
Scott Kirsner’s Friday 5 - The Friday Five, with Rudina Seseri of Fairhaven Capital - Sonian discussed at minute 4:43 thru 7:15
GigaOM - Sonian gets new funding — but no more from Amazon
For the past year I held the unelected position of “Cloud Cost Czar.” I have written about the duties such a role entails in A Day in the Life of a Cloud Cost Czar. Recently I handed over the cost czar responsibility to a colleague who will carry on the daily routines and continue to improve our cloud cost management endeavors. In the handoff process, almost a year to the day of assuming the czar’s responsibilities, I reflected on the previous twelve months and all the accomplishments the company made as a united team to “tame the cloud.”
I created a graph to visualize the dramatic change over one calendar year. To the right is an area graph that shows subscriber seats (in green) overlaid on subscriber costs (blue, orange and red; our principle costs are cloud compute and two types of cloud storage.) As subscriber growth increased, costs went up, peaked, and then went down over the course of one year. The rise, peak, and subsequent decline all map to various cost cutting efforts initiated by Sonian engineering and support groups.
Throughout the year we got smarter on how to “purchase” compute time for less than retail, how to store more customer data while consuming less cloud storage, and how to process more customer data using fewer CPU hours. In the cloud, we re-affirmed with a high-five on each improvement, we were in control of our cost destiny. This is when the phrase “infrastructure as code” really means something.
Sonian Cloud Search and Amazon Cloud Search. Their names may sound the same, but they couldn’t be further apart in terms of how much they cost to operate and their intended use cases.
Sonian is a veteran “Cloud Search” pioneer. In 2008 we launched the first version of search in the cloud, and today the service operates simultaneously across multiple public clouds using a single reference architecture.
Over the past 4 years we have perfected cloud search scaling and cost efficiencies. It’s been a steep learning curve, but well worth the effort. Today there are over seven billion documents indexed, with fifteen million new documents added each day. Daily index and retrieval volumes rise as new customers sign-up for the service.
The secret to Sonian Cloud Search mastery is a combination of open source and IP developed in-house and detailed metrics to show us information on cost and performance. Every few months improvements are deployed to lower costs and increase reliability. We’ve achieved per-document unit costs to fractions of a cent.
File this one under “amazing but true.”
Today Amazon Web Services customers awoke to find their prices have been lowered for EC2, RDS and Elasticache.
All standard EC2 customers get a 10% discount. This is for doing absolutely nothing. Didn’t have to write more code, didn’t have to plea-with/strong-arm a sales rep, didn’t have to threaten to change vendors. This is the promise of the cloud. A system running on AWS yesterday now costs 10% less to run today.
For AWS customers who “meet Amazon in the middle” … i.e. “you do some work, Amazon does some work,” the savings are more dramatic. Reserved purchase reductions range from 37% to 41%. This is the other positive aspect of the cloud: As a cloud customer, if you are willing and capable to make changes in small increments, savings will add up. The cloud has a continuous history of price reductions in the form of new features and service derivatives. But in order to take advantage you have to write code. S3 Reduced Redundancy is a good example. It’s a flavor of S3 that has a lower price and lower durability. But it’s perfectly fine for storing objects that are less important. But you need to write code to take advantage of this storage class.
The cloud has the dual concepts of “passive savings” and “active savings.”
In December 2002 the US Congress passed the Federal Information Security Management Act (FISMA). FISMA requires each government agency to implement policies, procedures, and documentation for information security. This includes internal and external government-run systems, and systems provided by third-party providers.
A flourishing information security practices industry has developed in FISMA’s wake to help guide the government and vendors through the numerous, byzantine certification activities.
The FISMA mission statement is to:
Protect the Nation’s Critical Information Infrastructure
FISMA has three assessment levels and risk profiles.
The majority of internal applications require FISMA Moderate. The moderate certification process is the focus of this series.
The moderate risk profile means addressing over three hundred controls ranging from information handling, physical media management and threat assessments. The hundreds of controls are categorized into the following “control families”:
Many start-ups address the above in various levels of completeness, but may not necessarily have all the supporting documentation to prove compliance. For SaaS systems operating in a cloud environment, the challenge is to describe the control boundaries between the cloud provider and the application layer. For example FISMA requires a policy for physical media disposal. The app layer (i.e. the cloud customer) doesn’t have access to physical media in a cloud environment, so that control is the responsibility of the cloud provider and the app layer inherits the control. Conversely, the cloud infrastructure has no control over the app layer, and the FISMA requirement to support two-factor web-app authentication is the responsibility of the app layer, not the cloud provider.
FISMA wasn’t designed for a world with cloud computing. It’s heritage back to 2002 is a world with hardware-centric design principles and best practices. Sonian and others are pioneering FISMA Moderate certification in a cloud environment.
Topics I will cover in upcoming issues of the FISMA Chronicles:
Image Credit: fismacenter.com
This is a true story from the “Cloud Cost Czar Chronicles.”
Our S3 “penny per one thousand” API costs started to rise rapidly in the second half of the cloud infrastructure billing period. We have seen this behavior before, and knew this could be attributed to increased usage, a new defect, or a design flaw that rears its head at a scaling tipping point. My job as “cost czar” is to raise the alarm and work with the team to figure out what was going wrong. At the observed rate of increase, the excess charges would push the monthly bill beyond the budget. One thing we have learned in the cloud, is that costs can rise quickly, but take awhile to go down, since the deceleration effect can be out of proportion to the acceleration if trying to manage expense in a single billing period.
When we started using Amazon Web Services S3 (a PaaS object store) back in 2007, we were acutely aware of the three pricing vectors in effect; storage consumed, price for API calls to store and list data and price for API calls to read and delete data. We’ve been using S3 heavily for five years and we tried to model the “all-in” costs as accurately as possible. But “guestimating” costs beyond the raw storage was stretch. PaaS services have an intrinsic “social engineering” element. If you color outside the lines the financial penalty can be significant. But if you master the pricing game, the rewards are equally as significant. So five years ago we thought as long as we point in the right general direction, “we’ll figure it out as we go along.” Some assumptions proved a positive surprise. Raw storage costs went down. Some surprises not so pleasant; continually wrangling the API usage fees, especially the transactions that cost a penny per thousand, proved to be a constant challenge. But I still like my options with S3 compared to buying storage from a hardware vendor and having to incur the administrative overhead. With S3 we can lower our costs by smarter engineering. With storage hardware, the only way to lower costs is to wrangle a better deal from an EMC sales person. As one of the original “cloud pioneers,” Sonian is not alone in this effort, and it’s been a real eye-opener for software designers to have to think about how their code consumes cloud resources (and expense) at scale. Because whether a penny per thousand or penny per ten thousand, when processing hundreds of millions of transactions a month, any miscalculation suddenly brings a dark cloud raining over your project.
Five years ago I wrote a business plan that described an archiving SaaS project built on cloud computing. In 2007 that was an uphill battle to convince prospective investors “the cloud was the future.” And at that time there was really only one cloud, from the e-commerce giant Amazon. Amazon Web Services really started the modern cloud movement. No existing IT provider (IBM, HP, Microsoft, Dell, etc.) would have had the gusto to upset their current business model with a “disruptively priced” cloud option. For the past four years those IT giants fought the cloud momentum until they had a credible cloud themselves. But for a lean start-up getting funded five years ago, it wasn’t a stretch to assume other clouds would appear to take on Amazon.
The graphic above was my crude way to visualize how a cloud-powered digital archive, anticipating someday living on multiple clouds, could in essence become a “cloud of clouds.” A lot of positive breakthroughs would need to occur to be able to successfully operate a single reference architecture software stack across more than one cloud. There was no terminology to describe this desire. We weren’t using terms like “Big Data” or “DevOps” nor many of the acronyms that today are common lingo in our modern cloud-enabled world. The business plan depicted a system designed to manage lots of data, and being an enterprise document archive, the data itself was large in size and numerous in quantity. We probably started one of the worlds first cloud-big-data projects.
In the beginning the multi-cloud goal was a fantasy dream, a placeholder for a future that seemed possible, but the actual crawl, walk, run steps not precisely defined because we didn’t yet know “what we didn’t know.”
So why in 2008 were we thinking about “multi-cloud?” The answer is we wanted to avoid single vendor lock-in and maintain a modicum of control over our infrastructure costs. The notion of an evolving multi-cloud strategy meant the ability to seek lowest cost of goods from multiple cloud vendors. In the pre-cloud IT world, when services were built on actual hardware, pricing flexibility was derived by negotiating better deals with hardware vendors. The customers didn’t know or care that their SaaS app might be powered by HP sever one day or a Dell 1U box the next. Those decisions were up to the discretion of the SaaS provider to get the best infrastructure value by shopping vendors. But in a single cloud, when there is only one choice, there’s no ability to negotiate between multiple vendors, unless you have multi-cloud dexterity.
Multi-cloud capable means the necessary infrastructure and abstraction layer is available to run a single common reference architecture on different clouds at the same time, with one master operator console. Multi-cloud is almost like, but not exactly, the concept of running a common program across IBM, DEC, Control Data mainframes. The clouds today somewhat resemble massive time-sharing mainframes of the previous decades.
Our early start five years ago, and all the hard lessons learned since, allows us to easily assume a commanding position in multi-cloud deployments. Engineering teams just now starting their “cloud journeys” will learn from us pioneers, but there is an old saying; “until you’ve walked a mile in my shoes, don’t claim to know anything otherwise.”
This is an essay that was published to the Sonian cloud compute blog. Cross posting here for this audience.
In the past I have written about the secret to successful cloud deployments and how to architect for the cloud. Being successful requires a “designed-for-the-cloud” architecture, best operational practices and DevOps on steroids.
A couple weeks ago Amazon notified a majority of their customers about an upcoming event that us early-to-the-cloud pioneers hadn’t seen before; a forced reboot of the host operating system. On a massive scale. For Sonian, 72% of our currently running EC2 instances will need to be restarted before Amazon’s deadline. There is no reprieve. There is no deferment. Welcome to Infrastructure as a Service!
We had to scramble in order to assess the impact. All we learned from the email notice was that a portion of our EC2 instances would need to be restarted. Actually there were two types of restarts. An operating system reboot, which would preserve the non-persistent ephemeral storage, and a more invasive full instance restart (meaning the hardware under the hypervisor would power-cycle) which would not preserve the ephemeral storage.
One of the major mistakes cloud customers can make is to get complacent and treat the cloud like traditional co-located hosting. The cloud has different operating characteristics, what one could call the “cloud laws of physics,” and this forced restart is a good example of this principle in action. It’s also a wake up call to not get lazy. A large scale forced restart is like an earthquake drill. Practice makes perfect, and if this were an actual un-scheduled emergency, then we would be scrambling.
Despite the headache, this event has some positive spins. First it’s encouraging there is an “EC2 fleet upgrade.” This means newer underlying hardware. Perhaps faster NIC cards in the hosts. But for the companies like Sonian that started in the cloud circa 2007, some of our original instances that have been running for more than a year needed a “freshening.” This event reminds us there is a “hardware” center to every amorphous cloud. Amazon just does a great job to allow us to not have to think about that too often, except for times like these. A stale part of the cloud gets a refresh.
The second “benefit” is the forced fire drill. I know, there’s never a good time for the fire drill. But this type of event has similar qualities to an unexpected outage. There is some luxury to pre-planning, but the shake-out will be the same. Something will be discovered in your architecture or deployment practices that will get improved by this reboot activity. Clusters may be too hard-coded. Config settings may be to restrictive. Reboot scripts may not work as you think.
Sonian survives unscathed due to our maniacal focus on 100% automated deployments, 100% commitment to “infrastructure as code,” and an investment in cloud control tools that allowed us to triage the situation and develop an action plan relatively quickly. We also employ the best darn DevOps team the cloud has seen.
Cross-posting here a two part essay I wrote for the Sonian blog on how Sonian is benefiting from, and contributing to (by amplification,) the innovation cadence in cloud computing.
I’ve been working in enterprise software since the late 1980′s, and what I am witnessing as a participant in “the cloud” is the pace of cloud technology innovation over the past five years blows away the previous two decades.
There is a real noticeable trend here. We didn’t see this in SaaS powered by co-location hosting. What we are seeing with the cloud, and the ISV’s that adopted the cloud five years ago, is truly amazing. Sonian is entering a release cadence updating production systems with substantial new features every month.
Cloud Innovation – Part 1
Cloud Innovation – Part 2
Hello! – I’m Greg Arnette, a repeat-entrepreneur and currently the founder and CTO for Sonian, a universal data management web service powered by cloud computing. This is my “personal content hub” to talk about email, collaboration, big data management, cloud computing, start-ups and whatever else comes to mind. See you soon!
Sonian provides a revolutionary Cloud based Hosted Email Archiving and eDiscovery service.
notebook G™ is your online organizer. Ditch the sticky notes, scraps of paper, and spiral notebooks.
“Cloud Killed the (SaaS) Rock Star”
“Cloud Killed the (SaaS) Rock Star”…
… well, not literally, but definitely in a figurative sense.
The press release below is the all-points-bulletin heralding the cloud has “won.” Why do I say this? Because LiveOffice, a non-cloud SaaS start-up, couldn’t compete against the new generation of SaaS start-ups powered by true public cloud computing like Sonian.
LiveOffice was the rock star of SaaS archiving. Ten years in business and they deserve the credit as one of the pioneers to legitimize the SaaS market. When LiveOffice launched a decade ago, they had to operate their own data centers. (This is called “Co-located Powered SaaS.”) But during the past five years, the world changed underneath them. Usually, market dynamics cause this kind of disruption, but the SaaS archiving market size didn’t get smaller, rather it’s bigger than ever. What changed starting in 2007? The advent of the public cloud. Suddenly, any SaaS company running their own data center became vulnerable to competitors able to harness the cloud. This is the beginning of the cloud-powered SaaS era.
Seriously, I wish all the best to the LiveOffice team. Sonian and LiveOffice competed vigorously from 2008 to 2011. Symantec acquired a great team, and the fit between LiveOffice and Symantec makes a ton of sense, and it’s understandable why Symantec made the acquisition.
Although LiveOffice called themselves a “cloud archiving” company, that was stretching the truth. The cloud moniker is so overused at this point, the public is deceived into believing they are using a cloud service, when in fact, it’s really just re-packaging the same old SaaS with a new label.
Why did this Happen?
Operating a SaaS infrastructure on a pure cloud environment is vastly different compared to a co-located system; it’s the reason we’re going to see more of old-world SaaS companies change control or fade away. It will be exceedingly difficult to re-tool a co-located hosted SaaS business to use the cloud. Not impossible, but very difficult. The whole architecture would need to change. I say this having lived in both worlds — with the cloud battle-scars to prove it.
Read more…
Posted on March 4th, 2012 in Archiving, Cloud Compute, Commentary FWIW, email, Sonian | No Comments »