At first it seemed the relationship would last forever. A rapidly growing cloud storage startup relies on Amazon Web Services’ cloud for most of its infrastructure needs, allowing Dropbox’s engineers to focus on its product and not investing in costly infrastructure to run it.
But DropBox has decided it would rather be single. The company announced in a blog post this week that it would build up its own infrastructure stack and move mostly off of AWS.
Could this be a sign that as companies grow their cloud that it could be more efficient to build their own cloud If so, what is that cut-off point where its more efficient to not use the public cloud
+MORE AT NETWORK WORLD: Dreamhost replaces VMware SDN with open source for big savings +
There were two factors that made Akhil Gupta, vice president of Infrastructure for Dropbox, realize that the company should get out of the cloud. The first is size and growth. Dropbox has 500 million users and is storing 500 petabytes of data. “The scale that we’re operating on is one that very few other companies will get to,” Gupta says.
Secondly, Gupta wanted to have end-to-end control of the infrastructure so that he could control the performance, reliability and overall user experience. “By optimizing the stack and customizing the infrastructure to our use case, we were able to provide a key differentiator in the market and a key value to our users,” Gupta says.
Not every company has the scale Dropbox operates at. And most companies would not see a huge benefit from customizing infrastructure to tailor it to their specific needs, Gupta says. Dropbox’s journey took two and a half years and required investments in personnel to figure out how infrastructure should be customized and other workers to manage their data centers.
“Customization is a lot of work upfront,” Gupta said. The return on investment is over a long period of time, so you need to have the scale to invest in the teams and develop relationships with vendors. Public clouds, Gupta admits, are very good at offering infrastructure services that are good enough for many workloads. Dropbox will still be using AWS for some services, but its main storage platform will be run internally.
Forrester Research Vice President Richard Fichera says unfortunately there is no simple rule for when it is more efficient to run something in your own data center compared to using the public loud. But generally the cloud is a very efficient place to run most workloads. “Unless you’re in the upper few percentiles of size, it probably doesn’t make sense,” Fichera says of going-it-alone. Even if you do have the scale that would make economic sense to run it in-house, there are other advantages that the cloud brings. Cloud vendors provide you storage as a service – there is no infrastructure hardware to manage.
That doesn’t mean the cloud doesn’t have disadvantages. Public cloud providers have what Fichera calls “tolls” on using and moving data. There are not usually fees for uploading data into the cloud, but there are networking bandwidth costs for getting it out.
Other companies have dropped the cloud with mixed degrees of success. Zynga is perhaps the most notable example. The mobile app company built its games atop AWS, which allowed them to scale as they went viral. Then, they decided to build their Z-Cloud, which provided the base-level of compute resources the company needed. It was still going to use AWS for the “peak” capacity beyond what Z-cloud could handle. Eventually as Zynga fell out of favor in the app world, the company went back in with AWS.
Dropbox is confident that it will be around for the long haul and therefore these investments are worth it. Could Dropbox be ushering in a new wave of mega-cloud customers who are realizing they can more efficiently run infrastructure themselves Or will Dropbox be a Zynga case study all over again