By using the Cloud Foundry open-source framework along with other open-source software, OpenCredo eliminated "heavy lifting" such as configuring virtual machines and adjusting the size of storage volumes, says CEO Russell Miles. The framework allowed developers to write code locally, share it with the client, and automate the integration, testing, and deployment of application components.
Among other advantages, Cloud Foundry makes it easier to scale an application by adding more instances without downtime, Miles says. Because of the way it works with other open-source software, new features can be added in minutes rather than hours.
Even with all those benefits, open-source cloud frameworks like Cloud Foundry are a work in progress. Many manage only physical servers or stand-alone applications, leading customers who need more sophisticated capabilities to create their own frameworks. However, they offer compelling value because they mask the complexity of cloud computing setups, and the open-source model is an attractive way to do that.
Understanding the Basics
The term "framework" is used to loosely describe collections of anything from development tools to middleware to database services that ease the creation, deployment and management of cloud applications. Those that work at the level of servers, storage and networks are infrastructure-as-a-service (IaaS) frameworks. Those that operate at the higher level of applications are platform-as-a-service (PaaS) frameworks.
Among the most popular IaaS frameworks are OpenStack, Eucalyptus, and the Ubuntu Cloud infrastructure. Citrix recently announced it was making its formerly proprietary CloudStack IaaS platform part of the open-source Apache project. Gartner analyst Lydia Leong wrote in her blog that this is "big news" because CloudStack is much more stable and production-ready than the "unstable" and "buggy" OpenStack.
Popular PaaS frameworks include Heroku, Cloud Foundry (backed by VMware), and Red Hat's OpenShift, which is built on a foundation of Red Hat Enterprise Linux with support for a variety of languages and middleware through the use of "cartridges."
Customers often use multiple frameworks and associated tools. One example is the use of OpenStack to provision virtual machines, and Opscode Chef to create "recipes" describing how servers should be configured, says Opscode co-founder Jesse Robbins. The further up the "stack" a platform operates, the less work the customer must do, but they also have less control over the infrastructure components, says Matt Conway, CTO at online backup vendor Backupify.
Beyond easing cloud creation, most frameworks claim to make it easier to move cloud deployments among public and private clouds to get the lowest cost and best service. For example, Eucalyptus is meant to provide an Amazon EC2-compatible API that runs on top of Ubuntu Linux (the version of Linux underpinning the Ubuntu Cloud), "so apps authored for EC2 should be transplantable to one's own data center running Eucalyptus," says Conway. "Deltacloud was an initiative by Red Hat to create a 'cloud API' to abstract your application away from vendors like Amazon, and it would proxy your requests to the actual Amazon API."
For online storage vendor CX, OpenStack provides the flexibility to use other cloud vendors besides Amazon "if [Amazon's] services become too expensive or otherwise unsuitable," says CX CTO Jan Vandenbos.
Anthony Roby, a senior executive in Accenture's advanced systems and technology group, says the word "framework" is often misused, and offerings such as Eucalyptus or OpenStack are "not frameworks at all," but "products you can extend or use to build your own infrastructure cloud." However, most observers define frameworks as software building blocks used to create cloud-based services for users.
Three Steps
Choosing an Open-Source Framework
1. Evaluate which components of a "framework" you need, as well as the level (i.e., infrastructure, platform or both) at which you need to reduce your workload.
2. Evaluate the level of support and professional services you'll need.
3. Check support for the specific cloud providers you might want to use, and whether your platform supports the added services (such as replication) your applications may require.
- Robert L. Scheier
The Role of Open Source
Open-source projects range from "pure" open-source development initiatives directed by nonprofit foundations that aren't associated with any commercial vendors, to those getting financial, marketing and development help from leading companies.
Canonical, which provides support for open-source efforts and plays a leading role in Ubuntu, has seen interest in open source "from the Fortune 50 to a ton of SMBs and startup companies," says Kyle McDonald, head of cloud at Canonical. Most of the company's OpenStack business has come from Fortune 1,000 companies seeking to reduce software costs, he says.
Over the past five years, "there's been a sea change towards open source being viewed as [a] safer bet" than proprietary software, says Chris Haddad, vice president of technology evangelism at PaaS framework provider WSO2. With the rising quality of open-source software, and the backing of major vendors, "large commercial organizations do not see it as a threat," he says. In fact, because of economic uncertainties, "to bet your farm on one company is not seen as a good decision these days," he adds.
Unlike developers working to meet the goals of a corporation subject to the ups and downs of the economy, open-source contributors "are writing software because that is what they love to do," says Conway.
While most early users of open-source products, such as Chef, were cloud providers that sold services to others customers, Robbins says he is "seeing a pretty quick shift to pretty rapid adoption in the enterprise" among banks, large media companies and other organizations that are building their own private clouds.
Most users, however, are not yet moving critical applications to the cloud, because they don't have the tools necessary to provide proper IT oversight and security, says Bryan Che, senior director of product management and marketing at Red Hat's cloud business unit. He says Red Hat's OpenShift will help meet these needs, in part by leveraging the security mechanisms already within Red Hat Enterprise Linux.
State Street overcomes security concerns by never acquiring open-source software directly from the Web, but only through trusted partners from which "we can get a support structure as well as the software," says chief architect Kevin Sullivan. Moreover, he says, the company also carefully checks contracts to ensure compliance with the terms of the license, and it scans all open-source software for malicious code.
WSO2 Stratos is already addressing such needs with products to support not only application development and deployment, but also integration, rules, business process management, governance, complex event processing and identity management, says Haddad.
Case Studies
State Street and Backupify Write Their Own Cloud Frameworks
When none of the cloud frameworks on the market matched their needs, financial services giant State Street and online backup provider Backupify each wrote their own.
State Street began looking at frameworks in 2009 as a way to more efficiently make use of its global data centers, but it found that most of them were focused only on provisioning and control of virtualized servers "in sort of an abstract way," says chief architect Kevin Sullivan. They couldn't simplify more complex tasks such as the ongoing need to update its 800 applications, provide the level of authentication needed to protect customer data, and discover and reuse services ranging from security to data warehousing that State Street had already developed and wanted to move to its private cloud.
"We wanted to get much higher in the stack from provisioning VMs," Sullivan says, with a platform that "would sit between the application and the infrastructure," making it easier to move applications and services among data centers as business needs changed. In addition, "because we have 3,000 application developers around the world and 800 applications, we're continually introducing new code into the environment," he says. "None of the cloud frameworks dealt with any of that. I didn't think there was anything out there mature enough [and] ready for the kinds of volumes we were going to need."
So, Sullivan says, the financial services firm created "our own internal service registry, where we house all of our metadata describing all the different categories of data in our service framework." Reusing those services was key to developing new applications more quickly and increasing profitability, he explains.
He says he still has a "bias" for open source over proprietary frameworks, not only because of the lower cost of the software, but also because of the ease with which it integrates new capabilities. Sullivan says he is continuing to evaluate open-source frameworks and to use components of them where possible.
Backupify CTO Matt Conway wanted tighter integration between a framework and his infrastructure than was provided by open-source software. So, working "on and off" for five years, Conway wrote his own open-source "rubber" framework for managing multi-instance deployments of Ruby on Rails applications to Amazon's Elastic Compute Cloud (EC2). That was the only way, he says, to manage the number of applications and the amount of data he was storing on EC2, and to provision and deprovision services quickly enough.
- Robert L. Scheier
Questionable Benefits
Some observers question whether open-source frameworks really deliver the benefits they're said to offer -- such as portability among clouds providers. "Eucalyptus replicates some of the Amazon APIs, but if you're using something on Amazon [that] Eucalyptus doesn't support, you're out of luck," says Roby. "Similarly, if you're trying to run Java apps and using the Spring [application development] framework, you've got a fair amount of support." But as soon as a customer begins using features, such as data storage, that can't be accessed via Spring, those features may not run correctly with a different provider. Without the ability to move underlying services as well as the application code, he says, "you don't have any portability."
With open source, users (or a group of users) theoretically could take the source code and tweak it to meet their own needs if a vendor can't or won't. However, few users would want to do that, says Roby. "If you're a big telco, maybe you are interested in being able to change the code... but most organizations wouldn't do that. The last thing they want is to have their own specific variant of the product" that they would have to support, while losing the ability to take advantage of upgrades from others in the community, he says.
Creating a unique open-source "fork" is usually not something you want to do "unless you absolutely have to," agrees Conway, noting that the fork could stagnate without contributions from others.
Much buzz surrounds open source, but proprietary frameworks such as Microsoft Azure or Salesforce.com's Force.com can be better choices "if you have specific needs and that platform already has built-in [elements] to make the job easier," says Shriram Nataraj, senior director in the cloud technology practice at Persistent Systems, a global software development firm. "If you're already a Salesforce customer and want to migrate part of your workload onto a different platform, Force.com can be a very good option for you. If you're already an Office 365 customer and have workloads on [Microsoft's .Net framework]... it makes sense to go towards Microsoft Azure."
Good fits for open-source frameworks tend to include experimental cloud applications built by developers who are comfortable with newer, open-source tools. Other likely candidates include applications deployed by organizations such as universities or research labs, which have the technical skills to learn and work with these new technologies, and/or the need for specialized capabilities such as massive databases or advanced analytics, says Roby.
Typical apps deployed using open-source frameworks include Web and social applications, as well as mobile or customer-facing websites, says Jerry Chen, vice president of cloud and application services at Cloud Foundry. Such frameworks are also useful when organizations need to deploy applications quickly and scale them up and down as needed.
Legacy applications requiring hardware or software that may not be supported on the Web tend to be less attractive candidates. "While it is very possible to migrate many data center applications from local servers onto [virtual] cloud-based ones, the ROI is not always clear," says Bill Weinberg, senior director of Olliance Group at software and services provider Black Duck Software. "The downside can lie in potential security issues, divergent response to loading, throughput bottlenecks and availability."
OpenStack and Cloudscale are better choices for complex applications than Eucalyptus, says Nataraj, because they do a better job of hiding the complexity of networking. For an application that, for example, requires a user "to connect from a different IP range," a customer would "have to write custom code to make that happen with Eucalyptus," he says. With OpenStack, the "switches" required to make those new network connections are already present.
The number and quality of developers involved in an open-source project can also be a good indication of the project's quality, many observers say. If developers from several companies are involved, vendor lock-in is less likely to be a problem, says Nataraj.
Roby, however, suggests focusing on a commercial vendor's level of commitment, rather than that of the community. "It's largely a myth that there's a lot of new code being developed by a large group of people," he says. "Any of these successful products are developed by a small group of people," with the community at large "providing feedback and maybe doing testing or providing documentation."
Miles also warns of "token" open-source efforts by partnerships among major vendors. "If both those companies don't really rely on the product for revenue, at any point in time either or both will just walk away, and the product will die," he warns.
The unconventional licensing terms that some open-source developers impose on their software, such as one requiring that "the Software shall be used for Good, not Evil," raise eyebrows in corporate legal departments. Posing a more serious problem are licenses that require a company to share any enhancements with other members of the community -- which creates the possibility that the company may have to reveal "best practices" to competitors.
Most experts interviewed say mainstream licenses such as Apache's don't impose such troublesome requirements. In any case, says Conway, his staff's processes and skills are just as important as any code he shares with others. And, he points out, open source also lets him use improvements made by others.
Open-source cloud frameworks have the potential to make it far easier for organizations to meet changing business needs by quickly deploying Web applications across public and private clouds. But to get those benefits, IT architects must sift through the various meanings that different vendors have for their "frameworks" and whether each framework can deliver the level of ease of use they need to meet their specific requirements.
Scheier is a veteran technology writer. He can be reached at bob@scheierassociates.com.
Mixed Marriages
Some Commercial Vendors Dabble in Open Source
Open-source software is born when programmers write great software for the love of it, sharing their greatest enhancements freely because they know that others will do the same.
Major software vendors such as VMware and hosting providers such as Rackspace are contributing developer and marketing time to open-source frameworks to drive sales of their other products and services, particularly the 24/7 enterprise-grade support that customers can't always get from the open-source community.
Some commercial vendors offer not only service and support, but also their own open-source frameworks. For example, Dell, a member of the OpenStack community since its creation, has unveiled Crowbar, an IaaS framework that was developed using Chef and includes a reference architecture based on its own servers, OpenStack open-source software, and services from Dell and Rackspace.
The main authors of the OpenNebula IaaS framework founded C12G labs to provide professional services to OpenNebula customers.
Network virtualization software vendor Nicera is leading the development of the Quantum project within OpenStack, which will provide network connectivity for devices managed by other OpenStack services. While its network virtualization layer isn't open source, it does support the open-source Open vSwitch virtual switch. It hopes to use OpenStack to drive sales of its software, much as server vendors rode the adoption of Linux to drive hardware sales, says Alan Cohen, vice president of marketing.
- Robert L. Scheier
Read more about cloud computing in Computerworld's Cloud Computing Topic Center.