Cascading big data framework gets Apache Tez support
Cascading is a stand-alone open source Java application framework designed as an alternative API to MapReduce. Cascading gives Java developers the capability to build big data applications on Hadoop using their existing skillset. Concurrent founder and CTO Chris Wensel says he created Cascading in anger after having used MapReduce once. He vowed never to use it again.
Cascading now averages more than 275,000 user downloads a month.
Spinning new compute fabrics
Cascading 3.0, the latest release, is a milestone that adds native support for new compute fabrics, in addition to existing portability across programming languages (Java, SQL, Scala) and Hadoop distributions (Cloudera, Hortonworks, MapR). Out of the gate, Cascading 3.0 adds native support for the Apache Tez compute fabric, but Wensel says others will be quickly added. Wensel says he's currently working on support for Apache Spark.
[ Related: Cascading allows apps to execute on All big data fabrics ]
Wensel notes that while Concurrent promised Spark support more than a year ago, Spark proved extremely young and constant changes to the API made working with it difficult. And while there have been many requests for Spark support, he believes that people aren't so much fixated on Spark as they are interested in "faster than MapReduce."
That's where Apache Tez comes in. Wensel chose to concentrate efforts on Tez while Spark matured. Even so, Tez is also a relatively young project.
"Tez removes some of the overhead of MapReduce, but it comes with a cost as well," Wensel says. "We thought it important that Cascading 3.0 simultaneously support MapReduce and Apache Tez. Tez is still early compared with the amount of time it's taken to make MapReduce stable."
"You can write your business logic once on Cascading, have it up and running on MapReduce, and then switch it over to Apache Tez to see whether it runs more performantly and reliably," he adds. "Installing Tez is really just a matter of dropping some .jar files on HDFS."
What's new in Cascading 3.0
New features and benefits in Cascading 3.0 include the following: