Although MapReduce has found wide spread usage within the startups and SMBs community, its enterprise adoption is just beginning. We are seeing increasingly more enterprises evaluating the technology and developing PoC (Proof of Concept) to see where the technology may fit in the enterprise landscape. Also, there are several pioneering clients who have already implemented Hadoop in production. For example, I am informed recently that one of our government clients has already deployed 12 instances of Hadoop clusters.
For enterprises looking to deploy the technology, they now have one more choice to choose from. Accenture recently has partnered up with Appistry -- an internal cloud platform provider -- to release the next version of Cloud MapReduce (CMR). Building on a different architecture, the CMR implementation achieves many advantages over other implementations, such as Hadoop, including better scalability and higher performance. The Appistry integration allows an enterprise to run CMR inside its firewall instead of or in addition to running in the Amazon Cloud environment. This is officially announced by Appistry, which is also covered by GigaOM and New York Times. You can read about technical details on Cloud MapReduce on Appistry.
For an enterprise choosing which platform to deploy, they have several dimensions to consider. In addition to scalability and performance, there are several key reasons to consider the Appistry's version of CMR.
- No single point of failure. If you cannot tolerate downtime and potential data loss, then you should look into a fully distributed architecture as used by CMR instead of a master/slave architecture as used by Hadoop.
- Streaming support and incremental batch. If you constantly have new data coming in and need to continuously run your batch analysis to including the new data, then think about deploying a framework that supports streaming. The latest CMR implements streaming support, conceptually similar to what is provided by HOP, but it is implemented on a commercial grade.
- Support. If you are looking for a product company (instead of a consulting company) to support your deployment, Appistry is there for you.
Obviously, there are lots of reasons to choose Hadoop as well. For one, there is a huge community around the Hadoop platform. It is our intention to continue to develop CMR to make it interface compatible with Hadoop, so that CMR can be an integral part of the community. Stay tuned for the next version.