Mesos

By B Hindman,Andrew Konwinski, Matei Zaharia, Ali Ghodsi, Anthony Joseph, Randy Katz, Scott Shenker, Ion Stocia Submitted by: ------Subject: ------------Submission date:--------------Summary With the advent of cloud technologies people realized that no single framework is optimal for all applications and also running each framework on its dedicated cluster is (1) Expensive (2)It is difficult to share data with other clusters. The designers came to conclusion that they need to run multiple frameworks on same cluster. Mesos is a cluster sharing and isolation platform over which Hadoop, SCADS, MPI, and other cluster applications can run. Mesos framework gives applications a common interface for resource sharing. The main goals for Mesos are:(1)Scalability to 10,000s of nodes(2)Robustness, even with a master failure(3)Flexibility to support a wide range of current and future cluster applications. The resultant design is small microkernel core that pushes scheduling logic to applications. Objectives of the paper Datacenters are running an increasingly diverse set of applications, both for analytics and user facing services. They may be Programming models(like MapReduce, Dryad, Pregel)Storage systems(like HDFS, HBase, MySQL) and many diverse Web cluster applications. The main objective of the Mesos technology is efficiently share resources across these diverse cluster applications. Main approaches/methodologies Run multiple instances of the same framework Isolate production and experimental jobs. Run multiple versions of the framework concurrently. Build specialized frameworks targeting particular problem domains which give us better performance than general abstractions

Results/contributions of this paper The main contribution of Mesos is the fine grained sharing architecture. Fine-grained sharing does the following: (1) It allocates at the level of tasks within a job (2)It Improves utilization, latency and data locality. The Resource framework offers: Simple, scalable, application controlled scheduling mechanism. Major Strengths of the paper Scalability: Mesos is only responsible for inter-application scheduling (e.g. fair sharing).Mesos is typically simpler than intra-app scheduling .The result is that the Mesos has scaled to 50,000 nodes with 200 applications. Fault tolerance: Mesos master has only soft state list of currently running applications and tasks. The most important feature of Mesos is that its new master can recover this state when apps and slave nodes reregister Major Weakness of the paper The Mesos paper should have given us the sample of the data load versus the number of virtual machines they create versus the number of applications they are running. The Mesos paper should have given us a tabular from of result for different instances for C++ vs Java language. Avenues for future work The authors have implemented the Mesos architecture in C++ language .We know that different architecture can compile the same code into a different machine level language. When we use cloud computing we integrate various architectures into a same datacenter which can introduce heterogeneity in the datacenter. Suggested improvement for the authors/corrections on the paper The definition fair sharing for multiple resources is not well presented. What are tradeoff between performance and consistency? The authors should have included the topic with a graph.