As compute clusters scale, making efficient use of cluster resources becomes very important. Peloton is a Unified Resource Scheduler to co-schedule mixed types of workloads such as batch, stateless and stateful jobs in a single cluster for better resource utilization. Peloton is designed for web-scale companies like Uber with millions of containers and tens of thousands of nodes. Peloton features advanced resource management capabilities such as elastic resource sharing, hierarchical max-min fairness, resource overcommit, workload preemption, etc. Peloton is also Cloud agnostic and can be run in on-premise datacenters or in the Cloud.
Co-scheduling Mixed Workloads: Support mixed workloads such as batch, stateless and stateful jobs in a single cluster.
See the Tutorial for step-by-step instructions to start a local minicluster and submit a HelloWorld job to Peloton.
To achieve high-availability and scalability, Peloton uses an active-active architecture with four separate daemon types: job manager, resource manager, placement engine, and host manager. The interactions among those daemons are designed so that the dependencies are minimized and only occur in one direction. All four daemons depend on Zookeeper for service discovery and leader election.
Figure , below, shows the high-level architecture of Peloton built on top of Mesos, Zookeeper, and Cassandra:
Peloton consists of the following components:
See the User Guide for more detailed information on how to use Peloton.
Peloton CLI is a command line interface for interacting with Peloton clusters, such as creating jobs, check job status etc. For detailed Peloton CLI commands and arguments, see CLI Reference.
Peloton defines the APIs using Protobuf as the IDL and the clients can access Peloton API via gRPC. Peloton supports three client bindings by default including Python, Golang and Java. Any other language bindings supported by gRPC should work as well.
See the Developer Guide on how to build Peloton from source code.
© 2022 Anant Corporation, All Rights Reserved. All logos, trademarks and registered trademarks are the property of their respective owners.