Skip to content

Overview

Celeborn Client is separated into two roles:

  • LifecycleManager for control plane, responsible for managing all shuffle metadata for the application, resides in driver for Apache Spark and JobMaster for Apache Flink. See LifecycleManager
  • ShuffleClient for data plane, responsible for write/read data to/from Workers, resides in executors for Apache Spark and TaskManager for Apache Flink. See ShuffleClient