Apache Celeborn™ (Incubating) 0.2.0 Release Notes
Highlight
-
Support load-aware slots allocation strategy.
-
Support columnar shuffle.
-
Enhanced flow control.
-
Enhanced HA stability.
-
Enhanced disk management.
-
Support graceful and fast upgrade.
-
Support quota management.
-
Support zstd compression.
-
Support spark 2.4 to 3.3.
-
Major improvements of Celeborn on Kubernetes, including documents and Helm Chart.
-
Code cleanup and refactor.
-
Shade netty completely.
-
Enhanced build system.
-
Refactor configurations.
-
Support JDK11.
-
Remove log4j1.
Performance
-
Batch and parallelize RPC.
-
Add bitmap to filter unnecessary partition read.
-
Cache RPC to reduce Spark driver memory pressure.
Others
- Bugfixes and improvements.
Credits
Thanks to the following contributors who helped to review and commit to Apache Celeborn(Incubating) 0.2.0-incubating version, and the order is based on the commit time:
| Contributors | |||||
|---|---|---|---|---|---|
| zwangsheng | AngersZhuuuu | FMX | waitinfuture | dxheming | haiming |
| pan3793 | lichaojacobs | liugs0213 | nafiyAix | fanyilun | 942011334 |
| kerwin-zk | xunxunmimi5577 | mcdull-zhang | wForget | RexXiong | leesf |
| zouxxyy | Gabriel39 | zhongqiangczq | szyWilliam | boneanxs | jxysoft |
| cfmcgrady | zy-jordan | kaijchen | Rex(Hui) An | jiaoqingbo |