Increase the number of parallel jobs in GitHub Actions at ASF organization level

Hi all, I am an Apache Spark PMC, and would like to know the future plan about GitHub Actions in ASF. Please also see the INFRA ticket I filed: https://issues.apache.org/jira/browse/INFRA-21646. I am aware of the limited GitHub Actions resources that are shared across all projects in ASF, and man

Re: Support User Defined Types in pandas_udf for Spark's own Python API

Yeah, we still should improve PySpark APIs together. I am currently stuck at some work and porting Koalas at this moment so couldn't have a chance to take a very close look (but drop some comments and skim). 2021년 4월 6일 (화) 오후 5:31, Darcy Shen 님이 작성: > was: [DISCUSS] Support pandas API layer on P

Re: Shutdown cleanup of disk-based resources that Spark creates

On Thu, 11 Mar 2021 at 19:58, Attila Zsolt Piros < piros.attila.zs...@gmail.com> wrote: > I agree with you to extend the documentation around this. Moreover I > support to have specific unit tests for this. > > > There is clearly some demand for Spark to automatically clean up > checkpoints on shu