I have reported this issue to the Parquet community:
https://github.com/apache/parquet-java/issues/3193
On Tue, Apr 15, 2025 at 9:47 AM Wenchen Fan wrote:
> Hi Yuming,
>
> 1.51.1 is the latest release of Apache Parquet for the 1.x line. Is it a
> known issue the Parquet community is working on,
Hi Yuming,
1.51.1 is the latest release of Apache Parquet for the 1.x line. Is it a
known issue the Parquet community is working on, or are you still
investigating it? If the issue is confirmed by the Parquet community, we
can probably roll back to the previous Parquet version for Spark 4.0.
Than
Yuming, you replied to a wrong thread mistakenly I suspect :-).
On Tue, 15 Apr 2025 at 08:13, Yuming Wang wrote:
> This release uses Parquet 1.15.1. It seems Parquet 1.15.1 may cause
> deadlocks.
>
>
> Found one Java-level deadlock:
>
> =
>
> "Executor 566 task launch
This release uses Parquet 1.15.1. It seems Parquet 1.15.1 may cause
deadlocks.
Found one Java-level deadlock:
=
"Executor 566 task launch worker for task 202024534, task 19644.1 in stage
13967543.0 of app application_1736396393732_100191":
waiting to lock monitor
Thanks for noticing, Jungtaek. No worries, Jacky.
Amended total: 30 +1s (14 binding +1s) and no -1s.
On Sun, Apr 13, 2025 at 4:37 AM Jacky Lee wrote:
> I'm just interested in this, but I'm not a PMC member of Apache Spark,
> sorry for mistake.
>
> Jungtaek Lim 于2025年4月13日周日 15:05写道：
> >
> > C
