Thanks great Rauf.
Regards
On Tuesday, 23 May 2023 at 13:18:55 BST, Rauf Khan <[email protected]>
wrote:
Hi ,
PartitionBy(<column>) is analogous to group by, all rows that will have the
same value in the specified column will form one window.The data will be
shuffled to form group.
RegardsRaouf
On Fri, May 12, 2023, 18:48 [email protected]
<[email protected]> wrote:
Hello,
In Spark windowing does call with Window().partitionBy(<column>) can cause
shuffle to take place?
If so what is the performance impact if any if the data result set is large.
Thanks
