deciding Spark tasks & optimization resource

[rajat kumar]

Hello Members,

I have a query for spark stages:-

why every stage has a different number of tasks/partitions in spark. Or how
is it determined?

Moreover, where can i see the improvements done in spark3+


Thanks in advance
Rajat