Hi all,
Do we have any idea on this.
Thanks
On Tue, 23 Jul, 2024, 12:54 pm Sudharshan V,
wrote:
> We removed the explicit broadcast for that particular table and it took
> longer time since the join type changed from BHJ to SMJ.
>
> I wanted to understand how I can find what wen
table hoping I could see the table size in the
storage tab of spark UI of EMR.
But I see no data there .
Thanks
On Tue, 23 Jul, 2024, 12:48 pm Sudharshan V,
wrote:
> Hi all, apologies for the delayed response.
>
> We are using spark version 3.4.1 in jar and EMR 6.11 runtime.
>
> W
mentioned was taken from S3.
Thanks,
Sudharshan
On Wed, 17 Jul, 2024, 1:53 am Meena Rajani, wrote:
> Can you try disabling broadcast join and see what happens?
>
> On Mon, Jul 8, 2024 at 12:03 PM Sudharshan V
> wrote:
>
>> Hi all,
>>
>> Been facing a weird issue
Hi all,
Been facing a weird issue lately.
In our production code base , we have an explicit broadcast for a small
table.
It is just a look up table that is around 1gb in size in s3 and just had
few million records and 5 columns.
The ETL was running fine , but with no change from the codebase nor
