Re: [Issue] Spark SQL - broadcast failure

Hi all, Do we have any idea on this. Thanks On Tue, 23 Jul, 2024, 12:54 pm Sudharshan V, wrote: > We removed the explicit broadcast for that particular table and it took > longer time since the join type changed from BHJ to SMJ. > > I wanted to understand how I can find what went wrong with the

Re: [Issue] Spark SQL - broadcast failure

It will help if you mention the Spark version and the piece of problematic code HTH Mich Talebzadeh, Technologist | Architect | Data Engineer | Generative AI | FinCrime PhD Imperial College London

Re: [Issue] Spark SQL - broadcast failure

On Mon, 8 Jul, 2024, 7:53 pm Sudharshan V, wrote: > Hi all, > > Been facing a weird issue lately. > In our production code base , we have an explicit broadcast for a small > table. > It is just a look up table that is around 1gb in size in s3 and just had > few million records and 5 columns. > >