Just a suggestion,
Looks like its timing out when you are broadcasting big object. Generally
its not advisable to do so, if you can get rid of that, program may behave
consistent.
On Tue, Jul 21, 2020 at 3:17 AM Piyush Acharya
wrote:
> spark.conf.set("spark.sql.broadcastTimeout", ##)
>
> O
spark.conf.set("spark.sql.broadcastTimeout", ##)
On Mon, Jul 20, 2020 at 11:51 PM Amit Sharma wrote:
> Please help on this.
>
>
> Thanks
> Amit
>
> On Fri, Jul 17, 2020 at 9:10 AM Amit Sharma wrote:
>
>> Hi, sometimes my spark streaming job throw this exception Futures timed
>> out after
"spark.sql.broadcastTimeout" is the config you can use:
https://github.com/apache/spark/blob/fe07521c9efd9ce0913eee0d42b0ffd98b1225ec/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala#L863
Thanks,
Terry
On Mon, Jul 20, 2020 at 11:20 AM Amit Sharma wrote:
> Please help on t
Please help on this.
Thanks
Amit
On Fri, Jul 17, 2020 at 9:10 AM Amit Sharma wrote:
> Hi, sometimes my spark streaming job throw this exception Futures timed
> out after [300 seconds].
> I am not sure where is the default timeout configuration. Can i increase
> it. Please help.
>
>
>
> Thanks
