Hi,
Why spark and why scala?
Regards,
Gourav
On Wed, 7 Sept 2022, 21:42 Mayur Benodekar, <askma...@gmail.com> wrote:
> am new to scala and spark both .
>
> I have a code in scala which executes quieres in while loop one after the
> other.
>
> What we need to do is if a particular query takes more than a certain time
> , for example # 10 mins we should be able to stop the query execution for
> that particular query and move on to the next one
>
> for example
>
> do {
> var f = Future(
> spark.sql("some query"))
> )
>
> f onSucess {
> case suc - > println("Query ran in 10mins")
> }
>
> f failure {
> case fail -> println("query took more than 10mins")
> }
>
> }while(some condition)
>
> var result = Await.ready(f,Duration(10,TimeUnit.MINUTES))
>
> I understand that when we call spark.sql the control is sent to spark
> which i need to kill/stop when the duration is over so that i can get back
> the resources
>
> I have tried multiple things but I am not sure how to solve this. Any help
> would be welcomed as i am stuck with this.
>
> --
> Cheers,
> Mayur
>
