This is a case where resources are fixed in the same SparkContext, but sqls
have different priorities.
Some SQLs are only allowed to be executed if there are spare resources,
once the high priority sql comes in, those sqls taskset either are killed
or stalled.
If we set a high priority pool's mi
gt;
>
>
>
>
> On Tue, Nov 12, 2019 at 7:18 AM Chang Chen wrote:
>
>>
>> Hi all
>>
>> I meet a case where I need cache a source RDD, and then create different
>> DataFrame from it in different threads to accelerate query.
>>
>> I know that
Hi all
I meet a case where I need cache a source RDD, and then create different
DataFrame from it in different threads to accelerate query.
I know that SparkSession is thread safe(
https://issues.apache.org/jira/browse/SPARK-15135), but i am not sure
whether RDD si thread safe or not
Thanks
Cha
t;
> Co-Founder & CTO | Equalum
>
> Mobile: +972-54-7801286 | Email: ofir.ma...@equalum.io
>
> On Wed, Jul 27, 2016 at 11:24 AM, Chang Chen wrote:
>
>>
>> I don't understand what kind of low level control that DStream can do
>> while Structure Streaming ca
ut when to switch to
> Structured Streaming - each of us have a different risk/value tradeoff,
> based on our specific situation...
>
> Ofir Manor
>
> Co-Founder & CTO | Equalum
>
> Mobile: +972-54-7801286 | Email: ofir.ma...@equalum.io
>
>
> On Wed, Jul 27, 2016
Hi guys
Structure Stream is coming with spark 2.0, but I noticed that DStream is
still here
What's the future of the DStream, will it be deprecated and removed
eventually? Or co-existed with Structure Stream forever?
Thanks
Chang
