the discussion (re-loading batch RDD from file within spark
> steraming context) - lets leave that since we are not getting anywhere
>
> -Original Message-
> From: Sean Owen [mailto:so...@cloudera.com]
> Sent: Wednesday, April 15, 2015 8:30 PM
> To: Evo Eftimov
> Cc: user@
e not getting anywhere
-Original Message-
From: Sean Owen [mailto:so...@cloudera.com]
Sent: Wednesday, April 15, 2015 8:30 PM
To: Evo Eftimov
Cc: user@spark.apache.org
Subject: Re: adding new elements to batch RDD from DStream RDD
What API differences are you talking about? a DStream gives
only way to keep reloading a batch RDD from file is from a
>> separate thread still using the same spark context. The thread will
>> reload the batch RDD with the same reference ie reassign the reference
>> to the newly instantiated/loaded batch RDD - is that what you mean by
&
h RDDs from file for e.g. a second
time moreover after specific period of time
-Original Message-
From: Sean Owen [mailto:so...@cloudera.com]
Sent: Wednesday, April 15, 2015 8:14 PM
To: Evo Eftimov
Cc: user@spark.apache.org
Subject: Re: adding new elements to batch RDD from DStream RDD
loading batch RDD from file
>
> -Original Message-
> From: Sean Owen [mailto:so...@cloudera.com]
> Sent: Wednesday, April 15, 2015 7:43 PM
> To: Evo Eftimov
> Cc: user@spark.apache.org
> Subject: Re: adding new elements to batch RDD from DStream RDD
>
> What do you m
iginal Message-
From: Sean Owen [mailto:so...@cloudera.com]
Sent: Wednesday, April 15, 2015 7:43 PM
To: Evo Eftimov
Cc: user@spark.apache.org
Subject: Re: adding new elements to batch RDD from DStream RDD
What do you mean by "batch RDD"? they're just RDDs, though store their d
g DStream RDDs will be able to join with the thus
> previously updated BAtch RDD and produce a result DStream RDD
>
> Something almost like that can be achieved with updateStateByKey, but is
> there a way to do it as described here
>
>
>
> --
> View this message in context:
lmost like that can be achieved with updateStateByKey, but is
there a way to do it as described here
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/adding-new-elements-to-batch-RDD-from-DStream-RDD-tp22504.html
Sent from the Apache Spark User List mailing li
