That way, you will eventually end up bloating up that list. Instead, you
could push the stream to a noSQL database (like hbase or cassandra etc) and
then read it back and join it with your current stream if that's what you
are looking for.
Thanks
Best Regards
On Thu, Oct 15, 2015 at 6:11 PM, Yogesh Vyas <informy...@gmail.com> wrote:
> ---------- Forwarded message ----------
> From: Yogesh Vyas <informy...@gmail.com>
> Date: Thu, Oct 15, 2015 at 6:08 PM
> Subject: Get the previous state string
> To: user@spark.apache.org
>
>
> Hi,
> I am new to Spark and was trying to do some experiments with it.
>
> I had a JavaPairDStream<String, List<String>> RDD.
> I want to get the list of string from its previous state. For that I
> use updateStateByKey function as follows:
>
> final Function2<List<String>, Optional<List<String>>,
> Optional<List<String>>> updateFunc =
> new Function2<List<String>, Optional<List<String>>,
> Optional<List<String>>>() {
>
> public Optional<List<String>> call(List<String> arg0,
> Optional<List<String>> arg1) throws Exception {
> // TODO Auto-generated method stub
> if(arg1.toString()==null)
> return Optional.of(arg0);
> else {
> arg0.add(arg1.toString());
> return Optional.of(arg0);
> }
> }
> };
>
> I want the function to append the new list of string to the previous
> list and return the new list. But I am not able to do so. I am getting
> the " java.lang.UnsupportedOperationException" error.
> Can anyone which help me out in getting the desired output?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>
