Hi TD,
Thank you for the quick replying and backing my approach. :)
1) The example is this:
1. In the first 2 second interval, after updateStateByKey, I get a few keys
and their states, say, ("a" -> 1, "b" -> 2, "c" -> 3)
2. In the following 2 second interval, I only receive "c" and "d" and their
value. But I want to update/display the state of "a" and "b" accordingly.
* It seems I have no way to "access" the "a" and "b" and get their states.
* also, do I have a way to show all the existing states?
I guess the approach to solve this will be similar to what you mentioned
for 2). But the difficulty is that, if I want to display all the existing
states, need to bundle all the rest keys to one key.
Thank you.
Cheers,
Fang, Yan
yanfang...@gmail.com
+1 (206) 849-4108
On Thu, Jul 17, 2014 at 11:36 AM, Tathagata Das <tathagata.das1...@gmail.com
> wrote:
> For accessing previous version, I would do it the same way. :)
>
> 1. Can you elaborate on what you mean by that with an example? What do you
> mean by "accessing" keys?
>
> 2. Yeah, that is hard to do with the ability to do point lookups into an
> RDD, which we dont support yet. You could try embedding the related key in
> the values of the keys that need it. That is, B will is present in the
> value of key A. Then put this transformed DStream through updateStateByKey.
>
> TD
>
