That makes sense! Maybe we can make MapState implement Iterable interface.
> 在 2016年10月19日，下午5:48，SHI Xiaogang 写道：
>
> Hi Jark
>
> If the state is very big, it may occupy a lot of memory if we return
> Set>.
>
> By wrapping the returned iterator, we can easily implement a method
> returning I
That makes sense! Maybe we can make MapState implement Iterable interface, so
that we can use foreach directly on MapState.
- Jark Wu
> 在 2016年10月19日，下午5:48，SHI Xiaogang 写道：
>
> Hi Jark
>
> If the state is very big, it may occupy a lot of memory if we return
> Set>.
>
> By wrapping the ret
Hi Jark
If the state is very big, it may occupy a lot of memory if we return
Set>.
By wrapping the returned iterator, we can easily implement a method
returning Iterable>.
Users can use that returned Iterable in the foreach loop.
Regards
Xiaogang
2016-10-19 17:43 GMT+08:00 Jark Wu :
> Hi Xi
Perfect! Then it's pretty much what we discussed here:
https://issues.apache.org/jira/browse/FLINK-3947 and I'm very much in
favour of that. Just the implementation of RocksDB could be a bit tricky
but it should be doable.
Cheers,
Aljoscha
On Wed, 19 Oct 2016 at 11:43 Jark Wu wrote:
> Hi Xiaoga
Hi Xiaogang,
I think maybe return Set> is better than
Iterator>.
Because users can use foreach on Set but not Iterator, and can use iterator
access via set.iterator().
Maybe Map.entrySet() is a more familiar way to users.
- Jark Wu
> 在 2016年10月19日，下午5:18，SHI Xiaogang 写道：
>
> Agreed.
>
>
The keys in the MapState are different from the keys in the KeyedStream.
There are two types of keys: Stream Keys and User Keys.
StreamKeys are those created by the keyBy() operation.
As well as other states, MapStates are created under StreamKeys.
The keys of the MapState are called UserKeys. U
Hi,
just making sure I understand this correctly. Would the MapState keys be
the same keys as the one provided when creating the KeyedStream or a
different key.
As an example, would it be like this:
DataStream> input = ...;
KeyedStream keyed = input.keyBy(0)
keyed.map( Tuple2 input -> mapState.pu
Agreed.
contains(K key) should be provided.
The iterator() method should return Iterator> instead of
Iterator>.
Besides, size() may also be provided.
With these methods, MapStates appear very similar to Java Maps. Users will
be very happy to use them.
Regards,
Xiaogang
2016-10-19 16:55 GMT+08
Hi Xiaogang,
I really like your proposal and think that this would be a valuable
addition to Flink :-)
For convenience we could maybe add contains(K key), too.
Java's Map interface returns a Set of Entry when calling entrySet() (which
is the equivalent of iterator() in our interface). The Entry
Hi, all. I created the JIRA https://issues.apache.org/jira/browse/FLINK-4856 to
propose adding MapStates into Flink.
MapStates are very useful in our daily jobs. For example, when implementing
DistinctCount, we store the values into a MapState and the result of each
group(key) is exactly the numbe
Xiaogang Shi created FLINK-4856:
---
Summary: Add MapState for keyed streams
Key: FLINK-4856
URL: https://issues.apache.org/jira/browse/FLINK-4856
Project: Flink
Issue Type: New Feature
11 matches
Mail list logo
