Re: Partitioning key range

2019-04-08 Thread Davood Rafiei
a custom KeySelector can be helpful, you can define the key used to >> partition the stream. You can ref the code[1] for detail. >> >> [1] >> https://github.com/apache/flink/blob/8d05e91945c6c8d83f9924c00890ccf350f1f36f/flink-streaming-java/src/main/java/org/apache/flink/stre

Partitioning key range

2019-04-04 Thread Davood Rafiei
Hi all, I partition DataStream (say dsA) with parallelism 2 and get KeyedStream (say ksA) with parallelism 2. Depending on my keys in dsA, one partition remains empty in ksA. For example when my keys are 10 and 20 in dsA, then both partitions in ksA are full. However, with keys 1000 and 1001, only

Flink strange stream join behavior

2016-10-15 Thread Davood Rafiei
Hi, I am experiencing strange flink stream windowed join behavior. I want to do windowed (processing time) join between two partitioned streams. I read data from socket. I have two cases: 1. data speed in socket is relatively slow (say 1K ps) 2. data speed in socket is high (say 37K). The number

Flink timestamps

2016-08-06 Thread Davood Rafiei
Hi, I want to assign timestamps to tuples when they enter source operator , get those timestamps in sink operator and extract them from current time and output. Basically my goal is to measure latency. To set timestamps in source operator I used: * env.setStreamTimeCharacteristic(TimeCharact