Hello,

I have a question about TumblingProcessingTimeWindow and
ContinuousProcessingTimeTrigger.

The code I tried is below. Output the distinct count of the words,
counts are printed every 5 seconds and window is reset every 1 minute.

---
    val input =
env.readFileStream(fileName,100,FileMonitoringFunction.WatchType.PROCESS_ONLY_APPENDED)
      .flatMap { _.toLowerCase.split("\\W+") filter { _.nonEmpty } }
      .timeWindowAll(Time.of(60, TimeUnit.SECONDS))
      .trigger(ContinuousProcessingTimeTrigger.of(Time.seconds(5)))
      .fold(Set[String]()){(r,i) => { r + i}}
      .map{x => (new Timestamp(System.currentTimeMillis()), x.size)}

    input print
---

I wrote data to the input file with some interval.

---
echo "aaa" >> input.txt
echo "aaa" >> input.txt
sleep 10
echo "bbb" >> input.txt
sleep 60
echo "ccc" >> input.txt
---

The result I got was just 1 record. The expected output was 1 -> (10+
sec later) 2 -> (60+ sec later) 1 .
---
(2016-03-18 13:08:59.288,2)
---

Even after several minutes, I never got additional record. In my
understanding, with
ContinuousProcessingTimeTrigger.of(Time.seconds(5)), the last two
operator (fold, map) in the code above will be evaluated every 5
seconds.
Am I mis-understand something?

Regards,
Hironori

Reply via email to