Re: Debugging watermarks?

2016-05-21 Thread Aljoscha Krettek
Hi Niels, you can log the watermarks by implementing a custom operator. (Operators have access to the watermarks.) The map operator is a good example for this: @Internal public class StreamMap extends AbstractUdfStreamOperator> implements OneInputStreamOperator { private stati

Debugging watermarks?

2016-05-21 Thread Niels Basjes
Hi, I was working on a streaming application last week and I got stuck in a situation where I got the same time based window many times. I expect that I made a mistake in creating the watermarks in relation to the data I have and the watermark generating code. Writing the events to the console (f

Re: Weird Kryo exception (Unable to find class: java.ttil.HashSet)

2016-05-21 Thread Flavio Pompermaier
I think this bug comes from something in SpillingAdaptiveSpanningRecordDeserializer..I've tried to find a common point of failure in all those messages and I found that it contains also this error message that I got once: private static final String BROKEN_SERIALIZATION_ERROR_MESSAGE =

Re: Weird Kryo exception (Unable to find class: java.ttil.HashSet)

2016-05-21 Thread Flavio Pompermaier
I tried to move flink tmp dir from hdd disks to sdd ones (in order to exclude faulty disks) and I had another of those Exception: java.lang.IllegalArgumentException: The datetime zone id 'Europe/Romd' is not recognised at org.joda.time.DateTimeZone.forID(DateTimeZone.java:229) at de.javakaffee.kr

Re: [RichFlattMapfunction] Configuration File

2016-05-21 Thread Aljoscha Krettek
Hi Simon, regarding 1. yes, the value that you get from state_item.value() and that you set using state_item.update() is scoped to the key of the incoming element. regarding 2. the open(conf: Configuration) signature is legacy from how Functions used to work quite a while back. In the streaming AP