Re: Finding the average temperature

2016-02-13 Thread Nirmalya Sengupta
Hello Stefano Many thanks for responding so quickly. Your explanation not only confirms my understanding but gives a much simpler solution. The facility of associating a specific parallelism to a given operator didn't strike me at all. You are right that for my particular UseCase, that is the si

Re: streaming using DeserializationSchema

2016-02-13 Thread Martin Neumann
I ended up not using the DeserializationSchema and instead going for a AvrioInputFormat in case of reading From file. I would have preferred to keep the code simpler but the map solution was a lot more complicated since the raw data I have is in Avro binary format so I cannot just read it and map i

Re: Finding the average temperature

2016-02-13 Thread Stefano Baghino
Hi Nirmalaya, I think for your use case in particular it's enough to specify the reducer that computes the average to have a parallelism of 1 by calling the `setParallelism` API when you apply it. Keep in mind that you can still enjoy a high level of parallelism up until the last operator by using

Finding the average temperature

2016-02-13 Thread Nirmalya Sengupta
Hello Flinksters, This is perhaps too trivial for most here in this forum, but I want to have my understanding clear. I want to find the average of temperatures coming in as a stream. The tuple as the following fields: probeID,readingTimeStamp,radiationLevel,photoSensor,humidity,ambientTemperatu

Regarding Concurrent Modification Exception

2016-02-13 Thread Biplob Biswas
Hi, We are getting a ConcurrentModificationException, the complete stack trace is as follows: org.apache.flink.optimizer.CompilerException: Error translating node 'Data > Source "at compute(ArpackSVD.java:367) > (org.apache.flink.api.java.io.CollectionInputFormat)" : NONE [[ > GlobalProperties [pa