Re: DeserializationSchema isEndOfStream usage?

2016-01-22 Thread David Kim
Hi Robert, Awesome, thanks for the fast turnaround! Cheers, David On Fri, Jan 22, 2016 at 11:17 AM, Robert Metzger wrote: > Hi David, > > you are right. I'll fix the issue in this pull request: > https://github.com/apache/flink/pull/1541 > > I guess everything in your topology runs with a para

Re: DeserializationSchema isEndOfStream usage?

2016-01-22 Thread Robert Metzger
Hi David, you are right. I'll fix the issue in this pull request: https://github.com/apache/flink/pull/1541 I guess everything in your topology runs with a parallelism of 1? Running it with a parallelism higher than 1 will also work around the issue (because then the two Sinks are not executed in

Re: DeserializationSchema isEndOfStream usage?

2016-01-22 Thread David Kim
Hi Robert, Thanks for the workaround. Unfortunately I think I found a bug in the code that controls the metrics logic. Should Boolean.getBoolean be Boolean.valueOf instead? https://github.com/apache/flink/blob/81320c1c7ee98b9a663998df51cc4d5aa73d9b2a/flink-streaming-connectors/flink-connector-ka

Re: Redeployements and state

2016-01-22 Thread Niels Basjes
Hi, @Max: Thanks for the new URL. I noticed that a lot (in fact almost all) of links in the new manuals lead to 404 errors. Maybe you should run an automated test to find them all. I did a bit of reading about the savepoints and that in fact they are written as "Allow to trigger checkpoints manua

Re: JDBCInputFormat GC overhead limit exceeded error

2016-01-22 Thread Robert Metzger
@Max: If you're too busy right now to open a PR for this, I can also do it ... just let me know. On Wed, Jan 20, 2016 at 7:28 PM, Stephan Ewen wrote: > Super, thanks for finding this. Makes a lot of sense to have a result set > that does hold onto data. > > Would be great if you could open a pul

Re: Backpressure in the context of JDBCOutputFormat update

2016-01-22 Thread Robert Metzger
Hi, have you thought about making two independent jobs out of this? (or you call execute() for the two separate parts) One job for the update() and one for the insert() ? Even though the update operation should not be expensive, I think its helpful to understand the performance impact of having c

Re: Results of testing Flink quickstart against 0.10-SNAPSHOT and 1.0-SNAPSHOT (re. Dependency on non-existent org.scalamacros:quasiquotes_2.11:)

2016-01-22 Thread Robert Metzger
Hi Prez, https://repository.apache.org/content/repositories/snapshots is the only repository distributing snapshot builds. The latest scala 2.11 1.0-SNAPSHOT flink runtime: https://re

Re: DeserializationSchema isEndOfStream usage?

2016-01-22 Thread Robert Metzger
Hi David, thank you for reporting the issue. I'll look into it. In the meantime, you can set "flink.disable-metrics" to "true" in the properties. This way, you disable the metrics. I'll probably have to introduce something like a client id to differentiate between the producers. Robert On Thu, J