[jira] [Created] (FLINK-3528) Add Incremental Fold for Non-Keyed Window Operator

2016-02-26 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-3528: --- Summary: Add Incremental Fold for Non-Keyed Window Operator Key: FLINK-3528 URL: https://issues.apache.org/jira/browse/FLINK-3528 Project: Flink Issue

[jira] [Created] (FLINK-3527) Scala DataStream has no transform method

2016-02-26 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-3527: --- Summary: Scala DataStream has no transform method Key: FLINK-3527 URL: https://issues.apache.org/jira/browse/FLINK-3527 Project: Flink Issue Type: Impr

[jira] [Created] (FLINK-3526) ProcessingTime Window Assigner and Trigger are broken

2016-02-26 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-3526: --- Summary: ProcessingTime Window Assigner and Trigger are broken Key: FLINK-3526 URL: https://issues.apache.org/jira/browse/FLINK-3526 Project: Flink Issue Type:

Re: JDBC Streaming Connector

2016-02-26 Thread Timothy Farkas
Thanks everyone. I'll take a look at JDBCOutputFormat, and FLINK-3332 along with other related tickets. Tim On Fri, Feb 26, 2016 at 1:55 AM, Chesnay Schepler wrote: > I'm actually working on a JDBC streaming connector with transactions. > > > O

[jira] [Created] (FLINK-3525) Missing call to super#close() in TimestampsAndPeriodicWatermarksOperator#close()

2016-02-26 Thread Ted Yu (JIRA)
Ted Yu created FLINK-3525: - Summary: Missing call to super#close() in TimestampsAndPeriodicWatermarksOperator#close() Key: FLINK-3525 URL: https://issues.apache.org/jira/browse/FLINK-3525 Project: Flink

[jira] [Created] (FLINK-3524) Provide a JSONSerialisationSchema in the kafka connector package

2016-02-26 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-3524: - Summary: Provide a JSONSerialisationSchema in the kafka connector package Key: FLINK-3524 URL: https://issues.apache.org/jira/browse/FLINK-3524 Project: Flink

[jira] [Created] (FLINK-3523) Storm SpoutSplitExample fails with a ClassCastException

2016-02-26 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-3523: --- Summary: Storm SpoutSplitExample fails with a ClassCastException Key: FLINK-3523 URL: https://issues.apache.org/jira/browse/FLINK-3523 Project: Flink Issue Typ

[jira] [Created] (FLINK-3522) Storm examples "PrintSampleStream" throws an error if called without arguments

2016-02-26 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-3522: --- Summary: Storm examples "PrintSampleStream" throws an error if called without arguments Key: FLINK-3522 URL: https://issues.apache.org/jira/browse/FLINK-3522 Project: F

[jira] [Created] (FLINK-3521) Make Iterable part of method signature for WindowFunction

2016-02-26 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-3521: --- Summary: Make Iterable part of method signature for WindowFunction Key: FLINK-3521 URL: https://issues.apache.org/jira/browse/FLINK-3521 Project: Flink

Re: Inconvenient (unforeseen?) consequences of PR #1683

2016-02-26 Thread Márton Balassi
Thanks, +1. On Fri, Feb 26, 2016 at 12:35 PM, Stephan Ewen wrote: > Hi! > > I think it is a release blocker. > > That was a change to make it possible to use Flink with SBT (and make it > easier to use it with Maven). > Just had an offline chat with Till, and we suggest the following solution: >

Re: Congrats on 1000 stars on Github

2016-02-26 Thread Márton Balassi
Great to see that. :) On Fri, Feb 26, 2016 at 1:56 PM, Theodore Vasiloudis < theodoros.vasilou...@gmail.com> wrote: > I'm sure others noticed this as well yesterday, but the project has passed > 1000 stars on Github, > just in time for the 1.0 release ;) > > Here's to the next 1000! > > --Theo >

Congrats on 1000 stars on Github

2016-02-26 Thread Theodore Vasiloudis
I'm sure others noticed this as well yesterday, but the project has passed 1000 stars on Github, just in time for the 1.0 release ;) Here's to the next 1000! --Theo

Re: [VOTE] Release Apache Flink 1.0.0 (RC1)

2016-02-26 Thread Robert Metzger
@Greg: I've merged the two PRs you've mentioned. Since nobody objected, I'll create now a branch for "release-1.0" and I'll update the version in master to 1.1-SNAPSHOT. Please remember that I'm aiming for creating the next RC later today. I keep an eye on blocking issues / open pull requests. If

Re: Inconvenient (unforeseen?) consequences of PR #1683

2016-02-26 Thread Stephan Ewen
Hi! I think it is a release blocker. That was a change to make it possible to use Flink with SBT (and make it easier to use it with Maven). Just had an offline chat with Till, and we suggest the following solution: All libraries should be split into two projects: - library - library-examples

[jira] [Created] (FLINK-3520) Periodic watermark operator should emit current watermark in close()

2016-02-26 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-3520: --- Summary: Periodic watermark operator should emit current watermark in close() Key: FLINK-3520 URL: https://issues.apache.org/jira/browse/FLINK-3520 Project: Flink

Re: Playing with EventTime in DataStreams

2016-02-26 Thread Stephan Ewen
Nice catch, actually. I think we should let the timestamp extracting operator emit the current watermark prior to shutting down. On Fri, Feb 26, 2016 at 11:49 AM, Aljoscha Krettek wrote: > Hi, > I think the problem is that the source finished before the extractor has > the chance to emit even a

Re: Playing with EventTime in DataStreams

2016-02-26 Thread Aljoscha Krettek
Hi, I think the problem is that the source finished before the extractor has the chance to emit even a single watermark. This means that the topology will shut down and the window operator does not emit in-flight windows upon shutdown. Cheers, Aljoscha > On 26 Feb 2016, at 11:40, Nam-Luc Tran w

Re: Playing with EventTime in DataStreams

2016-02-26 Thread Nam-Luc Tran
Great, that did it, thanks Robert ;) While I'm at it: Sometimes results are correctly returned, sometimes, the output of the job (print or writeAsText) is plain empty, like the job finished too quickly before the results are written. One way of "forcing" results to happen is to insert a "delay" i

[jira] [Created] (FLINK-3519) Subclasses of Tuples don't work if the declared type of a DataSet is not the descendant

2016-02-26 Thread Gabor Gevay (JIRA)
Gabor Gevay created FLINK-3519: -- Summary: Subclasses of Tuples don't work if the declared type of a DataSet is not the descendant Key: FLINK-3519 URL: https://issues.apache.org/jira/browse/FLINK-3519 Pro

Re: JDBC Streaming Connector

2016-02-26 Thread Chesnay Schepler
I'm actually working on a JDBC streaming connector with transactions. On 26.02.2016 10:33, Stephan Ewen wrote: Hi Timothy! I would first just write a stream data sink based on the JDBCOutputFormat, and then let's look into making it exactly-once. For JDBC with support for transactions, the bas

Re: JDBC Streaming Connector

2016-02-26 Thread Stephan Ewen
Hi Timothy! I would first just write a stream data sink based on the JDBCOutputFormat, and then let's look into making it exactly-once. For JDBC with support for transactions, the basic idea would be to start a transaction, insert elements, commit when a checkpoint is confirmed, and start a new t

Re: JDBC Streaming Connector

2016-02-26 Thread Till Rohrmann
Hi Timothy, you can use the `JDBCOutputFormat` in combination with the `DataStream.writeUsingOutputFormat` method. However, this won't give you exactly once guarantees since the output formats don't take part in the checkpointing mechanism. Currently, Chesnay is working on a generic framework for

Re: Running stream examples (newbie)

2016-02-26 Thread Till Rohrmann
I just tested executing a streaming example on the current master and everything worked. Can you try clearing the IntelliJ cache and rebuild the project? Cheers, Till On Thu, Feb 25, 2016 at 5:13 PM, Tara Athan wrote: > Hi, I am just exploring Flink, and have run into a curious issue. I have >

[jira] [Created] (FLINK-3518) Stale docs for quickstart setup

2016-02-26 Thread Stefano Baghino (JIRA)
Stefano Baghino created FLINK-3518: -- Summary: Stale docs for quickstart setup Key: FLINK-3518 URL: https://issues.apache.org/jira/browse/FLINK-3518 Project: Flink Issue Type: Bug C