[jira] [Created] (FLINK-1670) Collect method for streaming

2015-03-10 Thread JIRA
Márton Balassi created FLINK-1670: - Summary: Collect method for streaming Key: FLINK-1670 URL: https://issues.apache.org/jira/browse/FLINK-1670 Project: Flink Issue Type: New Feature

Re: Website documentation minor bug

2015-03-10 Thread Maximilian Michels
So here are the proposed changes. New Old If there are no objections, I will merge this by the end of the day. Best regards, Max On Mon, Mar 9, 2015 at 4:22 PM, Hermann Gábor wrote: > Thanks Gyula, that helps a lot :D > > Nice solution. Thank you Max! > I also support the reduced header

Re: Website documentation minor bug

2015-03-10 Thread Stephan Ewen
Looks the same to me ;-) The mailing lists do not support attachments... On Tue, Mar 10, 2015 at 11:15 AM, Maximilian Michels wrote: > So here are the proposed changes. > > New > > > Old > > > > > If there are no objections, I will merge this by the end of the day. > > Best regards, > Max > > O

Re: [DISCUSS] Make a release to be announced at ApacheCon

2015-03-10 Thread Robert Metzger
Hey, whats the status on this? There is one week left until we are going to fork off a branch for 0.9 .. if we stick to the suggested timeline. The initial email said "I am very much in favor of doing this, under the strong condition that we are very confident that the master has grown to be stabl

Re: Website documentation minor bug

2015-03-10 Thread Maximilian Michels
Seems like my smart data crawling web mail took the linked images out. So here we go again: New http://i.imgur.com/KK7fhiR.png Old http://i.imgur.com/kP2LPnY.png On Tue, Mar 10, 2015 at 11:17 AM, Stephan Ewen wrote: > Looks the same to me ;-) > > The mailing lists do not support attachments...

Re: [DISCUSS] Offer Flink with Scala 2.11

2015-03-10 Thread Robert Metzger
Hey Alex, I don't know the exact status of the Scala 2.11 integration. But I wanted to point you to https://github.com/apache/flink/pull/454, which is changing a huge portion of our maven build infrastructure. If you haven't started yet, it might make sense to base your integration onto that pull

Re: Website documentation minor bug

2015-03-10 Thread Hermann Gábor
Looks nice, +1 for the new one. On Tue, Mar 10, 2015 at 11:24 AM Maximilian Michels wrote: > Seems like my smart data crawling web mail took the linked images out. > So here we go again: > > New > http://i.imgur.com/KK7fhiR.png > > Old > http://i.imgur.com/kP2LPnY.png > > On Tue, Mar 10, 2015 at

[jira] [Created] (FLINK-1671) Add execution modes for programs

2015-03-10 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-1671: --- Summary: Add execution modes for programs Key: FLINK-1671 URL: https://issues.apache.org/jira/browse/FLINK-1671 Project: Flink Issue Type: Bug Affects Vers

Re: Website documentation minor bug

2015-03-10 Thread Chiwan Park
Looks good! +1 for the new one. Regards. Chiwan Park (Sent with iPhone) > On Mar 10, 2015, at 7:28 PM, Hermann Gábor wrote: > > Looks nice, +1 for the new one. > > On Tue, Mar 10, 2015 at 11:24 AM Maximilian Michels wrote: > >> Seems like my smart data crawling web mail took the linked imag

Re: Website documentation minor bug

2015-03-10 Thread Márton Balassi
+1 On Tue, Mar 10, 2015 at 11:28 AM, Hermann Gábor wrote: > Looks nice, +1 for the new one. > > On Tue, Mar 10, 2015 at 11:24 AM Maximilian Michels > wrote: > > > Seems like my smart data crawling web mail took the linked images out. > > So here we go again: > > > > New > > http://i.imgur.com/K

[DISCUSS] Add method for each Akka message

2015-03-10 Thread Ufuk Celebi
Hey all, I currently find it a little bit frustrating to navigate between different task manager operations like cancel or submit task. Some of these operations are directly done in the event loop (e.g. cancelling), whereas others forward the msg to a method (e.g. submitting). For me, navigati

[jira] [Created] (FLINK-1672) Refactor task registration/unregistration

2015-03-10 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-1672: -- Summary: Refactor task registration/unregistration Key: FLINK-1672 URL: https://issues.apache.org/jira/browse/FLINK-1672 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Make a release to be announced at ApacheCon

2015-03-10 Thread Márton Balassi
On the streaming side: Must have: * Tests for the fault tolerance (My first priority this week) * Merging Gyula's recent windowing PR [1] Really needed: * Self-join for DataStreams (Gabor has a prototype, PR coming today) [1] * ITCase tests for streaming examples (Peter & myself, review a

[jira] [Created] (FLINK-1673) Colocate Flink Kafka consumer

2015-03-10 Thread JIRA
Márton Balassi created FLINK-1673: - Summary: Colocate Flink Kafka consumer Key: FLINK-1673 URL: https://issues.apache.org/jira/browse/FLINK-1673 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Add method for each Akka message

2015-03-10 Thread Stephan Ewen
+1, let's change this lazily whenever we work on an action/message, we pull the handling out into a dedicated method. On Tue, Mar 10, 2015 at 11:49 AM, Ufuk Celebi wrote: > Hey all, > > I currently find it a little bit frustrating to navigate between different > task manager operations like canc

[jira] [Created] (FLINK-1674) Add test with nested avro type

2015-03-10 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-1674: - Summary: Add test with nested avro type Key: FLINK-1674 URL: https://issues.apache.org/jira/browse/FLINK-1674 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Offer Flink with Scala 2.11

2015-03-10 Thread Alexander Alexandrov
We have is almost ready here: https://github.com/stratosphere/flink/commits/scala_2.11_rebased I wanted to open a PR today 2015-03-10 11:28 GMT+01:00 Robert Metzger : > Hey Alex, > > I don't know the exact status of the Scala 2.11 integration. But I wanted > to point you to https://github.com/a

[gelly] Tests fail, but build succeeds

2015-03-10 Thread Stephan Ewen
It seems JobExecution failures are not recognized in some of the Gelly tests. Also, the tests are logging quite a bit, would be nice to make them a bit more quiet. How is the logging is created, btw. The log4j-tests.properties have the log level set to OFF afaik. Here is a log from my latest buil

Streaming Fault Tolerance

2015-03-10 Thread Stephan Ewen
Hi all! I am about to merge the Pull Request from Gyula, Paris, Marton about the streaming Fault Tolerance. Nice work, guys! There are a few things we need to do as a followup in my opinion: --- State Handling --- The state handling of operators and the triggering of checkpoints should be sepa

Re: Website documentation minor bug

2015-03-10 Thread Maximilian Michels
Thanks for the feedback. Merged. On Tue, Mar 10, 2015 at 11:34 AM, Márton Balassi wrote: > +1 > > On Tue, Mar 10, 2015 at 11:28 AM, Hermann Gábor > wrote: > >> Looks nice, +1 for the new one. >> >> On Tue, Mar 10, 2015 at 11:24 AM Maximilian Michels >> wrote: >> >> > Seems like my smart data cr

[jira] [Created] (FLINK-1675) Rework Accumulators

2015-03-10 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-1675: --- Summary: Rework Accumulators Key: FLINK-1675 URL: https://issues.apache.org/jira/browse/FLINK-1675 Project: Flink Issue Type: Bug Components: JobMana

[jira] [Created] (FLINK-1676) enableForceKryo() is not working as expected

2015-03-10 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-1676: - Summary: enableForceKryo() is not working as expected Key: FLINK-1676 URL: https://issues.apache.org/jira/browse/FLINK-1676 Project: Flink Issue Type: Bug

Re: [DISCUSS] Offer Flink with Scala 2.11

2015-03-10 Thread Robert Metzger
Very nice work. The changes are probably somewhat easy to merge. Except for the version properties in the parent pom, there should not be a bigger issue. Can you also add additional build profiles to travis for scala 2.11 ? On Tue, Mar 10, 2015 at 2:50 PM, Alexander Alexandrov < alexander.s.alexa

Re: [jira] [Commented] (FLINK-1106) Deprecate old Record API

2015-03-10 Thread Fabian Hueske
Yeah, I spotted a good amount of optimizer tests that depend on the Record API. I implemented the last optimizer tests with the new API and would volunteer to port the other optimizer tests. 2015-03-10 16:32 GMT+01:00 Stephan Ewen (JIRA) : > > [ > https://issues.apache.org/jira/browse/FLINK-1

Re: [gelly] Tests fail, but build succeeds

2015-03-10 Thread Vasiliki Kalavri
Great, thanks Andra! On Mar 10, 2015 5:52 PM, "Andra Lungu" wrote: > Fixed. My bad. I had a trailing local job manager still running. > > On Tue, Mar 10, 2015 at 5:36 PM, Andra Lungu > wrote: > > > So the sysout suppression worked like a charm. Thanks! > > > > However, this mini cluster setup is

Re: [DISCUSS] Offer Flink with Scala 2.11

2015-03-10 Thread Alexander Alexandrov
Yes, will do. 2015-03-10 16:39 GMT+01:00 Robert Metzger : > Very nice work. > The changes are probably somewhat easy to merge. Except for the version > properties in the parent pom, there should not be a bigger issue. > > Can you also add additional build profiles to travis for scala 2.11 ? > > O

[jira] [Created] (FLINK-1678) Extend internals documentation: program flow, optimizer, ...

2015-03-10 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-1678: - Summary: Extend internals documentation: program flow, optimizer, ... Key: FLINK-1678 URL: https://issues.apache.org/jira/browse/FLINK-1678 Project: Flink

[jira] [Created] (FLINK-1680) Upgrade Flink dependencies for Tachyon to 0.6.0

2015-03-10 Thread Henry Saputra (JIRA)
Henry Saputra created FLINK-1680: Summary: Upgrade Flink dependencies for Tachyon to 0.6.0 Key: FLINK-1680 URL: https://issues.apache.org/jira/browse/FLINK-1680 Project: Flink Issue Type: Tas

Re: [jira] [Commented] (FLINK-1106) Deprecate old Record API

2015-03-10 Thread Till Rohrmann
+1 for removal of old API On Mar 10, 2015 5:41 PM, "Fabian Hueske" wrote: > And I'm +1 for removing the old API with the next release. > > 2015-03-10 17:38 GMT+01:00 Fabian Hueske : > > > Yeah, I spotted a good amount of optimizer tests that depend on the > Record > > API. > > I implemented the l

[jira] [Created] (FLINK-1681) Remove the old Record API

2015-03-10 Thread Henry Saputra (JIRA)
Henry Saputra created FLINK-1681: Summary: Remove the old Record API Key: FLINK-1681 URL: https://issues.apache.org/jira/browse/FLINK-1681 Project: Flink Issue Type: Task Affects Versions

Re: [jira] [Commented] (FLINK-1106) Deprecate old Record API

2015-03-10 Thread Henry Saputra
Thanks guys, I have filed FLINK-1681 [1] to track this issue. Maybe Fabian would like to take stab at this? [1] https://issues.apache.org/jira/browse/FLINK-1681 On Tue, Mar 10, 2015 at 12:28 PM, Till Rohrmann wrote: > +1 for removal of old API > On Mar 10, 2015 5:41 PM, "Fabian Hueske" wrote:

[jira] [Created] (FLINK-1682) Port Record-API based optimizer tests to new Java API

2015-03-10 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-1682: Summary: Port Record-API based optimizer tests to new Java API Key: FLINK-1682 URL: https://issues.apache.org/jira/browse/FLINK-1682 Project: Flink Issue Typ

[jira] [Created] (FLINK-1683) Scheduling preferences for non-unary tasks are not correctly computed

2015-03-10 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-1683: Summary: Scheduling preferences for non-unary tasks are not correctly computed Key: FLINK-1683 URL: https://issues.apache.org/jira/browse/FLINK-1683 Project: Flink

Re: [DISCUSS] Deprecate Spargel API for 0.9

2015-03-10 Thread Andra Lungu
Big +1 for deprecating Spargel :D On Tue, Mar 10, 2015 at 10:02 PM, Vasiliki Kalavri < vasilikikala...@gmail.com> wrote: > Hi all, > > I would like your opinion on whether we should deprecate the Spargel API in > 0.9. > > Gelly doesn't depend on Spargel, it actually contains it -- we have copied

Re: [DISCUSS] Offer Flink with Scala 2.11

2015-03-10 Thread Alexander Alexandrov
The PR is here: https://github.com/apache/flink/pull/477 Cheers! 2015-03-10 18:07 GMT+01:00 Alexander Alexandrov < alexander.s.alexand...@gmail.com>: > Yes, will do. > > 2015-03-10 16:39 GMT+01:00 Robert Metzger : > >> Very nice work. >> The changes are probably somewhat easy to merge. Except fo

Re: [DISCUSS] Deprecate Spargel API for 0.9

2015-03-10 Thread Henry Saputra
Thanks for bringing up for discussion, Vasia I am +1 for deprecating Spargel for 0.9 release. It is confusing for new comer (well even for me) to Flink and found out there are 2 sets of Graph APIs. We could use 0.9 release as stabilization period for Gelly, which is why Spargel is deprecated an

Inconsistent git master

2015-03-10 Thread Gyula Fóra
Hey, I pushed some commits yesterday evening and it seems like the git repos somehow became inconsistent, the https://github.com/apache/flink mirror doesnt show the changes: https://git-wip-us.apache.org/repos/asf/flink.git/?p=flink.git;a=summary vs https://github.com/apache/flink Any ideas on

Re: Inconsistent git master

2015-03-10 Thread Ufuk Celebi
Hey Gyula, Syncing between the two sometimes takes time. :( I don't think that anything is broken. Let's wait a little longer. – Ufuk On Wednesday, March 11, 2015, Gyula Fóra wrote: > Hey, > > I pushed some commits yesterday evening and it seems like the git repos > somehow became inconsistent

Re: Inconsistent git master

2015-03-10 Thread Fabian Hueske
Apparently Github sync was/is down https://issues.apache.org/jira/browse/INFRA-9259 On Mar 11, 2015 7:18 AM, "Ufuk Celebi" wrote: > Hey Gyula, > > Syncing between the two sometimes takes time. :( I don't think that > anything is broken. Let's wait a little longer. > > – Ufuk > > On Wednesday, Mar

Re: Inconsistent git master

2015-03-10 Thread Gyula Fóra
Thanks :) On Wed, Mar 11, 2015 at 7:30 AM, Fabian Hueske wrote: > Apparently Github sync was/is down > https://issues.apache.org/jira/browse/INFRA-9259 > On Mar 11, 2015 7:18 AM, "Ufuk Celebi" wrote: > > > Hey Gyula, > > > > Syncing between the two sometimes takes time. :( I don't think that >