[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2016-03-11 Thread ggevay
Github user ggevay commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-195402038 This is dead. Everything has totally changed that this was based on. The fundamental problem is that this is based on the assumption of the old 0.9 windowing code that ev

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2016-03-11 Thread ggevay
Github user ggevay closed the pull request at: https://github.com/apache/flink/pull/684 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2016-03-11 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-195395736 Is this still a relevant PR with all the changes to the runtime in recent months? If not, can we close this? --- If your project is set up for it, you can reply to this emai

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-06-22 Thread ggevay
Github user ggevay commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-114086691 I see your point, no problem. Although, I would like to put my next pull request (856) to the core, because that code would be very awkward to put into contrib. Bu

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-06-22 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-114083769 Concerning my comment about where to put the code: There is so much code being added to Flink right now. The committers have to be ultimately the ones that ma

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-06-21 Thread ggevay
Github user ggevay commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-113907348 I merged Marton's changes (moving to contrib) to this PR. I also added a paragraph to the streaming programming guide. I couldn't think of any better name for WindowedD

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-06-21 Thread ggevay
Github user ggevay commented on a diff in the pull request: https://github.com/apache/flink/pull/684#discussion_r32892426 --- Diff: flink-staging/flink-streaming/flink-streaming-core/src/test/java/org/apache/flink/streaming/util/FieldAccessorTest.java --- @@ -0,0 +1,75 @@ +/*

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-06-20 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/684#discussion_r32888221 --- Diff: flink-staging/flink-streaming/flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/WindowedDataStream.scala --- @@ -339,4 +335

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-06-20 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/684#discussion_r32888195 --- Diff: flink-staging/flink-streaming/flink-streaming-core/src/test/java/org/apache/flink/streaming/util/FieldAccessorTest.java --- @@ -0,0 +1,75 @@ +/

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-06-20 Thread mbalassi
Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-113820597 Merged the first commit as that was already clarified. As for the second one the placement to either `streaming-core` or `streaming-contrib` is still unclear. I moved it

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-06-10 Thread ggevay
Github user ggevay commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-110739389 I did the Scala part, and added a Jira (FLINK-2145). As for the placement: @StephanEwen: > I think we should put the code into a streaming operator library. The

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-05-26 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-105519930 Concerning Marton's comment: I think we should put the code into a streaming operator library. The streaming core is undergoing heavy changes, the windowing m

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-05-26 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/684#discussion_r31030179 --- Diff: flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/util/FieldAccessor.java --- @@ -0,0 +1,201 @@ +/

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-05-25 Thread ggevay
Github user ggevay commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-105249478 I coded the grouped case and did the refactoring of SumAggregator and ComparableAggregator to use the new FieldAccessor class to access the user-specified field. The first

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-05-18 Thread ggevay
Github user ggevay commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-103178282 Thank you for your comments. @rmetzger: We will discuss creating the Jira issues with @mbalassi. I fixed the catching of the IllegalAccessException.

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-05-18 Thread mbalassi
Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-103035236 In general I think we should discuss whether we make Gabor's GSoC project part of the core streaming API or have them separately in a streaming windowing statistics pack

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-05-18 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/684#discussion_r30488224 --- Diff: flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/windowing/windowbuffer/MedianPreReducer.java --- @@

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-05-18 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/684#discussion_r30488171 --- Diff: flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/windowing/helper/fieldaccessors/PojoFieldAccessor.java

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-05-18 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/684#issuecomment-102964703 Hi, thank you for contributing to Flink! Is there a JIRA issue available for this task? --- If your project is set up for it, you can reply to this email and have yo

[GitHub] flink pull request: [streaming] Fast calculation of medians of win...

2015-05-17 Thread ggevay
GitHub user ggevay opened a pull request: https://github.com/apache/flink/pull/684 [streaming] Fast calculation of medians of windows [WIP] I started working on my Google Summer of Code Project. I decided to start with the fast calculation of medians of windows. The MedianPreReducer