Apache Flink transactions

Hi Admin Do we have insert, update and remove operations on Apache Flink? For example: I have 10 million records in my test file. I want to add one record, update one record and remove one record from this test file. How to implement it by Flink? Thanks. Best regards Email:

Re: Failing tests policy

I tend to agree with Ufuk, although it would be nice to fix them very quickly. On Thu, Jun 4, 2015 at 1:26 AM, Stephan Ewen wrote: > @matthias: That is the implicit policy right now. Seems not to work... > > On Thu, Jun 4, 2015 at 12:40 AM, Matthias J. Sax < > mj...@informatik.hu-berlin.de> wrote

Re: Failing tests policy

@matthias: That is the implicit policy right now. Seems not to work... On Thu, Jun 4, 2015 at 12:40 AM, Matthias J. Sax < mj...@informatik.hu-berlin.de> wrote: > I basically agree that the current policy on not optimal. However, I > would rather give failing tests "top priority" to get fixed (if

Re: MultipleLinearRegression - Strange results

Any form of generalized linear regression should use adaptive learning rates rather than simple SGD. One of the current best methods is adagrad although there are variants such as RMS prop and adadelta. All are pretty easy to implement. Here is some visualization of various methods that provides

Re: Failing tests policy

I basically agree that the current policy on not optimal. However, I would rather give failing tests "top priority" to get fixed (if possible within one/a-few days) and not disable them. -Matthias On 06/04/2015 12:32 AM, Ufuk Celebi wrote: > Hey all, > > we have certain test cases, which are fai

Failing tests policy

Hey all, we have certain test cases, which are failing regularly on Travis. In all cases I can think of we just keep the test activated. I think this makes it very hard for regular contributors to take these failures seriously. I think the following situation is not unrealistic with the current p

Re: Changed the behavior of "DataSet.print()"

I'll prepare a fix... On Wed, Jun 3, 2015 at 10:24 PM, Stephan Ewen wrote: > +1 for printOnTaskManager(prefix) > > +1 for deprecating the print(prefix) method. > > On Tue, Jun 2, 2015 at 5:24 PM, Aljoscha Krettek > wrote: > >> By the way, we also should rename the corresponding Streaming API >>

Re: Gelly Roadmap

On 03 Jun 2015, at 23:51, Stephan Ewen wrote: > It may make sense to link the JIRAs from the roadmap, or add comments what > people are working on and what is in the queue. Linking JIRAs from the roadmap would solve all these points. ;) So +1 for issue links.

Re: ClassNotFoundException : org.apache.flink.api.common.operators.util.UserCodeObjectWrapper, while trying to run locally

What IDE are you using, that you have to refer to the dependencies as plugins? On Wed, Jun 3, 2015 at 11:25 PM, santosh_rajaguru wrote: > sorry for the mistake. that is referred to plugin development. > So i had to manage the flink dependencies as plugins to make it work. > but due to some reaso

Re: How do we want to maintain our documentation?

+1 for Robert's suggestion. In fact, I thought this was already our practice. Also I would not allow exceptions from that rule in the "stable" codebase. Writing documentation and describing how stuff should be used lets you think about it in a different way and can help to make the feature better.

Re: Gelly Roadmap

It may make sense to link the JIRAs from the roadmap, or add comments what people are working on and what is in the queue. Stephan On Wed, Jun 3, 2015 at 4:12 PM, Vasiliki Kalavri wrote: > Hey, > > yes, this is kind of an ambitious and long-term roadmap :) > > As Andra said, tasks 1-2-3-4 are a

Re: Send events to parallel operator instances

I think that it may be a bit pre-mature to invest heavily into the parallel delta-policy windows just yet. We have not even answered all questions on the key-local delta windows yet: - How does it behave with non-monotonous changes? What does the delta refer to, the max interval in the window, th

Re: ClassNotFoundException : org.apache.flink.api.common.operators.util.UserCodeObjectWrapper, while trying to run locally

sorry for the mistake. that is referred to plugin development. So i had to manage the flink dependencies as plugins to make it work. but due to some reason it cannot find that particular class. -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Class

Re: ClassNotFoundException : org.apache.flink.api.common.operators.util.UserCodeObjectWrapper, while trying to run locally

Can you tell us what you mean with "Plugin environment" ? On Tue, Jun 2, 2015 at 9:38 PM, santosh_rajaguru wrote: > Thanks Robert and Stephan. > > I executed the program in maven environment. It works fine. > However, in Plugin environment, though i have included all the dependencies > and depe

Re: Fwd: Discussion: Storm Comparability Layer

Thanks for the info. I am just a little bit "disappointed". The whole rewrite to the new interface was unnecessary... We need to "revert" everything again... I can also stop working on the new StormSpoutWrapper and StormSpoutCollector implementation in this case... Let's see how it goes. But from

Re: Changed the behavior of "DataSet.print()"

+1 for printOnTaskManager(prefix) +1 for deprecating the print(prefix) method. On Tue, Jun 2, 2015 at 5:24 PM, Aljoscha Krettek wrote: > By the way, we also should rename the corresponding Streaming API > method accordingly. > > On Tue, Jun 2, 2015 at 3:24 PM, Maximilian Michels wrote: > > +1

[jira] [Created] (FLINK-2155) Add an additional checkstyle validation for illegal imports

Lokesh Rajaram created FLINK-2155: - Summary: Add an additional checkstyle validation for illegal imports Key: FLINK-2155 URL: https://issues.apache.org/jira/browse/FLINK-2155 Project: Flink

[jira] [Created] (FLINK-2154) ActiveTriggerPolicy collects elements to a closed buffer after job finishes

Márton Balassi created FLINK-2154: - Summary: ActiveTriggerPolicy collects elements to a closed buffer after job finishes Key: FLINK-2154 URL: https://issues.apache.org/jira/browse/FLINK-2154 Project:

Re: checkstyle failure

Thanks Marton. Sent from my iPhone > On Jun 3, 2015, at 11:59 AM, Márton Balassi wrote: > > Assigned the JIRA to you Lokesh, happy coding. :) > > Best, > > Marton > > On Wed, Jun 3, 2015 at 5:02 PM, Márton Balassi > wrote: > >> Sure, feel free to take it. :) >> >> On Wed, Jun 3, 2015 at 5

Re: Fwd: Discussion: Storm Comparability Layer

Thanks for the updates, Matthias. Both of your questions get an other context, because we have decided to go back to the run()/cancel() type of source interface - but with a slightly changed signature to enable "transactional" operator state checkpointing. You can check out the new source interfac

Re: checkstyle failure

Assigned the JIRA to you Lokesh, happy coding. :) Best, Marton On Wed, Jun 3, 2015 at 5:02 PM, Márton Balassi wrote: > Sure, feel free to take it. :) > > On Wed, Jun 3, 2015 at 5:00 PM, Lokesh Rajaram > wrote: > >> Hello Marton, >> >> Nice find. can I work on this JIRA? >> >> Thanks, >> Lokes

Re: JIRA guidelines

Thanks Till. On Wed, Jun 3, 2015 at 9:06 AM, Till Rohrmann wrote: > Hi Lokesh, > > there are no real guidelines for opening JIRA issues. You should select the > right project to which assign the JIRA, namely Flink. Then choose an > appropriate type, e.g. bug, improvement or feature. Give a descr

Re: Buffer re-ordering problem

Hi Sebastian! The first error that you report looks could also be a lost buffer, rather than a reordering. The second error that you post seems a serialization issue. Both can be consequences of a network stream corruption. Would be good to figure out why the netty event loop does not encode/deco

AW: MultipleLinearRegression - Strange results

We should probably look into this nevertheless. Requiring full grid search for a simple algorithm like mlr sounds like overkill. Do you have written down the math of your implementation somewhere? -M - Ursprüngliche Nachricht - Von: "Till Rohrmann" Gesendet: ‎02.‎06.‎2015 11:31 An: "de

Re: Fwd: Discussion: Storm Comparability Layer

I just pushed my changes to Marton's "storm" branch. It is still open how to process with the following (please give feedback): StormSpoutWrapper: - do we still need "isRunning" and "cancel()"? The new API should make them obsolete from my point of view. - I would avoid "busy wait" in "next()"

Re: Buffer re-ordering problem

Hey Sebastian, would you mind sharing the code and dataset with me (privately would work fine). I want to try to reproduce this as well. – Ufuk On Wed, Jun 3, 2015 at 11:07 AM, Kruse, Sebastian wrote: > I am currently using 0.9-SNAPSHOT. All the non-jar files are from an older > build, but I r

Re: Old new web interface

Sorry, forgot the link: https://github.com/apache/flink/pull/677 On Wed, Jun 3, 2015 at 6:53 PM, Stephan Ewen wrote: > Hi Matthias! > > The new web-frontend effort that I have started is initially mainly > rewriting the web interface that the JobManager offers to monitor current > jobs. > > The

Re: Old new web interface

Hi Matthias! The new web-frontend effort that I have started is initially mainly rewriting the web interface that the JobManager offers to monitor current jobs. The improvements are to use a more lightweight and easier extensible backend, and to have more frequent updates on statistics during job

Re: How do we want to maintain our documentation?

Hi, in principle I agree with Robert's suggestion. I can only see two cases where this might not work well: - if the change is something big / totally new and the documentation to be written is large. In this case, I would prefer to have a separate issue for docs to ensure better quality and rev

Re: JIRA guidelines

On 03 Jun 2015, at 18:06, Till Rohrmann wrote: > Hi Lokesh, > > there are no real guidelines for opening JIRA issues. You should select the > right project to which assign the JIRA, namely Flink. Then choose an > appropriate type, e.g. bug, improvement or feature. Give a descriptive > summary o

Re: JIRA guidelines

Hi Lokesh, there are no real guidelines for opening JIRA issues. You should select the right project to which assign the JIRA, namely Flink. Then choose an appropriate type, e.g. bug, improvement or feature. Give a descriptive summary of the problem and if necessary a further explanation in descri

Re: Null pointer exception on pl.project13.maven:git-commit-id-plugin:2.1.5:revision

No, just the version upgrade didn't fix the issue. Only after I upgraded the version to 2.1.10 and setting the flag flink-runtime builds successfully. The latest version of git-commit-id-plugin that can be used for the build to be successful is the 2.1.13. 2015-06-03 17:14 GMT+02:00 Robert Metzge

Re: How do we want to maintain our documentation?

For me it also makes sense that the contributor who has implemented the changes and thus knows them best should update the documentation accordingly. +1 On Wed, Jun 3, 2015 at 5:25 PM, Chiwan Park wrote: > +1 Good. PR should contain documentation. > > Regards, > Chiwan Park > > > On Jun 4, 2015

Re: "Validate" (commons) versus "checkArgument" (guava)

Sure, will add this to the list. On Wed, Jun 3, 2015 at 8:20 AM, Ufuk Celebi wrote: > Can you please also mark everything in the > > org.apache.flink.shaded.* > > namespace as illegal? > > On 03 Jun 2015, at 17:08, Lokesh Rajaram wrote: > > > Awesome. Will create a JIRA and assign it to me. > >

Re: Build failure with maven-junction-plugin

The WARNING is okay. How did you call maven, and from which directory (flink root or flink-dist?) On Wed, Jun 3, 2015 at 5:10 PM, Theodore Vasiloudis < theodoros.vasilou...@gmail.com> wrote: > Hello, > > I'm having some problems with building the project. The error I get is for > the flink-dist

Re: How do we want to maintain our documentation?

+1 Good. PR should contain documentation. Regards, Chiwan Park > On Jun 4, 2015, at 12:24 AM, Lokesh Rajaram wrote: > > +1. I like this idea, not sure if my vote counts :) > > On Wed, Jun 3, 2015 at 8:21 AM, Robert Metzger wrote: > >> Hi, >> >> as part of making our codebase ready for the u

Re: How do we want to maintain our documentation?

+1. I like this idea, not sure if my vote counts :) On Wed, Jun 3, 2015 at 8:21 AM, Robert Metzger wrote: > Hi, > > as part of making our codebase ready for the upcoming 0.9 release, I've > started to go over the documentation of Flink. > > It seems that our current approach for documenting stuf

Re: Build failure with maven-junction-plugin

mvn clean package -Dmaven.javadoc.skip=true -DskipTests Called from the flink root. On Wed, Jun 3, 2015 at 5:21 PM, Robert Metzger wrote: > The WARNING is okay. > > How did you call maven, and from which directory (flink root or > flink-dist?) > > On Wed, Jun 3, 2015 at 5:10 PM, Theodore Vasilo

How do we want to maintain our documentation?

Hi, as part of making our codebase ready for the upcoming 0.9 release, I've started to go over the documentation of Flink. It seems that our current approach for documenting stuff: - We implement and merge a feature - We open a JIRA for documenting it. Before the release, we realize that we have

Re: "Validate" (commons) versus "checkArgument" (guava)

Can you please also mark everything in the org.apache.flink.shaded.* namespace as illegal? On 03 Jun 2015, at 17:08, Lokesh Rajaram wrote: > Awesome. Will create a JIRA and assign it to me. > > Thanks, > Lokesh > > On Wed, Jun 3, 2015 at 8:03 AM, Robert Metzger wrote: > >> Hi, >> >> the i

JIRA guidelines

Hello All, Are there any guidelines followed in Flink community while raising a JIRA Issue? First time creating a JIRA here, your pointers will be very helpful to me. Thanks, Lokesh

Re: Null pointer exception on pl.project13.maven:git-commit-id-plugin:2.1.5:revision

But the version upgrade itself didn't fix the issue? On Wed, Jun 3, 2015 at 5:12 PM, F. Beligianni wrote: > Thank you Robert, > > I set useNativeGit to true just in flink-runtime package and the build was > successful :) > > Just a comment I had to change also the version of the git-commit-id-pl

Re: Null pointer exception on pl.project13.maven:git-commit-id-plugin:2.1.5:revision

Thank you Robert, I set useNativeGit to true just in flink-runtime package and the build was successful :) Just a comment I had to change also the version of the git-commit-id-plugin to 2.1.10 which is the first version that has this configuration flag. The version in the flink-runtime pom file w

Build failure with maven-junction-plugin

Hello, I'm having some problems with building the project. The error I get is for the flink-dist module is: [ERROR] Failed to execute goal com.pyx4j:maven-junction-plugin:1.0.3:link > (default) on project flink-dist: Can't create junction source > [/long/dir/here//flink-dist/target/flink-0.9-SNAP

Re: "Validate" (commons) versus "checkArgument" (guava)

Awesome. Will create a JIRA and assign it to me. Thanks, Lokesh On Wed, Jun 3, 2015 at 8:03 AM, Robert Metzger wrote: > Hi, > > the illegalImport check is exactly what I had in mind. Would be great if > you could add such a check. > You can also create JIRA issues yourself. > > On Wed, Jun 3, 2

Re: checkstyle failure

Hello Marton, Nice find. can I work on this JIRA? Thanks, Lokesh On Wed, Jun 3, 2015 at 7:48 AM, Márton Balassi wrote: > Actually it is really important that you are using a Mac, it look like: > > Spark had the same issue with hbase annotations. [1] > I have added a JIRA for Flink, should be e

Re: "Validate" (commons) versus "checkArgument" (guava)

Hi, the illegalImport check is exactly what I had in mind. Would be great if you could add such a check. You can also create JIRA issues yourself. On Wed, Jun 3, 2015 at 6:39 AM, Lokesh Rajaram wrote: > Hello Ufuk, Robert, > > @Ufuk if you can create a ticket and assign it to me that would be v

Re: checkstyle failure

Sure, feel free to take it. :) On Wed, Jun 3, 2015 at 5:00 PM, Lokesh Rajaram wrote: > Hello Marton, > > Nice find. can I work on this JIRA? > > Thanks, > Lokesh > > On Wed, Jun 3, 2015 at 7:48 AM, Márton Balassi > wrote: > > > Actually it is really important that you are using a Mac, it look l

Re: Planning the 0.9 Release

What is the status of the 0.9 release planning. It seems like many of the open issues from the document have been closed. When do you think are we able to fork off the "release-0.9" branch and create the first RC ? On Wed, May 13, 2015 at 2:17 PM, Robert Metzger wrote: > +1 for cutting a releas

Re: checkstyle failure

Thanks Robert. It works with the recommended changes. On Wed, Jun 3, 2015 at 7:40 AM, Robert Metzger wrote: > Hi Lokesh, > > can you try this fix? > > http://dbknickerbocker.blogspot.de/2013/04/simple-fix-to-missing-toolsjar-in-jdk.html > > On Wed, Jun 3, 2015 at 4:27 PM, Lokesh Rajaram > wrote

Re: GSoC work

Hi Sachin, great idea to keep a blog! Thanks a lot for sharing :)) -V. On 3 June 2015 at 16:41, Sachin Goel wrote: > Hi everyone > I'm maintaining a blog detailing my work here: > https://tolkienaboutcode.wordpress.com/ > I've already made two posts about my work so far. Any feedback would be

Re: GSoC work

Cool. Thank you for starting this blog. I'll check it from time to time. It is advised by google for students to have a blog, because it makes it easier to understand what the students are working on. Its great to see that you are following the advise. On Wed, Jun 3, 2015 at 4:41 PM, Sachin Goel

Re: Null pointer exception on pl.project13.maven:git-commit-id-plugin:2.1.5:revision

It seems to be an issue of the eclipse egit module: https://bugs.eclipse.org/bugs/show_bug.cgi?id=351855, https://www.eclipse.org/forums/index.php/t/361344/, https://netbeans.org/bugzilla/show_bug.cgi?id=243088 >From reading the source code [1] it seems that the jgit tool is trying to resolve a c

Re: checkstyle failure

Actually it is really important that you are using a Mac, it look like: Spark had the same issue with hbase annotations. [1] I have added a JIRA for Flink, should be easy as Spark had a PR for it. [2, 3] [1] https://issues.apache.org/jira/browse/SPARK-4455 [2] https://issues.apache.org/jira/brows

[jira] [Created] (FLINK-2153) Exclude dependency on hbase annotations module

Márton Balassi created FLINK-2153: - Summary: Exclude dependency on hbase annotations module Key: FLINK-2153 URL: https://issues.apache.org/jira/browse/FLINK-2153 Project: Flink Issue Type: Bu

GSoC work

Hi everyone I'm maintaining a blog detailing my work here: https://tolkienaboutcode.wordpress.com/ I've already made two posts about my work so far. Any feedback would be welcome. :) The relevant JIRA issues are Flink-1727 and Flink-2131

Re: checkstyle failure

Hi Lokesh, can you try this fix? http://dbknickerbocker.blogspot.de/2013/04/simple-fix-to-missing-toolsjar-in-jdk.html On Wed, Jun 3, 2015 at 4:27 PM, Lokesh Rajaram wrote: > Hi, > > When I ran mvn checkstyle:checkstyle with the latest master branch. I see > the following failure: > > [ERROR] F

checkstyle failure

Hi, When I ran mvn checkstyle:checkstyle with the latest master branch. I see the following failure: [ERROR] Failed to execute goal on project flink-hbase: Could not resolve dependencies for project org.apache.flink:flink-hbase:jar:0.9-SNAPSHOT: Could not find artifact jdk.tools:jdk.tools:jar:1.7

Re: Null pointer exception on pl.project13.maven:git-commit-id-plugin:2.1.5:revision

@Robert No, I couldn't. 2015-06-03 16:21 GMT+02:00 Szabó Péter : > I have the same problem on one of my computers but not on the other one. > I've read somewhere that it can be related to the git > version/configuration. I'm not completely sure, though. > > Peter > > 2015-06-03 15:46 GMT+02:00 F.

Re: Null pointer exception on pl.project13.maven:git-commit-id-plugin:2.1.5:revision

I have the same problem on one of my computers but not on the other one. I've read somewhere that it can be related to the git version/configuration. I'm not completely sure, though. Peter 2015-06-03 15:46 GMT+02:00 F. Beligianni : > Hello, > > I have a problem to build Flink master branch. Spec

Re: Null pointer exception on pl.project13.maven:git-commit-id-plugin:2.1.5:revision

I think Péter had a similar issue recently. Did you resolve it? I'll look into it now ... On Wed, Jun 3, 2015 at 3:46 PM, F. Beligianni wrote: > Hello, > > I have a problem to build Flink master branch. Specifically, when I am > running "mvn clean install" in master branch I take the following

Re: Gelly Roadmap

Hey, yes, this is kind of an ambitious and long-term roadmap :) As Andra said, tasks 1-2-3-4 are already work in progress. Regarding the rest, we could discuss on the priority or leave it on the personal interest of contributors. Personally, I think bipartite support should be quite easy to add

[jira] [Created] (FLINK-2152) Provide zipWithIndex utility in flink-contrib

Robert Metzger created FLINK-2152: - Summary: Provide zipWithIndex utility in flink-contrib Key: FLINK-2152 URL: https://issues.apache.org/jira/browse/FLINK-2152 Project: Flink Issue Type: Imp

Null pointer exception on pl.project13.maven:git-commit-id-plugin:2.1.5:revision

Hello, I have a problem to build Flink master branch. Specifically, when I am running "mvn clean install" in master branch I take the following exception: [ERROR] Failed to execute goal pl.project13.maven:git-commit-id-plugin:2.1.5:revision (default) on project flink-runtime: Execution default o

Fwd: Discussion: Storm Comparability Layer

-- Forwarded message -- From: Szabó Péter Date: 2015-06-03 15:31 GMT+02:00 Subject: Re: Discussion: Storm Comparability Layer To: Márton Balassi Hey, Matthias, Of course, you can remove my last commit. I just wanted to remove the failing tests, and some unnecessary comments. Pl

[jira] [Created] (FLINK-2151) Provide interface to distinguish close() calls in error and regular cases

Robert Metzger created FLINK-2151: - Summary: Provide interface to distinguish close() calls in error and regular cases Key: FLINK-2151 URL: https://issues.apache.org/jira/browse/FLINK-2151 Project: Fl

Re: Gelly Roadmap

Hi Stephan, Well the operators fro skewed graphs and the Gelly streaming project will be done by the end of the semester. Otherwise Daniel and I won't graduate :); The Scala API is ongoing work (there is a link to the corresponding JIRA in the roadmap); The Library methods are in the making; First

Re: Gelly Roadmap

Wow, this is an extremely big and ambitious roadmap. Are all these things realistically possible in the foreseeable future? Or is this more a wish-list? Do you have an idea what issues to approach in what order? On Wed, May 20, 2015 at 8:58 PM, Andra Lungu wrote: > The Roadmap is now available

Re: [DISCUSS] State Management API for Streaming

Hi! Sorry for not responding earlier. Thanks for taking an initiative there and drafting this. I would like to have a close look after the 0.9 release is out. I am in favor of making this a major item for 0.10, to get the state with all its characteristics defined in a principles fashion: repart

Re: Send events to parallel operator instances

I am talking of course about global delta windows. On the full stream not on a partition. Delta windows per partition happens as you said currently as well. On Wednesday, June 3, 2015, Aljoscha Krettek wrote: > Yes, this is obvious, but if we simply partition the data on the > attribute that we

Re: Send events to parallel operator instances

Yes, this is obvious, but if we simply partition the data on the attribute that we use for the delta policy this can be done purely on one machine. No need for complex communication/synchronization. On Wed, Jun 3, 2015 at 1:32 PM, Gyula Fóra wrote: > Yes, we define a delta function from the first

Re: Send events to parallel operator instances

Yes, we define a delta function from the first element to the last element in a window. Now let's discretize the stream using this semantics in parallel. Aljoscha Krettek ezt írta (időpont: 2015. jún. 3., Sze, 12:20): > Ah ok. And by distributed you mean that the element that starts the > window

[jira] [Created] (FLINK-2150) Add a library method that assigns unique Long values to vertices

Vasia Kalavri created FLINK-2150: Summary: Add a library method that assigns unique Long values to vertices Key: FLINK-2150 URL: https://issues.apache.org/jira/browse/FLINK-2150 Project: Flink

[jira] [Created] (FLINK-2149) Simplify Gelly Jaccard similarity example

Vasia Kalavri created FLINK-2149: Summary: Simplify Gelly Jaccard similarity example Key: FLINK-2149 URL: https://issues.apache.org/jira/browse/FLINK-2149 Project: Flink Issue Type: Improveme

Re: Send events to parallel operator instances

Ah ok. And by distributed you mean that the element that starts the window can be processed on a different machine than the element that finishes the window? On Wed, Jun 3, 2015 at 12:11 PM, Gyula Fóra wrote: > This is not connected to the current implementation. So lets not talk about > that. >

[jira] [Created] (FLINK-2148) Approximately calculate the number of distinct elements of a stream

Gabor Gevay created FLINK-2148: -- Summary: Approximately calculate the number of distinct elements of a stream Key: FLINK-2148 URL: https://issues.apache.org/jira/browse/FLINK-2148 Project: Flink

Re: Send events to parallel operator instances

This is not connected to the current implementation. So lets not talk about that. This is about theoretical limits to support distributed delta policies which has far reaching implications for the windowing policies one can implement in a prallel way. But you are welcome to throw in any construct

[jira] [Created] (FLINK-2147) Approximate calculation of frequencies in data streams

Gabor Gevay created FLINK-2147: -- Summary: Approximate calculation of frequencies in data streams Key: FLINK-2147 URL: https://issues.apache.org/jira/browse/FLINK-2147 Project: Flink Issue Type:

[jira] [Created] (FLINK-2146) Fast calculation of min/max with arbitrary eviction and triggers

Gabor Gevay created FLINK-2146: -- Summary: Fast calculation of min/max with arbitrary eviction and triggers Key: FLINK-2146 URL: https://issues.apache.org/jira/browse/FLINK-2146 Project: Flink I

[jira] [Created] (FLINK-2145) Median calculation for windows

Gabor Gevay created FLINK-2145: -- Summary: Median calculation for windows Key: FLINK-2145 URL: https://issues.apache.org/jira/browse/FLINK-2145 Project: Flink Issue Type: Sub-task Compo

[jira] [Created] (FLINK-2144) Implement count, average, and variance for windows

Gabor Gevay created FLINK-2144: -- Summary: Implement count, average, and variance for windows Key: FLINK-2144 URL: https://issues.apache.org/jira/browse/FLINK-2144 Project: Flink Issue Type: Sub-

Re: Send events to parallel operator instances

Part of the reason for my question is this: https://issues.apache.org/jira/browse/FLINK-1967. Especially my latest comment there. If we want this, I think we have to overhaul the windowing system anyways and then it doesn't make sense to explore complicated workarounds for the current system. On W

[jira] [Created] (FLINK-2143) Add an overload to reduceWindow which takes the inverse of the reduceFunction as a second parameter

Gabor Gevay created FLINK-2143: -- Summary: Add an overload to reduceWindow which takes the inverse of the reduceFunction as a second parameter Key: FLINK-2143 URL: https://issues.apache.org/jira/browse/FLINK-2143

Re: Gelly Blog Post

Hi all, thank you so much for your comments! Most of them have been addressed already. I would also like to add Fabian's suggestion on providing a library method for initializing vertices with unique IDs and we'd also like to run some experiments with the Echo Nest dataset. After that, I think i

[jira] [Created] (FLINK-2142) GSoC project: Exact and Approximate Statistics for Data Streams and Windows

Gabor Gevay created FLINK-2142: -- Summary: GSoC project: Exact and Approximate Statistics for Data Streams and Windows Key: FLINK-2142 URL: https://issues.apache.org/jira/browse/FLINK-2142 Project: Flink

RE: Buffer re-ordering problem

I am currently using 0.9-SNAPSHOT. All the non-jar files are from an older build, but I recently manually updated the flink-dist.jar with commit d163a817fa2e330e86384d0bbcd104f051a6fb48. Our setup consists of 10 workers and a master, all interconnected via a switch (100 Mbit, I think). The data

Re: Send events to parallel operator instances

There are simple ways of implementing it in a non-distributed or inconsistent fashion. On Wed, Jun 3, 2015 at 8:55 AM Aljoscha Krettek wrote: > This already sounds awfully complicated. Is there no other way to > implement the delta windows? > > On Wed, Jun 3, 2015 at 7:52 AM, Gyula Fóra wrote: >

[jira] [Created] (FLINK-2141) Allow GSA's Gather to perform this operation in more than one direction

Andra Lungu created FLINK-2141: -- Summary: Allow GSA's Gather to perform this operation in more than one direction Key: FLINK-2141 URL: https://issues.apache.org/jira/browse/FLINK-2141 Project: Flink

Re: Buffer re-ordering problem

This is a critical bug. - which version are you using? If snapshot, which commit? - what is your setup? Number of machines, datset etc? - is it reproducible? On Wednesday, June 3, 2015, Kruse, Sebastian wrote: > Hi everyone, > > I had some jobs running over the night and in two of them after ab

[jira] [Created] (FLINK-2140) Access the number of vertices from within the GSA functions

Andra Lungu created FLINK-2140: -- Summary: Access the number of vertices from within the GSA functions Key: FLINK-2140 URL: https://issues.apache.org/jira/browse/FLINK-2140 Project: Flink Issue

Buffer re-ordering problem

Hi everyone, I had some jobs running over the night and in two of them after about half an hour the following exception occurred. Do you know why this happens? Thanks, Sebastian tenem16.hpi.uni-potsdam.de Error: java.lang.Exception: The data preparation for task 'CHAIN GroupReduce (GroupReduce

Re: Storm compatibility layer currently does not support Storm's SimpleJoin example

It looks like there is a now a PR request available for the storm compatibility: https://github.com/apache/flink/pull/764 It seems were are not the only new stream processing system with compatibility to Storm: http://dl.acm.org/citation.cfm?id=2742788 On Tue, Jun 2, 2015 at 11:09 AM, Szabó Péter