Re: SQL on Flink

2015-05-27 Thread Ufuk Celebi
On 27 May 2015, at 17:05, Timo Walther wrote: > It's rather passion for the future of the project than passion for SQL ;-) > > I always try to think like someone from the economy. And IMO the guys from > economy are still thinking in SQL. If you want to persuade someone coming > from the SQL

Re: Get the current TM hostname from the ExecutionContext

2015-05-29 Thread Ufuk Celebi
Hey Alexander, no, it's not possible at the moment. +1 for the idea. I also needed it once. I think you will have to look at RuntimeEnvironment, Task, RuntimeContext. – Ufuk

Re: [DISCUSS] Streaming Sources (again)

2015-05-31 Thread Ufuk Celebi
+1 to the locking interface for the release. I agree with what Marton that it seems that interruptability is to much of a burden on the sources. The code docs should be very clear (and concise (!)) about why the locking is needed etc. – Ufuk On 31 May 2015, at 14:52, Gyula Fóra wrote: > Alri

Re: [DISCUSS] Inconsistent naming of intermediate results

2015-06-01 Thread Ufuk Celebi
ition => ResultPartition > > > > Not sure about these. Maybe we should change them to ExecutionResult and > ExecutionResultPartition because that's more specific and would relate to > the other class names. > > On Wed, Apr 1, 2015 at 10:39 AM, Ufuk Celebi wrote: >

Re: ResendUnfulfillableException at the end of failing job

2015-06-01 Thread Ufuk Celebi
Thanks for posting this. From what I've found online, this seems to be a problem related to Akka. For example there is this PR to fix this: https://github.com/akka/akka/issues/16623 Can you check whether this should have been fixed in the Akka version we are using and post to the Akka ML if so?

Re: Travis build failing

2015-06-02 Thread Ufuk Celebi
I'm looking into the last one already. On Tuesday, June 2, 2015, Sachin Goel wrote: > Ah. This is a known issue then. > > Regards > Sachin Goel > > On Tue, Jun 2, 2015 at 9:14 PM, Aljoscha Krettek > > wrote: > > > The last one is actually SuccessAfterNetworkBuffersFailureITCase which > > uses C

Re: Memory management overhaul

2015-06-02 Thread Ufuk Celebi
On 02 Jun 2015, at 20:12, Alexander Alexandrov wrote: > During an offline chat some time ago Stephan Ewen mentioned that there is > an ongoing effort for a dynamic memory allocation in some feature branch > lying around. Can you point me to that, as I would like to look at the > code? Thanks.

Re: "Validate" (commons) versus "checkArgument" (guava)

2015-06-02 Thread Ufuk Celebi
On 02 Jun 2015, at 21:18, Lokesh Rajaram wrote: > Hello Robert, > > I worked on that issue, if it's ok I can take this task. > > Btw, how is anything enforced in Flink? Do I have to update how to contribute > guide or any thing else need to be done? The how to contribute guide is a good star

Re: Send events to parallel operator instances

2015-06-02 Thread Ufuk Celebi
On 02 Jun 2015, at 22:45, Gyula Fóra wrote: > I am wondering, what is the suggested way to send some events directly to > another parallel instance in a flink job? For example from one mapper to > another mapper (of the same operator). > > Do we have any internal support for this? The first thin

Re: Buffer re-ordering problem

2015-06-03 Thread Ufuk Celebi
This is a critical bug. - which version are you using? If snapshot, which commit? - what is your setup? Number of machines, datset etc? - is it reproducible? On Wednesday, June 3, 2015, Kruse, Sebastian wrote: > Hi everyone, > > I had some jobs running over the night and in two of them after ab

Re: "Validate" (commons) versus "checkArgument" (guava)

2015-06-03 Thread Ufuk Celebi
Can you please also mark everything in the org.apache.flink.shaded.* namespace as illegal? On 03 Jun 2015, at 17:08, Lokesh Rajaram wrote: > Awesome. Will create a JIRA and assign it to me. > > Thanks, > Lokesh > > On Wed, Jun 3, 2015 at 8:03 AM, Robert Metzger wrote: > >> Hi, >> >> the i

Re: JIRA guidelines

2015-06-03 Thread Ufuk Celebi
On 03 Jun 2015, at 18:06, Till Rohrmann wrote: > Hi Lokesh, > > there are no real guidelines for opening JIRA issues. You should select the > right project to which assign the JIRA, namely Flink. Then choose an > appropriate type, e.g. bug, improvement or feature. Give a descriptive > summary o

Re: Buffer re-ordering problem

2015-06-03 Thread Ufuk Celebi
> org.apache.flink.runtime.io.network.api.reader.AbstractRecordReader.getNextRecord(AbstractRecordReader.java:64) > at > org.apache.flink.runtime.io.network.api.reader.MutableRecordReader.next(MutableRecordReader.java:34) > at > org.apache.flink.runtime.operators.util.ReaderIterator.nex

Re: Gelly Roadmap

2015-06-03 Thread Ufuk Celebi
On 03 Jun 2015, at 23:51, Stephan Ewen wrote: > It may make sense to link the JIRAs from the roadmap, or add comments what > people are working on and what is in the queue. Linking JIRAs from the roadmap would solve all these points. ;) So +1 for issue links.

Failing tests policy

2015-06-03 Thread Ufuk Celebi
Hey all, we have certain test cases, which are failing regularly on Travis. In all cases I can think of we just keep the test activated. I think this makes it very hard for regular contributors to take these failures seriously. I think the following situation is not unrealistic with the current p

Re: ALS implementation

2015-06-04 Thread Ufuk Celebi
I think both are bugs. They are triggered by the different memory configurations. @chiwan: is the 2nd error fixed by your recent change? @felix: if yes, can you try the 2nd run again with the changes? On Thursday, June 4, 2015, Felix Neutatz wrote: > Hi, > > I played a bit with the ALS recomme

Re: Failing tests policy

2015-06-04 Thread Ufuk Celebi
phan Ewen wrote: >>>> @matthias: That is the implicit policy right now. Seems not to work... >>>> >>>> On Thu, Jun 4, 2015 at 12:40 AM, Matthias J. Sax < >>>> mj...@informatik.hu-berlin.de> wrote: >>>> >>>>> I basically agree

Re: Send events to parallel operator instances

2015-06-04 Thread Ufuk Celebi
On 04 Jun 2015, at 12:46, Stephan Ewen wrote: > There is no "lateral communication" right now. Typical pattern is to break > it up in two operators that communicate in an all-to-all fashion. You can look at the iteration tasks: the iteration sync task is communicating with the iteration heads

Re: [DISCUSS] Inconsistent naming of intermediate results

2015-06-04 Thread Ufuk Celebi
On 04 Jun 2015, at 13:10, Maximilian Michels wrote: > Rename what to streams? Do you mean "ResultPartition" => "StreamPartition"? Exactly along those lines, but maybe "ResultStream". > I'm not sure if that makes it easier to understand what the classes do. It fits better into the terminology

Re: Planning the 0.9 Release

2015-06-04 Thread Ufuk Celebi
On 03 Jun 2015, at 17:00, Robert Metzger wrote: > What is the status of the 0.9 release planning. > > It seems like many of the open issues from the document have been closed. > When do you think are we able to fork off the "release-0.9" branch and > create the first RC ? It would be great to

Re: [DISCUSS] Inconsistent naming of intermediate results

2015-06-04 Thread Ufuk Celebi
On 04 Jun 2015, at 17:02, Maximilian Michels wrote: > I think ResultPartition is a pretty accurate description of what it is: a > partition of the result of an operator. ResultStream on the other hand, > seems very generic to me. Just because we like to think of Flink nowadays > as a "streaming

Re: QA-Bot

2015-06-05 Thread Ufuk Celebi
I didn't see these emails before. I think it needs more love at the moment. Your simple docs change (https://github.com/apache/flink/pull/786) is getting a -1 for various reasons. Is someone going to work on this in the near future? If not, I vote to disable it again. – Ufuk On 05 Jun 2015, a

Re: Planning the 0.9 Release

2015-06-05 Thread Ufuk Celebi
Ping. On 04 Jun 2015, at 14:11, Ufuk Celebi wrote: > Critical issues: > > - Skipped buffer > (http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html) > (I'm on it) > - Execution graph deadlock (FLINK-2133) I'm inve

Re: Planning the 0.9 Release

2015-06-05 Thread Ufuk Celebi
On 05 Jun 2015, at 11:46, Robert Metzger wrote: > I'll address the remaining documentation issues today. Thank you so much for doing this. > What about > Sync Streaming Java/Scala API > - Consolidate names across batch/streaming (discussion) > - Merge static code analysis > > They seem

Re: Planning the 0.9 Release

2015-06-07 Thread Ufuk Celebi
On 08 Jun 2015, at 00:22, Robert Metzger wrote: > What about https://issues.apache.org/jira/browse/FLINK-2177 and > https://issues.apache.org/jira/browse/FLINK-2054 ? They are both marked as > blockers. FLINK-2177 is resolved. Is 2054 ("StreamOperator rework removed copy calls when passing out

Testing Apache Flink 0.9.0-rc1

2015-06-08 Thread Ufuk Celebi
Hey Chiwan! Is the problem reproducible? Does it always deadlock? Can you please wait for it to deadlock and then post a stacktrace (jps and jstack) of the process? Please post it to this issue: FLINK-2183. Thanks :) – Ufuk On Monday, June 8, 2015, Chiwan Park > wrote: > Hi. I have a problem r

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Ufuk Celebi
Hey all, 1. it would be nice if we find more people to also do testing of the streaming API. I think it's especially good to have people on it, which did not use it before. 2. Just to make sure: the "assignee" field of each task is a list, i.e. we can and should have more people testing per ta

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Ufuk Celebi
On 09 Jun 2015, at 13:58, Sachin Goel wrote: > On my local machine, several flink runtime tests are failing on "mvn clean > verify". Here is the log output: http://pastebin.com/raw.php?i=VWbx2ppf Thanks for reporting this. Have you tried it multiple times? Is it failing reproducibly with the s

Re: Force enabling checkpoints for iterative streaming jobs

2015-06-09 Thread Ufuk Celebi
Hey Gyula, I understand your reasoning, but I don't think its worth to rush this into the release. As you've said, we cannot give precise guarantees. But this is arguably one of the key requirements for any fault tolerance mechanism. Therefore I disagree that this is better than not having any

Re: Testing Apache Flink 0.9.0-rc1

2015-06-09 Thread Ufuk Celebi
in part of the Streaming Java API and all of the >>> Streaming Scala API. I think this is a release blocker (in addition >>> the the other bugs found so far.) >>> >>> On Tue, Jun 9, 2015 at 2:35 PM, Aljoscha Krettek >>> wrote: >>>> I found the bu

Re: Testing Apache Flink 0.9.0-rc1

2015-06-10 Thread Ufuk Celebi
Hey Gyula, Max, On 10 Jun 2015, at 10:54, Gyula Fóra wrote: > This feature needs to be included in the release, it has been tested and > used extensively. And many applciations depend on it. It would be nice to announce/discuss this before just cherry-picking it into the release branch. The is

Re: Force enabling checkpoints for iterative streaming jobs

2015-06-10 Thread Ufuk Celebi
On 10 Jun 2015, at 14:29, Gyula Fóra wrote: > Max suggested that I add this feature slightly hidden to the execution > config instance. > > The problem then is that I either make a public field in the config or once > again add a method. > > Any ideas? I thought about this as well. The only wa

Re: Testing Apache Flink 0.9.0-rc1

2015-06-10 Thread Ufuk Celebi
On 10 Jun 2015, at 16:18, Maximilian Michels wrote: > I'm debugging the TaskManagerFailsWithSlotSharingITCase. I've located its > cause but still need to find out how to fix it. Very good find, Max! Max, Till, and I have looked into this and it is a reproducible deadlock in the scheduler duri

Re: built problem - flink 0.9-SNAPSHOT

2015-06-11 Thread Ufuk Celebi
This is on an IBM PowerPC machine, right? Since this MXBeans are from the com.sun.* namespace, I'm not sure if this can be fixed w/o loading the MX beans depending on the JVM. For your JVM, the classes are located in "com.ibm.lang.management.*" and not "com.sun.management.*". On 12 Jun 2015, a

Re: Testing Apache Flink 0.9.0-rc1

2015-06-11 Thread Ufuk Celebi
On 11 Jun 2015, at 20:04, Fabian Hueske wrote: > How about the following issues? > > 1. The Hbase Hadoop Compat issue, Ufuk is working on I was not able to reproduce this :( I ran HadoopInputFormats against various sources and confirmed the results and everything was fine so far. I think I w

Re: built problem - flink 0.9-SNAPSHOT

2015-06-11 Thread Ufuk Celebi
On 12 Jun 2015, at 00:42, Felix Neutatz wrote: > Yes, it is on a IBM PowerPC machine. So we change that in the documentation > to all Java 7,8 ( except IBM Java)? Yes, you can also open an issue to investiage this further.

Re: Testing Apache Flink 0.9.0-rc1

2015-06-11 Thread Ufuk Celebi
On 12 Jun 2015, at 00:49, Fabian Hueske wrote: > 2. is basically done. I have a patch which updates the counters on page > reload but that shouldn't be hard to extend to dynamic updates. Very nice! :-) Thanks!

Re: Testing Apache Flink 0.9.0-rc1

2015-06-12 Thread Ufuk Celebi
On 12 Jun 2015, at 09:45, Till Rohrmann wrote: > Hi guys, > > I just noticed while testing the TableAPI on the cluster that it is not > part of the dist module. Therefore, programs using the TableAPI will only > run when you put the TableAPI jar directly on the cluster or if you build a > fat j

Re: Testing Apache Flink 0.9.0-rc1

2015-06-12 Thread Ufuk Celebi
On 12 Jun 2015, at 10:29, Till Rohrmann wrote: > Well I think the initial idea was to keep the dist jar as small a possible > and therefore we did not include the libraries. I'm not sure whether we can > decide this here ad-hoc. If the community says that we shall include these > libraries then

Re: Testing Apache Flink 0.9.0-rc1

2015-06-12 Thread Ufuk Celebi
On 12 Jun 2015, at 10:44, Till Rohrmann wrote: > Yes you're right Ufuk. At the moment the user has to place the jars in the > lib folder of Flink. If this folder is not shared then he has to do it for > every node on which Flink runs. OK. I guess there is a nice way to do this with YARN as well

Re: Testing Apache Flink 0.9.0-rc1

2015-06-12 Thread Ufuk Celebi
After thinking about it a bit more, I think that's fine. +1 to document and keep it as it is.

Re: Testing Apache Flink 0.9.0-rc1

2015-06-12 Thread Ufuk Celebi
On 12 Jun 2015, at 00:40, Ufuk Celebi wrote: > > On 11 Jun 2015, at 20:04, Fabian Hueske wrote: > >> How about the following issues? >> >> 1. The Hbase Hadoop Compat issue, Ufuk is working on > > I was not able to reproduce this :( I ran HadoopInputForma

Re: Testing Apache Flink 0.9.0-rc1

2015-06-12 Thread Ufuk Celebi
@Max: for the new RC. Can you make sure to set the variables correctly with regard to stable/snapshot versions in the docs?

Re: Testing Apache Flink 0.9.0-rc1

2015-06-12 Thread Ufuk Celebi
change] Since this is not a vote yet, it doesn't really matter, but I'm leaning towards b). On Fri, Jun 12, 2015 at 11:43 AM, Till Rohrmann wrote: > What about the shaded jars? > > On Fri, Jun 12, 2015 at 11:32 AM Ufuk Celebi wrote: > > > @Max: for the new RC.

Re: Testing Apache Flink 0.9.0-rc1

2015-06-12 Thread Ufuk Celebi
I'm with Till on this. Robert's position is valid as well. Again, there is no core disagreement here. No one wants to add it to dist. On 12 Jun 2015, at 00:40, Ufuk Celebi > wrote: On 11 Jun 2015, at 20:04, Fabian Hueske > wrote: How about the following issues? 1. The Hba

Re: Listing Apache-2.0 dependencies in LICENSE file

2015-06-15 Thread Ufuk Celebi
To summarize: 1. Your PR changes are necessary. Thanks for doing it. 2. The consensus (PR comments + ML) is to skip other Apache licensed dependencies. 3. Shaded Jars need LICENSE and NOTICE in META-INF. Let's wrap this up today and get it out of the way of the release. :-) – Ufuk On 15 Jun

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Ufuk Celebi
On Mon, Jun 15, 2015 at 3:30 PM, Aljoscha Krettek wrote: > Regarding 1), thats why I said "bugs and features". :D But I think of it as > a bug, since people will normally set in in the flink-conf.yaml on the > master and assume that it works. That's what I assumed and it took me a > while to figu

Re: Testing Apache Flink 0.9.0-rc2

2015-06-15 Thread Ufuk Celebi
Please continue the discussion in the issue Aljoscha opened: https://issues.apache.org/jira/browse/FLINK-2221 I think it is better to only point to issues in this mail thread. Otherwise the discussions are very hard to follow.

Re: Iteration stats logging

2015-06-15 Thread Ufuk Celebi
Hey Tran Nam-Luc, there is currently no way to do this. The iteration sync tasks keeps track of iteration convergence/max number of iterations and signals termination to the iteration head. After this, the head flushes the produced result to the next task (after the iteration) and the intermed

Re: [VOTE] Release Apache Flink 0.9.0 (release-0.9.0-rc3)

2015-06-17 Thread Ufuk Celebi
On 17 Jun 2015, at 18:05, Aljoscha Krettek wrote: > -1 > > There is a bug in the newly introduced Null-Value support in RowSerializer: > The serializer was changed to write booleans that signify if a field is > null. For comparison this still uses the TupleComparatorBase (via > CaseClassCompara

Quickstart POMs

2015-06-18 Thread Ufuk Celebi
Hey all, looking at the quickstarts for the upcoming release, I am a little confused: We use the shade plugin to build a fat jar and exclude some of Flink's dependencies (actual and transitive). Then we have a build profile (build-jar), which sets the Flink dependencies to provided. 1. Is ther

Re: [VOTE] Release Apache Flink 0.9.0 (release-0.9.0-rc4)

2015-06-18 Thread Ufuk Celebi
+1 Overall, I'm +1, but I'm wondering whether the quickstart poms need a clean up (I've started a discussion about this). My main concern is that they might confuse new users (two ways of doing essentially the same thing). Verified: # Legal - Check if checksums and GPG files match the corresp

Re: Quickstart POMs

2015-06-18 Thread Ufuk Celebi
On 18 Jun 2015, at 16:49, Fabian Hueske wrote: > I don't think that many users care about the internals of the quickstart > pom file and are just happy if it works. I agree with this. That's exactly why I think it's better to keep it clean; with the current pom you actually do have to worry ab

Re: Quickstart POMs

2015-06-18 Thread Ufuk Celebi
On 18 Jun 2015, at 16:58, Fabian Hueske wrote: > Why? > > mvn package > > builds the program correctly, no? Yes, but: - Dependencies not specified by the user may be included (Metrics, javaassist) - Dependencies specified by the user may be excluded - If you use the build-jar profile you hav

Re: Quickstart POMs

2015-06-18 Thread Ufuk Celebi
On Thursday, June 18, 2015, Robert Metzger wrote: > Hi, > > I'm against cancelling a release for something that is not nice ;) It has > to be at least broken to cancel :) See vote thread +1. This was not about canceling but clearification. ;) > I agree that the pom looks complicated and I wou

Re: Quickstart POMs

2015-06-19 Thread Ufuk Celebi
On 19 Jun 2015, at 10:19, Maximilian Michels wrote: > @Robert @Stephan Thanks for clarifying! Of course it would be better to > have a concise quickstart pom.xml but the necessary workarounds justify the > current state. +1

Re: Failing Builds on Travis

2015-06-19 Thread Ufuk Celebi
On 19 Jun 2015, at 10:29, Maximilian Michels wrote: > Bringing this up again because of a recently discovered issue: > https://issues.apache.org/jira/browse/FLINK-2235 > > Flink 0.9.0 will still support Java 6. Just wanted to clarify again that we > drop support for Java 6 for any further major

Reduce combiner not chained

2015-06-19 Thread Ufuk Celebi
Hey all, on the current master running the WordCount example with a text file input/output results and a manual reduce function (instead of the sum(1)) results in a combiner, which is not chained. The corresponding issue is here: https://issues.apache.org/jira/browse/FLINK-2246 Can someone pl

Re: Reduce combiner not chained

2015-06-19 Thread Ufuk Celebi
the JIRA accordingly. > > 2015-06-19 13:04 GMT+02:00 Ufuk Celebi : > >> Hey all, >> >> on the current master running the WordCount example with a text file >> input/output results and a manual reduce function (instead of the sum(1)) >> results in a combiner, whic

Re: Flink Runtime Exception

2015-06-19 Thread Ufuk Celebi
On 19 Jun 2015, at 14:53, Andra Lungu wrote: > Another problem that I encountered during the same set of experiments(sorry > if I am asking too many questions, I am eager to get things fixed): > - for the same configuration, a piece of code runs perfectly on 10GB of > input, then for 38GB it runs

Re: [jira] [Created] (FLINK-2252) Reduce local input split assignment log level from INFO to DEBUG

2015-06-20 Thread Ufuk Celebi
Recently I was running a job and the only way to tell how far it was and to make sure that it was still running were the JM logs with the current split assignments. I like Fabians suggestion. But as long as we don't have good runtime monitoring in place there is a valid use case for the logs (whic

Re: Known minor streaming issue in 0.9.0

2015-06-22 Thread Ufuk Celebi
On 22 Jun 2015, at 14:00, Maximilian Michels wrote: > Hi Marton, > > Thanks for spotting this issue. It is a bug we should list under known > issues for the upcoming release. I agree with you that we can live with it > until the next bug fix release. > > Cheers, > Max It's OK to do a bugfix r

Re: Error while deserializing event

2015-06-23 Thread Ufuk Celebi
Hey Tran Nam-Luc, You don't have to register with a serializer. Can you share the event code? I will look into it asap. The runtime is buffer oriented and events arrive as buffers before they are deserialized. That's why you see the getNextBuffer call in the stack trace. – Ufuk On Tuesday, June

Re: New contributor

2015-06-23 Thread Ufuk Celebi
Hey nuno! Welcome to the Flink community. :) The points you mentioned sound very reasonable. There is also a how to contribute guide and a coding guidelines document on the web page you can check out. Is there a specific starter issue you are interested in? Then it will be easier to give pointers t

Re: Provide Hadoop pre-build Hadoop 2.4 and Hadoop 2.6 binaries

2015-06-24 Thread Ufuk Celebi
I think this is a very good idea and very urgent (because of the issues you outlined and for the user experience of *not* having to compile your own version). Big +1. On 24 Jun 2015, at 11:45, Robert Metzger wrote: > Hi, > > I am aware of at least two Flink users which were facing various iss

Re: Provide Hadoop pre-build Hadoop 2.4 and Hadoop 2.6 binaries

2015-06-25 Thread Ufuk Celebi
@Robert, Ma: can one of you start the vote today? Anyone who is against this can give a -1 in the vote thread. ;) – Ufuk On 25 Jun 2015, at 10:24, Maximilian Michels wrote: > +1 for different Hadoop bundles. Other projects do it as well. > > On Wed, Jun 24, 2015 at 2:25 PM, Vyacheslav Zholude

Re: Thoughts About Streaming

2015-06-25 Thread Ufuk Celebi
Thanks for writing this up and comparing to the current implementation. It's great to see that your mockup indicates correct/expected behaviour *and* better performance. :-) Regarding the results for the current mechanism: does this problem affects all window operators? – Ufuk On 25 Jun 2015,

Re: [VOTE] Release additional convenience binaries for Flink 0.9.0

2015-06-25 Thread Ufuk Celebi
+1 - Checked hashes and signatures - Tested local mode for all versions - Tested Flink on YARN 2.4 with https://github.com/aljoscha/FliRTT and built-in data for all versions – Ufuk

Re: [DISCUSS / VOTE] Signal name to "kill" streaming jobs

2015-06-30 Thread Ufuk Celebi
On 30 Jun 2015, at 10:26, Fabian Hueske wrote: > Hi, > > thanks for starting this thread! > > I prefer "stop" over "terminate" because it sounds more graceful. A > temporary stop could be called "pause". +1 to stop. – Ufuk

Re: Flink 0.9 built with Scala 2.11

2015-07-01 Thread Ufuk Celebi
On 01 Jul 2015, at 10:34, Stephan Ewen wrote: > +1, like that approach +1 I like that this is not breaking for non-Scala users :-)

Re: Replacing Checkpointed interface with field annotations

2015-07-01 Thread Ufuk Celebi
On 01 Jul 2015, at 10:57, Gyula Fóra wrote: > Hey, > > Thanks for the feedback guys: > > @Max: You are right, this is not top priority to changes, I was just > mocking up some alternatives to try to make the state usage even simpler so > that the user can keep his current implementations and j

Re: Redesigned "Features" page

2015-07-08 Thread Ufuk Celebi
+1 to publish now. On Wednesday, July 8, 2015, Maximilian Michels wrote: > +1 for updating it. It is a great improvement and we can still change > details subsequently. > > On Wed, Jul 8, 2015 at 11:32 AM, Aljoscha Krettek > > wrote: > > > I would publish it, it is definitely better than the ol

Re: How do network transmissions in Flink work?

2015-07-13 Thread Ufuk Celebi
Hey Niklas, there is also this Wiki entry: https://cwiki.apache.org/confluence/display/FLINK/Data+exchange+between+tasks On 09 Jul 2015, at 21:32, Niklas Semmler wrote: > 1. What does the number of ResultSubpartition instances in the > ResultPartition correspond to? Is one assigned to each co

Re: Student looking to contribute to Stratosphere

2015-07-15 Thread Ufuk Celebi
Hey Rohit, it's best to do the discussion related to a specific issue *in* the issue itself instead of the mailing list. In general, it's better to ask specific questions. But a general pointer would be to look into the existing ML algorithm implementations, Stephan's approximate PageRank impleme

Re: Failing Test

2015-07-16 Thread Ufuk Celebi
Hey, this has been merged yesterday. I guess it's a timing issue when verifying the results. Can you file an issue for this? – Ufuk On 16 Jul 2015, at 11:30, Matthias J. Sax wrote: > Hi, > > I hit another failing test (that is new to me): > >> Results : >> Failed tests: >> AccumulatorLiveIT

Re: Flink contributor list

2015-07-24 Thread Ufuk Celebi
of course, if you spot a mistake (duplicate or your name missing) > please let me know! > > Kostas > > -- The list -- > > Daniel Warneke > Fabian Hueske > Stephan Ewen > Arvid Heise > Robert Metzger > Aljoscha Krettek > Gyula Fora > Till Rohrm

Bug fix release 0.9.1

2015-07-31 Thread Ufuk Celebi
Hey all! I want to start a discussion about the next 0.9 release. Since 0.9.0 there have been 19 commits addressing 16 issues. Of these 16 issues, two were critical issues regarding the runtime, which require a new release urgently. What's do you think about this? --- If we agree about doing

Re: Failing test in Gelly

2015-08-09 Thread Ufuk Celebi
PS what about the yarn test case... Is that one known (with that trace)? On Sunday, August 9, 2015, Ufuk Celebi wrote: > There is an issue for this from last week. Couldn't look into it last > week, will do tomorrow. Thanks for the logs. :) > > On Sunday, August 9, 2015

Re: Failing test in Gelly

2015-08-09 Thread Ufuk Celebi
There is an issue for this from last week. Couldn't look into it last week, will do tomorrow. Thanks for the logs. :) On Sunday, August 9, 2015, Matthias J. Sax wrote: > Wrong link... sorry. > > https://travis-ci.org/mjsax/flink/jobs/74787655 > > > > On 08/09/2015 04:02 PM, Maximilian Michels wr

Re: Improve performance of call system.currentTimeMillis()

2015-08-12 Thread Ufuk Celebi
Hey Fengbin, did you run a program and noticed some kind of performance degradation because of this? If yes, could you provide some details? If not, I would suggest to not do this. I can see how this "improves" performance in micro benchmarks, but not how it would affect the overall performance o

Re: Improve performance of call system.currentTimeMillis()

2015-08-12 Thread Ufuk Celebi
On Wed, Aug 12, 2015 at 3:57 PM, Stephan Ewen wrote: > I curious what the results are! Same here! :-)

Re: Gelly Blog Post

2015-08-25 Thread Ufuk Celebi
Andra just opened a PR for the post. Since there was consensus to publish, I will publish it later today. If there are any last minute objections, speak now or forever hold your peace. The PR is here: https://github.com/apache/flink-web/pull/7 – Ufuk On 23 Aug 2015, at 16:28, Vasiliki Kalavri

Re: Gelly Blog Post

2015-08-25 Thread Ufuk Celebi
Blog post is live: http://flink.apache.org/news/2015/08/24/introducing-flink-gelly.html Feel free to spread the word. :) On Tue, Aug 25, 2015 at 10:29 AM, Ufuk Celebi wrote: > Andra just opened a PR for the post. > > Since there was consensus to publish, I will publish it later

Re: [DISCUSSION] Release current master as 0.9.1 (mod few changes)

2015-08-26 Thread Ufuk Celebi
I think you are the best one to assess this at the moment since you are doing the hard work of back porting the changes. Are you suggesting this, because it is a) less error-prone/easier or b) faster to do? For those that haven't followed the discussion: Stephan is back porting fixes for the stre

Re: Bug fix release 0.9.1

2015-08-26 Thread Ufuk Celebi
gt; > > because we reverted null-value support for in the Table API for the > 0.9.0 > > > release. If the support on the master is fixed, we could include it > again > > > together with the fix for the RowSerializer and CaseClassComperator. > > > > > &g

Re: [DISCUSSION] Release current master as 0.9.1 (mod few changes)

2015-08-26 Thread Ufuk Celebi
+1 I very much like Robert's suggestion. This way we can proceed with the 0.9.1 release as planned for the remaining part and have 0.10-milestone1 with the fix. What about the others? Please give feedback early to allow me to proceed with the release.

Re: Bug fix release 0.9.1

2015-08-27 Thread Ufuk Celebi
Hey all, the artifacts are uploading at the moment. I will start a vote thread as soon as everything is up. :-) – Ufuk On Wed, Aug 26, 2015 at 10:26 AM, Ufuk Celebi wrote: > I went over all issues and pinged a couple of people about open issues. > Thanks for all the help! All JIRA

[VOTE] Release Apache Flink 0.9.1 (RC0)

2015-08-27 Thread Ufuk Celebi
Dear community, Please vote on releasing the following candidate as Apache Flink version 0.9.1. This is a maintenance release for Flink's latest stable version. The candidate fixes 37 issues [1] and adds 46 commits. - The commit to be vo

Testing the 0.9.1 Candidate

2015-08-27 Thread Ufuk Celebi
Hey all, we've just started the vote for a new bugfix release with version 0.9.1. I've created the following document to synchronize testing efforts. Feel free to assign yourself to certain tests in the document. The list of tests is taken from the Wiki. https://docs.google.com/document/d/1GwDYS

[RESULT] [VOTE] Release Apache Flink 0.9.1 (RC0)

2015-08-27 Thread Ufuk Celebi
that most release tests will not be affected by this and can be forwarded to the upcoming RC1. I will prepare a new RC shortly (will take some time though due to uploading etc.). – Ufuk On Thu, Aug 27, 2015 at 1:32 PM, Ufuk Celebi wrote: > Dear community, > > Please vote on releasing the

Re: Testing the 0.9.1 Candidate

2015-08-27 Thread Ufuk Celebi
The vote has been cancelled. Most tests can be forwarded. I will update the document as soon as the new RC is out. On Thu, Aug 27, 2015 at 1:38 PM, Ufuk Celebi wrote: > Hey all, > > we've just started the vote for a new bugfix release with version 0.9.1. > > I've creat

Release 0.10.0 Milestone 1 Forked Off

2015-08-28 Thread Ufuk Celebi
Hey all! I've forked off the branch for the 0.10.0 milestone release (release-0.10.0-milestone-1). All changes that we want in the milestone release need to be cherry-picked into this new branch. Feel free to merge any big changes to master now. – Ufuk

Re: [RESULT] [VOTE] Release Apache Flink 0.9.1 (RC0)

2015-08-28 Thread Ufuk Celebi
s.apache.org/jira/browse/FLINK-2584, I'm going to > add a script to our travis build which will automatically check the > contents of the fat jar for improperly shaded classes. > > On Thu, Aug 27, 2015 at 4:21 PM, Ufuk Celebi wrote: > > > This vote is cancelled in favor of

[VOTE] Release Apache Flink 0.9.1 (RC1)

2015-08-28 Thread Ufuk Celebi
Dear community, Please vote on releasing the following candidate as Apache Flink version 0.9.1. This is a maintenance release for Flink's latest stable version. The candidate fixes 38 issues [1] and adds 47 commits. This is the second RC for this release.

Re: Testing the 0.9.1 Candidate

2015-08-28 Thread Ufuk Celebi
3 PM, Ufuk Celebi wrote: > The vote has been cancelled. Most tests can be forwarded. I will update > the document as soon as the new RC is out. > > On Thu, Aug 27, 2015 at 1:38 PM, Ufuk Celebi wrote: > >> Hey all, >> >> we've just started the vote for a n

Re: [VOTE] Release Apache Flink 0.9.1 (RC1)

2015-08-30 Thread Ufuk Celebi
t; > On Fri, Aug 28, 2015 at 1:27 PM, Ufuk Celebi wrote: > > > Dear community, > > > > Please vote on releasing the following candidate as Apache Flink version > > 0.9.1. This is a maintenance release for Flink's latest stable version. > The > > candidate

[RESULT] [VOTE] Release Apache Flink 0.9.1 (RC1)

2015-08-31 Thread Ufuk Celebi
The vote is over and passes. :) Thanks to everyone who voted! Result: 8 +1s (7 binding): Stephan Ewen (binding) Ufuk Celebi (binding) Aljoscha Krettek (binding) Henry Saputra (binding) Robert Metzger (binding) Till Rohrmann (binding) Chiwan Park (non-binding) Maximilian Michels (binding) No +0s

[ANNOUNCE] Apache Flink 0.9.1 released

2015-09-01 Thread Ufuk Celebi
The Apache Flink community is pleased to announce the availability of the 0.9.1 release. Apache Flink is an open source platform for scalable stream and batch data processing. Flink’s core consists of a streaming dataflow engine that provides data distribution, communication, and fault tolerance f

Re: [ANNOUNCE] Welcome Matthias Sax as new committer

2015-09-02 Thread Ufuk Celebi
Welcome Matthias! The Storm compatibility is a great addition to Flink :-) – Ufuk On Wed, Sep 2, 2015 at 5:24 PM, Márton Balassi wrote: > Welcome, Matthias :) > > On Wed, Sep 2, 2015 at 5:21 PM, Aljoscha Krettek > wrote: > > > Welcome, welcome, welcome :D > > > > On Wed, 2 Sep 2015 at 15:53 Fa

Re: Too many changed files building Flink web page

2015-09-03 Thread Ufuk Celebi
> On 03 Sep 2015, at 09:56, Maximilian Michels wrote: > > Hi Matthias, > > I'm totally with you on this issue. However, enforcing a strict > version is not a trivial thing. For some people, it might be difficult > to install a specific Jekyll version because of the dependencies on > libraries a

<    1   2   3   4   5   6   7   8   9   10   >