Gelly and ML for Streaming

2015-05-11 Thread sirinath
It would be good if you would consider how Gelly and ML can be used on top of streaming also from the onset. This would mean that in Gelly you should be able to read Graph mutation operation from the stream. Initial Graph (Vertices and Edges) if any followed by mutation stream. -- View this mes

Re: [DISCUSS] Naming and Functionality of Stream Operators and Tasks

2015-05-11 Thread Aljoscha Krettek
My proposal for the runtime classes (per my Pull Request is this): StreamTask: base of streaming tasks, the task is the AbstractInvokable that runs in the TaskManager and invokes stream operators OneInputStreamTask and TwoOnputStreamTask and SourceStreamTask are the subclasses responsible for act

AW: New project website

2015-05-11 Thread Markl, Volker, Prof. Dr.
-1 Dear all, first and foremost, thanks to Ufuk, for the new design. However, I agree with Felix that the current page is more of a documentation than a front page. I wonder where would people who do not know about Flink already find what it is about quickly in a concise way? I believe that t

[jira] [Created] (FLINK-2001) DistanceMetric cannot be serialized

2015-05-11 Thread Chiwan Park (JIRA)
Chiwan Park created FLINK-2001: -- Summary: DistanceMetric cannot be serialized Key: FLINK-2001 URL: https://issues.apache.org/jira/browse/FLINK-2001 Project: Flink Issue Type: Bug Compo

Re: Setup development enviroment

2015-05-11 Thread Stephan Ewen
Fixed it in the current master... On Mon, May 11, 2015 at 10:17 PM, Stephan Ewen wrote: > I guess the links came from the README.md file. > > They should be fixed as well, or we will see this question again ;-) > > > On Mon, May 11, 2015 at 5:53 PM, 程浩 wrote: > >> Thank you all for the quick re

Re: Setup development enviroment

2015-05-11 Thread Stephan Ewen
I guess the links came from the README.md file. They should be fixed as well, or we will see this question again ;-) On Mon, May 11, 2015 at 5:53 PM, 程浩 wrote: > Thank you all for the quick response. > > 在 2015年5月11日,下午11:45,Aljoscha Krettek 写道: > > > The preferred IDE for Scala (and in gener

Re: [DISCUSS] Behaviour of Streaming Sources

2015-05-11 Thread Stephan Ewen
Thanks, Marton, for the summary. The PollingStreamSource essentially follows the API suggested by Aljoscha, and will probably internally use a backoff sleep time (as Matthias pointed out), so we really have arrived at a mix of techniques ;-) On Mon, May 11, 2015 at 4:12 PM, Márton Balassi wrote:

Re: New project website

2015-05-11 Thread Stephan Ewen
I agree that we should have a "more serious content, less happy hipster" website. The current one is very nice, but I also think it has a tendency to pack too much content into a small space. For me (and probably other old people with less-than-sharp eyes), it may help to space things out a bit...

Re: [DISCUSS] Naming and Functionality of Stream Operators and Tasks

2015-05-11 Thread Stephan Ewen
How about separating the discussions about runtime class renaming (there seems to be consensus) from the API class renaming (no consensus yet). To go ahead with the runtime classes, can you make a concrete suggestion for more memorable/describing names? For the API classes, kick off a thread, if

Re: [DISCUSS] Change Streaming Operators to be Push-Only

2015-05-11 Thread Aljoscha Krettek
There is already a Jira and a Pull Request: https://github.com/apache/flink/pull/659 On Mon, May 11, 2015 at 6:29 PM, Stephan Ewen wrote: > Yep, I would say: Move ahead :-) > > On Tue, May 5, 2015 at 4:48 PM, Aljoscha Krettek > wrote: > >> So I gather I should go forward with this? If no-one obj

Re: [Documentation] Broken links

2015-05-11 Thread Stephan Ewen
Great, thanks a lot! On Thu, May 7, 2015 at 10:18 PM, Andra Lungu wrote: > Hi Robert, > > Sorry I did not think of doing something like that before :) > https://github.com/apache/flink/pull/662 > > I tend to do lazy evaluation, insist a little bit more next time you see > something I could help

Re: Migrating our website from SVN to Git

2015-05-11 Thread Maximilian Michels
We're now on Git for our website! Instructions for changing the website have been updated in the "How to contribute" guide: http://flink.apache.org/how-to-contribute.html#contribute-changes-to-the-website On Thu, Apr 30, 2015 at 6:36 PM, Henry Saputra wrote: > Sound good to me =) > > - Henry >

Re: [DISCUSS] Change Streaming Operators to be Push-Only

2015-05-11 Thread Stephan Ewen
Yep, I would say: Move ahead :-) On Tue, May 5, 2015 at 4:48 PM, Aljoscha Krettek wrote: > So I gather I should go forward with this? If no-one objects I will > open a Jira and work on this. > > On Tue, May 5, 2015 at 4:14 PM, Aljoscha Krettek > wrote: > > Yes, because the handling of punctuati

Re: Setup development enviroment

2015-05-11 Thread 程浩
Thank you all for the quick response. 在 2015年5月11日,下午11:45,Aljoscha Krettek 写道: > The preferred IDE for Scala (and in general also, I think) is IntelliJ by now. > > On Mon, May 11, 2015 at 5:44 PM, Ufuk Celebi wrote: >> >> On 11 May 2015, at 17:42, 程浩 wrote: >> >>> Hi, I am trying to setup

Re: Setup development enviroment

2015-05-11 Thread Aljoscha Krettek
The preferred IDE for Scala (and in general also, I think) is IntelliJ by now. On Mon, May 11, 2015 at 5:44 PM, Ufuk Celebi wrote: > > On 11 May 2015, at 17:42, 程浩 wrote: > >> Hi, I am trying to setup the development env with Scala-Ide or InteiliJ, >> however seems both links >> https://github.

Re: Setup development enviroment

2015-05-11 Thread Ufuk Celebi
On 11 May 2015, at 17:42, 程浩 wrote: > Hi, I am trying to setup the development env with Scala-Ide or InteiliJ, > however seems both links > https://github.com/apache/flink/blob/master/docs/internal_setup_eclipse.md > https://github.com/apache/flink/blob/master/docs/internal_setup_intellij.md >

Setup development enviroment

2015-05-11 Thread 程浩
Hi, I am trying to setup the development env with Scala-Ide or InteiliJ, however seems both links https://github.com/apache/flink/blob/master/docs/internal_setup_eclipse.md https://github.com/apache/flink/blob/master/docs/internal_setup_intellij.md are dead. Is there any specific setting I should

[jira] [Created] (FLINK-2000) Add SQL-style aggregations for Table API

2015-05-11 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-2000: --- Summary: Add SQL-style aggregations for Table API Key: FLINK-2000 URL: https://issues.apache.org/jira/browse/FLINK-2000 Project: Flink Issue Type: Impr

Re: [DISCUSS] Behaviour of Streaming Sources

2015-05-11 Thread Márton Balassi
We had a conversation with Stephan, Aljoscha, Gyula and Paris and converged on the following outline for the streaming source interface. The question is tricky because we need to coordinate between the actual source computation and triggering the checkpointing of the state of the source. We should

Re: New project website

2015-05-11 Thread Timo Walther
I completely agree with Robert. Less marketing, more technical information targeted to the user group. I think the diagram could be a little bit larger and the blog post smaller, so that the lines meet in the middle ;) On 11.05.2015 14:39, Robert Metzger wrote: I think its fine when the fron

Re: New project website

2015-05-11 Thread Robert Metzger
I think its fine when the front page looks a little bit more like a documentation. Flink is no fancy hipster app, our users are either sysops running it on a cluster or developers programming against APIs. I think the new website will convince this target audience. I agree with you that aligning t

Re: New project website

2015-05-11 Thread Felix Neutatz
Hi Ufuk, I really like the idea of redesigning the start page. But in my opinion your page design looks more like a documentation webpage than a starting page. In my personal opinion I like the current design better, since you get a really quick overview with many fancy pictures. (So if you wanna

Re: New project website

2015-05-11 Thread Hermann Gábor
Great! It looks way better than the current site :) On Mon, May 11, 2015 at 1:28 PM Stephan Ewen wrote: > I think we may have to remove the term "Language Integrated Queries", I > think that is trademarked by Microsoft. > > Otherwise, +1 > > On Mon, May 11, 2015 at 1:19 PM, Maximilian Michels >

Re: New project website

2015-05-11 Thread Stephan Ewen
I think we may have to remove the term "Language Integrated Queries", I think that is trademarked by Microsoft. Otherwise, +1 On Mon, May 11, 2015 at 1:19 PM, Maximilian Michels wrote: > +1 very nice comprehensive overhaul of the website. I'd suggest to merge > this as soon as possible. We can

Re: New project website

2015-05-11 Thread Maximilian Michels
+1 very nice comprehensive overhaul of the website. I'd suggest to merge this as soon as possible. We can incrementally fix the remaining issues and add a twitter feed to the front page. On Mon, May 11, 2015 at 12:09 PM, Ted Dunning wrote: > If there is an active twitter entity for Flink, I reco

Re: Parquet Article / Tutorial

2015-05-11 Thread Felix Neutatz
Hi Flavio, changing the block size may help. But I haven't played around with it yet. Best regards, Felix Am 11.05.2015 12:23 nachm. schrieb "Flavio Pompermaier" < pomperma...@okkam.it>: > Hi Felix, > I was looking at your code and I don't see any parquet.block.size settings. > Do you think it i

Re: Parquet Article / Tutorial

2015-05-11 Thread Flavio Pompermaier
Hi Felix, I was looking at your code and I don't see any parquet.block.size settings. Do you think it is safe to keep defaults? On Fri, Apr 24, 2015 at 11:19 PM, Flavio Pompermaier wrote: > Thanks Felix, > Thanks fir the response! > I'm looking forward to use it! > On Apr 24, 2015 9:01 PM, "Feli

Re: New project website

2015-05-11 Thread Ted Dunning
If there is an active twitter entity for Flink, I recommend framing that on the home page. On Mon, May 11, 2015 at 8:51 AM, Ufuk Celebi wrote: > Hey all, > > I reworked the project website the last couple of days and would like to > share the preview: > > http://uce.github.io/flink-web/ > > I

Re: New project website

2015-05-11 Thread Fabian Hueske
Hi, I like the new website a lot. Great Job Ufuk! Here are some things that I notices while checking it out: General: - I find the text area a bit too wide for comfortable reading, especially for long texts such as blog posts or the how to contribute guide. Front page: - Stack Figure: Don't stac

Re: New project website

2015-05-11 Thread Vasiliki Kalavri
Hi, +1 for white websites indeed :-) Some comments from my side: - The logo in the menu is quite disproportional in Chrome and looks quite bad. I can send you a screenshot if you can't check. It looks good in Safari. - The stack image is a bit blurry and also quite confusing. It m

Re: [DISCUSS] Behaviour of Streaming Sources

2015-05-11 Thread Gyula Fóra
I would not go into this direction. Returning lists is messy I think. I would stick with hasNext and Next returning a single element On Mon, May 11, 2015 at 10:20 AM, Aljoscha Krettek wrote: > We could also change next() to return List and say that the method > must not sit and wait but simply r

Re: New project website

2015-05-11 Thread Timo Walther
+1 This version looks much clearer and more professional. Some comments: - I would increase the font difference between headings and subheadings. E.g. at http://uce.github.io/flink-web/features.html - Quickstart is missing On 11.05.2015 09:59, Chiwan Park wrote: Great! :) +1 for this version

Re: Hello Everyone

2015-05-11 Thread Aljoscha Krettek
I think you can replace Validate.NotNull(p) with require(p != null, "p is null (or something like this)"). On Mon, May 11, 2015 at 12:27 AM, Lokesh Rajaram wrote: > 1. I think I can use require for replacing Validate.isTrue > 2. What about Validate.notNull? If require is used it would throw > Ill

Re: [DISCUSS] Behaviour of Streaming Sources

2015-05-11 Thread Aljoscha Krettek
We could also change next() to return List and say that the method must not sit and wait but simply return stuff that is available without waiting while also being able to not return anything for the moment. On Fri, May 8, 2015 at 12:05 PM, Matthias J. Sax wrote: > You are right. That is why I po

Re: New project website

2015-05-11 Thread Robert Metzger
Amazing work! The new site looks really good. I like white websites more, they are easier to read. Also, its great that you're putting the community on the front page. I hear quite often that users like the good support they are getting from the mailing lists. I personally experienced other open so

Re: New project website

2015-05-11 Thread Aljoscha Krettek
+1 On Mon, May 11, 2015 at 9:59 AM, Chiwan Park wrote: > Great! :) > +1 for this version. > > Regards. > Chiwan Park (Sent with iPhone) > > > >> On May 11, 2015, at 4:51 PM, Ufuk Celebi wrote: >> >> Hey all, >> >> I reworked the project website the last couple of days and would like to >> share

Re: [DISCUSS] Naming and Functionality of Stream Operators and Tasks

2015-05-11 Thread Aljoscha Krettek
Come to think of it, why do we even need SingleOutputStreamOperator? It is just a subclass of DataStream that has almost no functionality that couldn't be implemented in DataStream. I think it makes people wonder why the result of a transformation is not a DataStream but this mouthful of a class.

Re: New project website

2015-05-11 Thread Chiwan Park
Great! :) +1 for this version. Regards. Chiwan Park (Sent with iPhone) > On May 11, 2015, at 4:51 PM, Ufuk Celebi wrote: > > Hey all, > > I reworked the project website the last couple of days and would like to > share the preview: > > http://uce.github.io/flink-web/ > > I would like to g

New project website

2015-05-11 Thread Ufuk Celebi
Hey all, I reworked the project website the last couple of days and would like to share the preview: http://uce.github.io/flink-web/ I would like to get this in asap. We can push incremental updates at any time, but I think this version is a big improvement over the current status quo. If I g