[jira] [Created] (FLINK-1806) Improve S3 file system error message when no access/secret key provided

2015-03-31 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-1806: -- Summary: Improve S3 file system error message when no access/secret key provided Key: FLINK-1806 URL: https://issues.apache.org/jira/browse/FLINK-1806 Project: Flink

Re: Travis-CI builds queuing up

2015-03-31 Thread Maximilian Michels
Very nice. Thanks Robert! On Mon, Mar 30, 2015 at 9:13 PM, Robert Metzger wrote: > It seems that the issue is fixed. I've just pushed two times to a pull > request and it immediately started building both. > I think the "apache" user has much more parallel builds available now (we > don't have a

Re: [DISCUSS] Make a release to be announced at ApacheCon

2015-03-31 Thread Maximilian Michels
+1 Would be great if we created a checklist with all the things to watch out for when testing the release candidate. On Mon, Mar 30, 2015 at 9:00 PM, Henry Saputra wrote: > +1 > > Would be good to have well documented release process with all the > black magic scripts we have =) > Thanks for dr

Re: Marking affected and fixed versions in JIRA

2015-03-31 Thread Ufuk Celebi
On Fri, Mar 27, 2015 at 7:41 PM, Henry Saputra wrote: > Developers that solve the problem by fixing the issue should change > the status to "Resolved" and the person who create the issue could > change the status to "Closed" to verify. > Yes. JIRA itself says the following (there is a small tex

Apache Flink connection with Hbase

2015-03-31 Thread santosh_rajaguru
Hi guys, I am facing problem while connecting remote hbase from Apache flink. I am able to connect successfully through simple hbase java program. However, when i try to connect and scan the table it says "java.lang.NoSuchMethodError: org.apache.hadoop.net.NetUtils.getInputStream(Ljava/net/Socke

Re: Apache Flink connection with Hbase

2015-03-31 Thread Flavio Pompermaier
Hi Santosh, which version of Flink are you using? And which version of hbase? On Mar 31, 2015 12:50 PM, "santosh_rajaguru" wrote: > Hi guys, > > I am facing problem while connecting remote hbase from Apache flink. > I am able to connect successfully through simple hbase java program. > However, w

[jira] [Created] (FLINK-1807) Stochastic gradient descent optimizer for ML library

2015-03-31 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-1807: Summary: Stochastic gradient descent optimizer for ML library Key: FLINK-1807 URL: https://issues.apache.org/jira/browse/FLINK-1807 Project: Flink Issue Type

[jira] [Created] (FLINK-1808) Omit sending checkpoint barriers when the execution graph is not running

2015-03-31 Thread Paris Carbone (JIRA)
Paris Carbone created FLINK-1808: Summary: Omit sending checkpoint barriers when the execution graph is not running Key: FLINK-1808 URL: https://issues.apache.org/jira/browse/FLINK-1808 Project: Flink

Re: Apache Flink connection with Hbase

2015-03-31 Thread santosh_rajaguru
Hi flavio, I got to manage the connection. The problem was with versioning of hbase client. i was using 0.8.1 version for flink-hbase. when i changed the version to 0.8.0 it is working fine. The hbase server is 0.98.11 . One more question flavio, is there any particular mechanism to follow while

Re: Question about Infinite Streaming Job on Mini Cluster and ITCase

2015-03-31 Thread Matthias J. Sax
Hi Robert, thanks for your answer. I get an InterruptedException when I call shutdown(): java.lang.InterruptedException at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1225) at java.lang.Thread.join(Thread.java:1278) at org.apache.flin

Re: Apache Flink connection with Hbase

2015-03-31 Thread Flavio Pompermaier
Strange, I'm using Flink 0.8.1 and HBase 0.98.6 and everything works fine (at least during reading). Remember to put the correct hbase-site.xml in the classpath! To output data I'm trying to find the best way to achieve it.. It came out that the hadoop compatibility layer of flink probably doesn't

Re: Apache Flink connection with Hbase

2015-03-31 Thread Stephan Ewen
Hi! Also important: Which Hadoop version are you using with Flink? The problem is a missing method in a Hadoop class, so I guess there is a Hadoop version mismatch. For all Flink versions, there is a package for Hadoop 1.x and a package for Hadoop 2.x . Make sure you pick the right one for HBase

[DISCUSS] Inconsistent naming of intermediate results

2015-03-31 Thread Ufuk Celebi
On a high level we call intermediate data produced by programs "intermediate results". For example in a WordCount map-reduce program the map function produces an intermediate result, which consists of (word, 1) pairs and the reduce function consumes this intermediate result. Kostas has recently

Re: [DISCUSS] Inconsistent naming of intermediate results

2015-03-31 Thread Kostas Tzoumas
I like the fact that the naming scheme follows some logic. I also like that we have two easy to understand concepts: - Operator (be that in any of the above representations) - Result (of executing an operator) +1 On Tue, Mar 31, 2015 at 4:50 PM, Ufuk Celebi wrote: > On a high level we call int

[jira] [Created] (FLINK-1809) Add standard scaler to ML library

2015-03-31 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-1809: Summary: Add standard scaler to ML library Key: FLINK-1809 URL: https://issues.apache.org/jira/browse/FLINK-1809 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Inconsistent naming of intermediate results

2015-03-31 Thread Stephan Ewen
I like getting the consistency in there. I was never thinking of the intermediate data sets to be strictly produced by a vertex, so I am unsure whether we should use that exact naming scheme, or one that disconnects the results from the term "VertexResult". On Tue, Mar 31, 2015 at 5:27 PM, Kostas

HBase TableOutputFormat fix (Flink 0.8.1)

2015-03-31 Thread Flavio Pompermaier
Hi Flink devs, this is my final report about the HBaseOutputFormat problem (with Flink 0.8.1) and I hope you could suggest me the best way to make a PR: 1) The following code produce the error reported below (this should be fixed in 0.9 right?) Job job = Job.getInstance(); myDataset.output

Re: [DISCUSS] Inconsistent naming of intermediate results

2015-03-31 Thread Henry Saputra
As one of the devs that recently been tracing the runtime portion of the code +1 for renaming for inlining with the concepts. One thing I would like to have is immediate change to the documentation [1] with renaming PR . Otherwise Then need to file followup ticket to update Kostas' awesome wiki p

[QUESTION] In the DataExchangeMode.BATCH how does the consumer being notified to start consumer the data?

2015-03-31 Thread Henry Saputra
HI Flinksters, Have one quick question, hopefully someone could help me get some answers: When is the DataExchangeMode.BATCH is actually being used, and how does the consumer know when to start consuming the buffer In the pipelined mode, the producer tell JobManager via JobManager ! ScheduleOrUp

Re: [QUESTION] In the DataExchangeMode.BATCH how does the consumer being notified to start consumer the data?

2015-03-31 Thread Ufuk Celebi
Hey Henry, 1) There is no extra message, but this is piggy backed with the finished state transition (see Execution#markAsFinished). There it is essentially the same mechanism. 2) It's part of my plan for this week to add documentation for exactly this flow of RPC messages related to the runtime

A small Project I've been working on

2015-03-31 Thread Aljoscha Krettek
Hi, I've been working on a little side project in my free time: Ruby on Flink (RoF). This should finally allow us to tap into the whole web developer ruby world. The design is also flexible enough to easily port this to node.js. The code is not yet in shape to be made public but this is a small pr

Re: A small Project I've been working on

2015-03-31 Thread Fabian Hueske
:-D This is awesome! Do you have some performance numbers? On Apr 1, 2015 8:43 AM, "Aljoscha Krettek" wrote: > Hi, > I've been working on a little side project in my free time: Ruby on > Flink (RoF). This should finally allow us to tap into the whole web > developer ruby world. The design is als