SQL query support in Flink

Hi, Just wanted to check whether one can directly run SQL queries on Flink. For example whether one can define a table on a dataset and then run queries like dataset.sql ("select column1, column2 from mytable"). I used to think that this is possible right now in 0.10.1. But when I checked the bl

Re: Flink with Yarn

Hi Gordon, Thanks for the explanation. It is much clear now. Looks like a much cleaner approach. In that way the driver program can run in a machine which does not need connectivity to all worker nodes. Regards, Sourav On Mon, Jan 11, 2016 at 9:22 AM, Tzu-Li (Gordon) Tai wrote: > Hi Sourav, >

Re: Flink with Yarn

e > able to connect to the workers. The only requirement for firewalled > clusters is to have two ports from the master node reachable by the client. > > Greetings, > Stephan > > > On Mon, Jan 11, 2016 at 5:18 PM, Sourav Mazumder < > sourav.mazumde...@gmail.com> wr

Flink with Yarn

I am going through the documentation of integrating Flink with YARN. However not sure whether Flink can be run on YARN in two modes (like Spark). In one mode the driver/client program of Flink is also managed by YARN. In the second mode where the client program is outside the control of YARN. Is t

Re: Security in Flink

ed any user involvement). > > > Let us know if that answers your questions, and if that meets your > requirements. > > Greetings, > Stephan > > > On Fri, Jan 8, 2016 at 3:23 PM, Sourav Mazumder < > sourav.mazumde...@gmail.com> wrote: > >> Hi, >

Security in Flink

Hi, Can anyone point me to ant documentation on support for Security in Flink ? The type of information I'm looking for are - 1. How do I do user level authentication to ensure that a job is submitted/deleted/modified by the right user ? Is it possible though the web client ? 2. Authentication a

Re: Streaming in Flink

ur question. Hope this helps. > > BR, > Gordon > > On Tue, Jan 5, 2016 at 12:42 PM, Sourav Mazumder < > sourav.mazumde...@gmail.com> wrote: > >> Hi, >> >> Does Flink support push based data streaming where the data source can >> push th

Streaming in Flink

Hi, Does Flink support push based data streaming where the data source can push the events/data to Flink cluster over a socket (instead of Flink pulling the data at a given frequency)? Regards, Sourav

Re: What is the equivalent of Spark RDD is Flink

s > (ExecutionEnvironment.setParallelism(int)) or per operation, by calling > setParallelism(int) on a DataSet. > > > > (Above you can always replace DataSet by DataStream, the same > explanations hold.) > > > > So, to get back to your question, yes, the operation o

What is the equivalent of Spark RDD is Flink

Hi, I am new to Flink. Trying to understand some of the basics of Flink. What is the equivalent of Spark's RDD in Flink ? In my understanding the closes think is DataSet API. But wanted to reconfirm. Also using DataSet API if I ingest a large volume of data (val lines : DataSet[String] = env.rea