Re: Apache Flink transactions

2015-06-10 Thread Fabian Hueske
05 9:58 GMT+02:00 Aljoscha Krettek : >>>> >>> >>>> >>> Hi, >>>> >>> I think the example could be made more concise by using the Table >>>> API. >>>> >>> >>>> http://ci.apache.org/projects/flink/flink-docs-master/libs/table.html >>>> >>> >>>> >>> Please let us know if you have questions about that, it is still >>>> quite >>>> >>> new. >>>> >>> >>>> >>> On Fri, Jun 5, 2015 at 9:03 AM, hawin >>>> wrote: >>>> >>> > Hi Aljoscha >>>> >>> > >>>> >>> > Thanks for your reply. >>>> >>> > Do you have any tips for Flink SQL. >>>> >>> > I know that Spark support ORC format. How about Flink SQL? >>>> >>> > BTW, for TPCHQuery10 example, you have implemented it by 231 >>>> lines of >>>> >>> > code. >>>> >>> > How to make that as simple as possible by flink. >>>> >>> > I am going to use Flink in my future project. Sorry for so many >>>> >>> > questions. >>>> >>> > I believe that you guys will make a world difference. >>>> >>> > >>>> >>> > >>>> >>> > @Chiwan >>>> >>> > You made a very good example for me. >>>> >>> > Thanks a lot >>>> >>> > >>>> >>> > >>>> >>> > >>>> >>> > >>>> >>> > >>>> >>> > -- >>>> >>> > View this message in context: >>>> >>> > >>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Re-Apache-Flink-transactions-tp1457p1494.html >>>> >>> > Sent from the Apache Flink User Mailing List archive. mailing list >>>> >>> > archive at Nabble.com. >>>> >> >>>> >> >>>> > >>>> >>> >>> >> >

Re: Apache Flink transactions

2015-06-09 Thread Hawin Jiang
; API. >>> >>> >>> http://ci.apache.org/projects/flink/flink-docs-master/libs/table.html >>> >>> >>> >>> Please let us know if you have questions about that, it is still >>> quite >>> >>> new. >>> >>> >>> >

Re: Apache Flink transactions

2015-06-09 Thread Hawin Jiang
example could be made more concise by using the Table >>> API. >>> >>> >>> http://ci.apache.org/projects/flink/flink-docs-master/libs/table.html >>> >>> >>> >>> Please let us know if you have questions about that, it is stil

Re: Apache Flink transactions

2015-06-09 Thread Aljoscha Krettek
> > >> >>> > Thanks for your reply. >> >>> > Do you have any tips for Flink SQL. >> >>> > I know that Spark support ORC format. How about Flink SQL? >> >>> > BTW, for TPCHQuery10 example, you have implemented it by 231 lines >> of >> >>> > code. >> >>> > How to make that as simple as possible by flink. >> >>> > I am going to use Flink in my future project. Sorry for so many >> >>> > questions. >> >>> > I believe that you guys will make a world difference. >> >>> > >> >>> > >> >>> > @Chiwan >> >>> > You made a very good example for me. >> >>> > Thanks a lot >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > -- >> >>> > View this message in context: >> >>> > >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Re-Apache-Flink-transactions-tp1457p1494.html >> >>> > Sent from the Apache Flink User Mailing List archive. mailing list >> >>> > archive at Nabble.com. >> >> >> >> >> > >> > >

Re: Apache Flink transactions

2015-06-08 Thread Hawin Jiang
> Thanks for your reply. > >>> > Do you have any tips for Flink SQL. > >>> > I know that Spark support ORC format. How about Flink SQL? > >>> > BTW, for TPCHQuery10 example, you have implemented it by 231 lines of > >>> > code. > >>> >

Re: Apache Flink transactions

2015-06-08 Thread Aljoscha Krettek
simple as possible by flink. >>> > I am going to use Flink in my future project. Sorry for so many >>> > questions. >>> > I believe that you guys will make a world difference. >>> > >>> > >>> > @Chiwan >>> > You made a very good example for me. >>> > Thanks a lot >>> > >>> > >>> > >>> > >>> > >>> > -- >>> > View this message in context: >>> > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Re-Apache-Flink-transactions-tp1457p1494.html >>> > Sent from the Apache Flink User Mailing List archive. mailing list >>> > archive at Nabble.com. >> >> >

Re: Apache Flink transactions

2015-06-05 Thread Hawin Jiang
am going to use Flink in my future project. Sorry for so many >> questions. >> > I believe that you guys will make a world difference. >> > >> > >> > @Chiwan >> > You made a very good example for me. >> > Thanks a lot >> &g

Re: Apache Flink transactions

2015-06-05 Thread Fabian Hueske
ference. > > > > > > @Chiwan > > You made a very good example for me. > > Thanks a lot > > > > > > > > > > > > -- > > View this message in context: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Re-Apache-Flink-transactions-tp1457p1494.html > > Sent from the Apache Flink User Mailing List archive. mailing list > archive at Nabble.com. >

Re: Apache Flink transactions

2015-06-05 Thread Aljoscha Krettek
ture project. Sorry for so many questions. > I believe that you guys will make a world difference. > > > @Chiwan > You made a very good example for me. > Thanks a lot > > > > > > -- > View this message in context: > http://apache-flink-user-mailing-list-arch

Re: Apache Flink transactions

2015-06-05 Thread hawin
project. Sorry for so many questions. I believe that you guys will make a world difference. @Chiwan You made a very good example for me. Thanks a lot -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Re-Apache-Flink-transactions

Re: Apache Flink transactions

2015-06-04 Thread Aljoscha Krettek
0 and TPCHQuery3. >> Do you have other examples for inserting, updating and removing data on HDFS >> by Apache flink >> >> Thanks >> >> >> >> >> -- >> View this message in context: >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Re-Apache-Flink-transactions-tp1457p1491.html >> Sent from the Apache Flink User Mailing List archive. mailing list archive >> at Nabble.com. > > > >

Re: Apache Flink transactions

2015-06-04 Thread Chiwan Park
3. > Do you have other examples for inserting, updating and removing data on HDFS > by Apache flink > > Thanks > > > > > -- > View this message in context: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Re-Apache-Flink-transactions-tp1

Re: Apache Flink transactions

2015-06-04 Thread hawin
removing data on HDFS by Apache flink Thanks -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Re-Apache-Flink-transactions-tp1457p1491.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Apache Flink transactions

2015-06-04 Thread Stephan Ewen
To run operations like Insert, Update, Delete on persistent files, you have to have support from the storage engine. The Apache ORC data format recently added support for transactional inserts, updates, deletes. http://orc.apache.org/ ORC has a Hadoop input format, and you can use that one with F

Re: Apache Flink transactions

2015-06-04 Thread Till Rohrmann
But what you can do to simulate an insert is to read the new data in a separate DataSet and then apply an union operator on the new and old DataSet . Cheers, Till ​ On Thu, Jun 4, 2015 at 9:00 AM, Chiwan Park wrote: > Hi. > > Flink is not DBMS. There is no equivalent operation of insert, update

Re: Apache Flink transactions

2015-06-04 Thread Chiwan Park
Hi. Flink is not DBMS. There is no equivalent operation of insert, update, remove. But you can use map[1] or filter[2] operation to create modified dataset. I recommend you some sildes[3][4] to understand Flink concepts. Regards, Chiwan Park [1] http://ci.apache.org/projects/flink/flink-docs-m