Re: table toRetractStream missing last record and adding extra column (True)

2019-07-17 Thread sri hari kali charan Tummala
yes even the delimiter can be replaced, have to test what happens if the data itself has a comma in it I need to test. table.toRetractStream(TypeInformation.of(classOf[Row])) .map(_._2.toString.replaceAll(",","~")) .writeAsText("/Users/kalit_000/Downloads/FlinkStreamAndSql/src/main/resources/

Re: table toRetractStream missing last record and adding extra column (True)

2019-07-17 Thread sri hari kali charan Tummala
Amazing all issues resolved in one go thanks Cheng , one issue though I can't write map.(_._2) to CSV looks like it doesn't support right now have to be TextFile. below is a full code if someone wants in Scala. Git Code is here:- https://github.com/kali786516/FlinkStreamAndSql package com.aws.ex

Re: table toRetractStream missing last record and adding extra column (True)

2019-07-17 Thread sri hari kali charan Tummala
Question 1:- I did tired map function end up having issue ( https://stackoverflow.com/questions/57063249/flink-scala-notinferedr-in-scala-type-mismatch-mapfunctiontuple2boolean-row-i ) I am trying to convert a Tuple[Boolean,Row] to Row using map function, I am getting this error asking me for Inf

Re: table toRetractStream missing last record and adding extra column (True)

2019-07-17 Thread Hequn Cheng
Hi Sri, Question1: You can use a map to filter the "true", i.e, ds.map(_._2). Note, it's ok to remove the "true" flag for distinct as it does not generate updates. For other query contains updates, such as a non-window group by, we should not filter the flag or the result is not correct. Question

Re: table toRetractStream missing last record and adding extra column (True)

2019-07-16 Thread sri hari kali charan Tummala
windows for question 1 or question 2 or both ? Thanks Sri On Tue, Jul 16, 2019 at 12:25 PM taher koitawala wrote: > Looks like you need a window > > On Tue, Jul 16, 2019, 9:24 PM sri hari kali charan Tummala < > kali.tumm...@gmail.com> wrote: > >> Hi All, >> >> I am trying to write toRetractSre

Re: table toRetractStream missing last record and adding extra column (True)

2019-07-16 Thread taher koitawala
Looks like you need a window On Tue, Jul 16, 2019, 9:24 PM sri hari kali charan Tummala < kali.tumm...@gmail.com> wrote: > Hi All, > > I am trying to write toRetractSream to CSV which is kind of working ok but > I get extra values like True and then my output data values. > > Question1 :- > I don