Hi Philip,
How about making the empty field of type String? Then you can read the CSV
into a DataSet and treat the empty string as a null value. Not very nice
but a workaround. As of now, Flink deliberately doesn't support null values.
Regards,
Max
On Thu, Oct 22, 2015 at 4:30 PM, Philip Lee <philjj...@gmail.com> wrote:
> Hi,
>
> I am trying to load the dataset with the part of null value by using
> readCsvFile().
>
> // e.g _date|_click|_sales|_item|_web_page|_user
>
> case class WebClick(_click_date: Long, _click_time: Long, _sales: Int, _item:
> Int,_page: Int, _user: Int)
>
> private def getWebClickDataSet(env: ExecutionEnvironment): DataSet[WebClick]
> = {
>
> env.readCsvFile[WebClick](
> webClickPath,
> fieldDelimiter = "|",
> includedFields = Array(0, 1, 2, 3, 4, 5),
> // lenient = true
> )
> }
>
>
> Well, I know there is an option to ignore malformed value, but I have to
> read the dataset even though it has null value.
>
> as it follows, dataset (third column is null) looks like
> 37794|24669||16705|23|54810
> but I have to read null value as well because I have to use filter or
> where function ( _sales == null )
>
> Is there any detail suggestion to do it?
>
> Thanks,
> Philip
>
>
>
>
>
>
>
> --
>
> ==========================================================
>
> *Hae Joon Lee*
>
>
> Now, in Germany,
>
> M.S. Candidate, Interested in Distributed System, Iterative Processing
>
> Dept. of Computer Science, Informatik in German, TUB
>
> Technical University of Berlin
>
>
> In Korea,
>
> M.S. Candidate, Computer Architecture Laboratory
>
> Dept. of Computer Science, KAIST
>
>
> Rm# 4414 CS Dept. KAIST
>
> 373-1 Guseong-dong, Yuseong-gu, Daejon, South Korea (305-701)
>
>
> Mobile) 49) 015-251-448-278 in Germany, no cellular in Korea
>
> ==========================================================
>
