Hi,
I am trying to load the dataset with the part of null value by using
readCsvFile().
// e.g _date|_click|_sales|_item|_web_page|_user
case class WebClick(_click_date: Long, _click_time: Long, _sales: Int,
_item: Int,_page: Int, _user: Int)
private def getWebClickDataSet(env: ExecutionEnvironment): DataSet[WebClick] = {
env.readCsvFile[WebClick](
webClickPath,
fieldDelimiter = "|",
includedFields = Array(0, 1, 2, 3, 4, 5),
// lenient = true
)
}
Well, I know there is an option to ignore malformed value, but I have to
read the dataset even though it has null value.
as it follows, dataset (third column is null) looks like
37794|24669||16705|23|54810
but I have to read null value as well because I have to use filter or where
function ( _sales == null )
Is there any detail suggestion to do it?
Thanks,
Philip
--
==========================================================
*Hae Joon Lee*
Now, in Germany,
M.S. Candidate, Interested in Distributed System, Iterative Processing
Dept. of Computer Science, Informatik in German, TUB
Technical University of Berlin
In Korea,
M.S. Candidate, Computer Architecture Laboratory
Dept. of Computer Science, KAIST
Rm# 4414 CS Dept. KAIST
373-1 Guseong-dong, Yuseong-gu, Daejon, South Korea (305-701)
Mobile) 49) 015-251-448-278 in Germany, no cellular in Korea
==========================================================
