In kylin-2.5.2, the result hbase table is always table is always empty when I
build cube with spark.
I found that the step "Load HFile to HBase Table" has some warn log:
/2019-01-27 00:49:30,067 WARN [Scheduler 448149092 Job
89a25959-e12d-7a5e-0ecb-80c978533eab-6419]
mapreduce.LoadIncrementalHFiles:204 : Skipping non-directory
hdfs://test/kylin/kylin_metadata/kylin-89a25959-e12d-7a5e-0ecb-80c978533eab/test_UUID_spark/hfile/_SUCCESS
2019-01-27 00:49:30,068 WARN [Scheduler 448149092 Job
89a25959-e12d-7a5e-0ecb-80c978533eab-6419]
mapreduce.LoadIncrementalHFiles:204 : Skipping non-directory
hdfs://test/kylin/kylin_metadata/kylin-89a25959-e12d-7a5e-0ecb-80c978533eab/test_UUID_spark/hfile/part-r-00000
2019-01-27 00:49:30,068 WARN [Scheduler 448149092 Job
89a25959-e12d-7a5e-0ecb-80c978533eab-6419]
mapreduce.LoadIncrementalHFiles:204 : Skipping non-directory
hdfs://test/kylin/kylin_metadata/kylin-89a25959-e12d-7a5e-0ecb-80c978533eab/test_UUID_spark/hfile/part-r-00001/

After read the source code, I found the step "Convert Cuboid Data to HFile"
with spark has bug. The above step's outputdir should has subdirectory with
column family. Indeed, SparkCubeHFile must set
mapreduce.job.outputformat.class with HFileOutputFormat2.class. 

Please check if I am correct!

--
Sent from: http://apache-kylin.74782.x6.nabble.com/

Reply via email to