Hi
I got a problem when reading a textfile which contains nested complex
type data and got a type unmatch problem.Any hint will be appreciated.
The problem take place at "map(s => s.map" as "type mismatch; found
: scala.collection.immutable.IndexedSeq[Array[com.redhadoop.bean.StructField1]]
required: Seq[com.redhadoop.bean.StructField1]"
here is my codeļ¼
import sqlContext.createSchemaRDD
val ultraComplexTextfile =
spark.textFile("/examples/src/main/resources/aggrnew.text").map(_.split("\001")).map(p
=> UltraComplexTextFileTable(
p(0).trim.toInt,
p(1).trim.map(_.toString.split("\002")).map(s =>
s.map(_.toString.split("\003")).map(t => StructField1(
t(0).toString.trim.toInt - 48,
t(1).toString.trim.toInt - 48,
t(2).toString.trim.toInt - 48,
t(3).toString.trim.toInt - 48,
t(4).toString.trim.toInt - 48,
t(5).toString.trim.toFloat - 48,
t(6).toString.trim.toFloat - 48,
t(7).toString.trim.toFloat - 48
))
),
p(2))
)
and my case class below:
case class UltraComplexTextFileTable (
frameid:Int,
detaillist:Seq[StructField1],
frame:String
)
case class StructField1(
targetNum:Int,
x:Int,
y:Int,
width:Int,
height:Int,
HM:Float,
HS:Float,
HT:Float
)
--------------------------------
Thanks&Best regards!
San.Luo
Redhadoop