[SPARK-SQL] How to return GenericInternalRow from spark udf

[Amit Joshi]

Hi,

<https://stackoverflow.com/posts/63277463/timeline>

I have a spark udf written in scala that takes couuple of columns and apply
some logic and output InternalRow. There is spark schema of StructType also
present. But when I try to return the InternalRow from UDF there is
exception

java.lang.UnsupportedOperationException: Schema for type
org.apache.spark.sql.catalyst.GenericInternalRow is not supported

  val getData = (hash : String, type : String) => {
    val schema = hash match {
      case "people" =>
        peopleSchema
      case "empl" =>  emplSchema
    }
    getGenericInternalRow(schema)
  }

  val data = udf(getData)

Spark Version : 2.4.5


Please Help.


Regards

Amit Joshi