Hi Spark users, Did anyone resolve this issue?
Encoder<AvroGeneratedClass> encoder = Encoders.bean(AvroGenereatedClass.class); Dataset<AvroGeneratedClass> ds = sparkSession.read().parquet(filename).as(encoder); I'm also facing the same problem: "Cannot have circular references in bean class, but got the circular reference of class class org.apache.avro.Schema" This happens due to getSchema() method in a generated Avro Java class. How can I get a typed dataset from Avro generated classes? Thanks. On Wed, Sep 27, 2017 at 3:23 AM Joaquin Tarraga <joaquintarr...@gmail.com> wrote: > Hi all, > > I have an Avro generated class (e.g., AvroGenerateClass) and I am using > Encoders.bean to get a typed dataset (e.g., Dataset<AvroGeneratedClass>): > > Encoder<AvroGeneratedClass> encoder = > Encoders.bean(AvroGenereatedClass.class); > > Dataset<AvroGeneratedClass> ds = > sparkSession.read().parquet(filename).as(encoder); > > I am getting an exception from the Encoders.bean call: > "java.lang.UnsupportedOperationException: Cannot have circular references > in bean class, but got the circular reference of class class > org.apache.avro.Schema" > > How can I get a typed dataset from Avro generated classes? > > Thanks. > -- > Joaquín > > -- Best regards, Elkhan Dadashov
