Re: Avro schema having Map of Records

2019-08-07 Thread Edgar H
Seems like the right time to share some Parquet vs Avro knowledge haha :) My god, exactly what you said! Untyped List within a POJO, problem solved. BTW, it was using ReflectData.getSchema(). Thanks a lot Ryan! Really appreciated! El mar., 6 ago. 2019 a las 17:35, Ryan Skraba () escribió: > Fun

Re: Avro schema having Map of Records

2019-08-06 Thread Ryan Skraba
Funny, I'm familiar with Avro, but I'm currently looking closely at Parquet! Interestingly enough, I just ran across the conversion utilities in Spark that could have answered your original question[1]. It looks like you're using ReflectData to get the schema. Is the exception occurring during t

Re: Avro schema having Map of Records

2019-08-06 Thread Edgar H
Thanks a lot for the quick reply Ryan! That was exactly what I was looking for :) Been trying including the changes within my code and currently it's throwing the following exception... Caused by: org.apache.avro.AvroRuntimeException: Can't find element type of Collection I'm thinking that it cou

Re: Avro schema having Map of Records

2019-08-06 Thread Ryan Skraba
Hello -- Avro supports a map type: https://avro.apache.org/docs/1.9.0/spec.html#Maps Generating an Avro schema from a JSON example can be ambiguous since a JSON object can either be converted to a record or a map. You're probably looking for something like this: { "type" : "record", "name" :