Re: Flink Stream Sink ParquetWriter Failing with ClassCastException

2020-03-03 Thread Till Rohrmann
Hi Anuj, if you use the exact same schema with which the data has been written for reading and if there is no bug in the parquet Avro support, then it should indeed not fail. Hence, I suspect that the producer of your data might produce slightly different Avro records compared to what Parquet is e

Re: Flink Stream Sink ParquetWriter Failing with ClassCastException

2020-02-29 Thread aj
Hi Till, Thanks for the reply . I have doubt that input has problem because : 1. if input has some problem than it should not come in the topic itself as schema validation fail at producer side only. 2. i am using the same schema that was used to writed the record in topic and i am able to parse

Re: Flink Stream Sink ParquetWriter Failing with ClassCastException

2020-02-29 Thread Till Rohrmann
Hi Anuj, it looks to me that your input GenericRecords don't conform with your output schema schemaSubject. At least, the stack trace says that your output schema expects some String field but the field was actually some ArrayList. Consequently, I would suggest to verify that your input data has t