Hi Spark Team, I've written a library for Apache Spark to flatten JSON/Avro/Parquet/XML using a DSL(Domain Specific Language) in Apache Spark. You actually don't even need to write the DSL, you can generate it as well :)
I've written an article to teach how to use: https://medium.com/@danielmantovani/flattening-json-in-apache-spark-with-quenya-dsl-b3af6bd2442d Project Page: https://github.com/modakanalytics/quenya-dsl For all data engineers who won't spend time anymore flattening nested data structures XOXO -- -- Daniel Mantovani
