Hi Mich, Unfortunately it doesn't support PySpark, just Scala/Java. Wouldn't be a big deal to implement the Quenya DSL for PySpark as well, I will add to the roadmap.
Thank you On Mon, Nov 22, 2021 at 1:24 PM Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > Ok interesting Daniel, > > I did not see support for PySpark. Is this work in progress? > > HTH > > > > view my Linkedin profile > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Mon, 22 Nov 2021 at 16:00, Daniel de Oliveira Mantovani < > daniel.oliveira.mantov...@gmail.com> wrote: > >> Hi Spark Team, >> >> I've written a library for Apache Spark to flatten JSON/Avro/Parquet/XML >> using a DSL(Domain Specific Language) in Apache Spark. You actually don't >> even need to write the DSL, you can generate it as well :) >> >> I've written an article to teach how to use: >> >> https://medium.com/@danielmantovani/flattening-json-in-apache-spark-with-quenya-dsl-b3af6bd2442d >> >> Project Page: >> https://github.com/modakanalytics/quenya-dsl >> >> For all data engineers who won't spend time anymore flattening nested >> data structures >> XOXO >> >> -- >> >> -- >> Daniel Mantovani >> >> -- -- Daniel Mantovani
