Re: Quick question regarding ParquetIO

2021-01-18 Thread Ismaël Mejía
Catching up on this thread sorry if late to the party :) and my excuses because this is going to be lng but worth. > It does look like BEAM-11460 could work for you. Note that relies on a dynamic > object which won't work with schema-aware transforms and SqlTransform. It's > likely this isn't

Iterative algorithm in BEAM

2021-01-18 Thread Tucker Barbour
I have a use-case where I'm extracting embedded items from archive file formats which themselves have embedded items. For example a zip file with emails with attachments. The goal in this example would be to create a PCollection where each email is an element as well as each attachment being an ele