Hi Renato, I would suggest reading the Arrow specification [1] which explains how nesting is handled.
-Micah [1] https://arrow.apache.org/docs/format/Columnar.html On Thu, Nov 12, 2020 at 6:40 AM Renato Marroquín Mogrovejo < renatoj.marroq...@gmail.com> wrote: > Hi Micah, > > Thanks for the answer! yeah basically that was my question, I was not > sure about the full extent of support for nested data, but it seems > that both (parquet binding and arrow format) do reading/writing nested > objects. > > Just a couple of follow up questions: > - If Arrow/Feather file format do not use Parquet's FSM for > reading/writing arbitrarily nested objects, what was the approach > taken? is there a PR or JIRA issue I could look at? > - Could you please point me to some docs (or source code) where I can > read more about how reading/writing arbitrarily nested objects is > done? > > Thanks again for your help! > Best, > > Renato M. > > El mié., 11 nov. 2020 a las 21:23, Micah Kornfield > (<emkornfi...@gmail.com>) escribió: > > > > Hi Renato, > > I'm not clear if you are asking if the Arrow/Feather file format support > > this or if Arrow's parquet binding support it. > > > > Regardless, both formats as of 2.0.0 now support arbitrarily nested data > > (there were some bugs discovered after the 2.0.0 release, and I just > > started a discussion on dev@ to see if we are going to do a patch > release. > > > > Thanks, > > Micah > > > > On Wed, Nov 11, 2020 at 1:21 AM Renato Marroquín Mogrovejo < > > renatoj.marroq...@gmail.com> wrote: > > > > > Hi Arrow experts, > > > > > > I am trying to find out if Arrow supports reading/writing arbitrary > > > nested objects similarly to what Parquet supports with its FSM. > > > I came across this PR https://github.com/apache/arrow/pull/4066 which > > > aimed to implement the Parquet specific approach (the FSM) but it was > > > declined because as far as I understand it was too Parquet specific. > > > Then, there is https://issues.apache.org/jira/browse/ARROW-1644 which > > > tracks the tasks related to writing nested objects, but I couldn't > > > make sense of it all. > > > > > > Basically my question is: Does arrow provide the functionality of > > > reading/writing arbitrarily nested objects (similar to Parquet's FSM > > > implementation)? Could you please point me to some documentation or > > > code for this? > > > > > > Thanks in advance! > > > > > > > > > Best, > > > > > > Renato M. > > > >
