Hi Folks, We've been working on GORA (Generic Object Representation using Avro) for some years now. https://gora.apache.org
The framework provides an in-memory data model and persistence for big data. Gora supports persisting to column stores, key value stores, document stores, distributed in-memory key/value stores, in-memory data grids, in-memory caches, distributed multi-model stores, and hybrid in-memory architectures. I am intersted in seeing how arrow can be used within GORA and would appreciate some input from the community here. In GORA we maintain the concept of modeling object structure through use of a JSON (Avro) schema similar to what we see within the Arrow schema design as documented at http://arrow.apache.org/docs/metadata.html#schemas. This is encouraging and although I am still learning about Arrow is seems that it would be a very nice fit for GORA to consider leveraging as a common data format/layer. A few questions I have as follows 1) Is Avro supported as an Arrow system? 2) If so, is there any mechanism(s) to transition/copy data which has been written using Gora (with Avro as the underlying data format) to Arrow memory format? I see there is mention of "Projects can share functionality (eg, Parquet-to-Arrow reader)", is such functionality available for Avro? Let's start with the above, I can take it from there. Thanks in advance for any heads up folks, Lewis
