Hi Sanjay, I think we are trying to work that out now. As you've seen with some of you initial investigation we have no coverage for big-endian machines yet. But in the long run, we should be able to make it work (it seems like there might be some difference of opinion on how to make it work).
Thanks, Micah On Mon, Aug 1, 2016 at 11:16 AM, Sanjay Rao <getsanjay...@live.com> wrote: > Hi Wes, Hi Micah, > I understood what you meant, so point 2. Arrow working with Big Endian > machine to Big Endian shouldn't be an issue right ? > Please confirm. > Thanks,Sanjay > > From: wesmck...@gmail.com > > Date: Mon, 1 Aug 2016 11:07:07 -0700 > > Subject: Re: Is there plan to support BigEndian Systems like SUN SPARC > Hardware ? > > To: dev@arrow.apache.org; emkornfi...@gmail.com > > > > hey Micah, > > > > On Mon, Aug 1, 2016 at 11:02 AM, Micah Kornfield <emkornfi...@gmail.com> > wrote: > > > Hi Wes, > > > The point I was trying to argue from an earlier thread is that the most > > > common cases for relocation are: > > > 1. Little endian machine to little endian machine (most likely same > > > machine) > > > 2. big endian machine to big endian machine (most likely same machine) > > > 3. big endian machine to little endian machine or vice versa > > > > > > The purpose of the metadata would be to make use-cases 1 and 2 possible > > > without byte-swapping. Use case 3 would obviously require byte > swapping > > > but for an initial implementation the code could simply indicate that > it is > > > not supported. > > > > > > This seems less complex to me than actually implementing any sort of > > > byte-swapping logic while still supporting the widest variety of > hardware > > > with the same code for the most common use-cases. > > > > This makes sense. My comments were for the situation that a big endian > > system would be exposing memory to an unknown consumer -- for example, > > if we implemented an RPC wire format for Arrow memory, then in general > > a big endian system would need to send little-endian integers to an > > arbitrary receiver. I'm not sure the best way to provide for easy > > native-endianness support for cases 1/2, but trying to fully solve > > this problem now seems premature until we've established some of these > > tools (so long as we haven't painted ourselves into a corner). > > > > - Wes > > > > > > > > Thanks, > > > Micah > > > > > > P.S. If anybody can provide pointers I'd be interested to understand > which > > > pieces of the java code make assumptions about little-endianness. > >
