So it sounds like right now it just works as long as there are no inter-system communication (with different endianness) because both java and c++ code just use the underlying endianness. Is that correct?
On Thu, Aug 4, 2016 at 11:17 AM, Micah Kornfield <emkornfi...@gmail.com> wrote: > Hi Sanjay, > I think we are trying to work that out now. As you've seen with some of > you initial investigation we have no coverage for big-endian machines yet. > But in the long run, we should be able to make it work (it seems like there > might be some difference of opinion on how to make it work). > > Thanks, > Micah > > On Mon, Aug 1, 2016 at 11:16 AM, Sanjay Rao <getsanjay...@live.com> wrote: > > > Hi Wes, Hi Micah, > > I understood what you meant, so point 2. Arrow working with Big Endian > > machine to Big Endian shouldn't be an issue right ? > > Please confirm. > > Thanks,Sanjay > > > From: wesmck...@gmail.com > > > Date: Mon, 1 Aug 2016 11:07:07 -0700 > > > Subject: Re: Is there plan to support BigEndian Systems like SUN SPARC > > Hardware ? > > > To: dev@arrow.apache.org; emkornfi...@gmail.com > > > > > > hey Micah, > > > > > > On Mon, Aug 1, 2016 at 11:02 AM, Micah Kornfield < > emkornfi...@gmail.com> > > wrote: > > > > Hi Wes, > > > > The point I was trying to argue from an earlier thread is that the > most > > > > common cases for relocation are: > > > > 1. Little endian machine to little endian machine (most likely same > > > > machine) > > > > 2. big endian machine to big endian machine (most likely same > machine) > > > > 3. big endian machine to little endian machine or vice versa > > > > > > > > The purpose of the metadata would be to make use-cases 1 and 2 > possible > > > > without byte-swapping. Use case 3 would obviously require byte > > swapping > > > > but for an initial implementation the code could simply indicate that > > it is > > > > not supported. > > > > > > > > This seems less complex to me than actually implementing any sort of > > > > byte-swapping logic while still supporting the widest variety of > > hardware > > > > with the same code for the most common use-cases. > > > > > > This makes sense. My comments were for the situation that a big endian > > > system would be exposing memory to an unknown consumer -- for example, > > > if we implemented an RPC wire format for Arrow memory, then in general > > > a big endian system would need to send little-endian integers to an > > > arbitrary receiver. I'm not sure the best way to provide for easy > > > native-endianness support for cases 1/2, but trying to fully solve > > > this problem now seems premature until we've established some of these > > > tools (so long as we haven't painted ourselves into a corner). > > > > > > - Wes > > > > > > > > > > > Thanks, > > > > Micah > > > > > > > > P.S. If anybody can provide pointers I'd be interested to understand > > which > > > > pieces of the java code make assumptions about little-endianness. > > > > > -- Julien
