> > * I think this could also form the basis for large transfers of arrow data > over a wasm boundary. Wasm has a concept of shared memory objects[1] and > a > wasm data library could use this to stream data into javascript without a > copy.
Hi Matt, Just wanna add that I have the exact use case now where we want zero-copy Arrow Array "view" over the Arrow data inside WASM. Existing framework [1] uses IPCBatch which is suboptimal. Right now I reconstruct Arrow Array using pointers to Buffers in WASM and each reconstructed Buffer in the host has custom deallocation which will call functions in wasm. This is hacky and lost some metadata like offset and length in ArrayData. Wondering will the experimental implementation work on WASM boundary (or in future support it)? [1]: https://github.com/arrow-udf/arrow-udf/blob/d7a3a52b6c229be94cb2936068f505a884b0d51e/arrow-udf-wasm/src/lib.rs#L312-L316 On Sat, Apr 13, 2024 at 1:35 PM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi Matt > > With default ASF settings, you can only assign issues/PRs to > committers and collaborators. > > The collaborators are defined in > https://github.com/apache/arrow/blob/main/.asf.yaml. > > NB: the max number of collaborators is 10. > > If there is no objection, I will create a PR to add myself as > collaborator (as I'm not committer on Arrow :) ). > > Thanks for the PR Matt, I will take a look ! > > Regards > JB > > On Sat, Apr 13, 2024 at 12:31 AM Matt Topol <zotthewiz...@gmail.com> > wrote: > > > > Hey again JB, > > > > I don't remember your GitHub handle offhand so I wasn't able to add you, > > but I've put up a PR [1] with the full write up for the protocol being > > added to the Arrow documentation. Please review and add any comments at > > your convenience! > > > > Thanks! > > > > --Matt > > > > [1]: https://github.com/apache/arrow/pull/41180 > > > > On Tue, Apr 9, 2024, 10:39 AM Matt Topol <zotthewiz...@gmail.com> wrote: > > > > > Hey JB, > > > > > > The next step for me is going to be converting the document into a > > > markdown version with more prose that we can add to the Arrow > documentation > > > site (marked as Experimental of course). > > > > > > > (if I can help in any way :) ) > > > > > > I'll definitely add you as a reviewer when I put the PR up :). But i > guess > > > the best way you could help would be to get anyone you know who might > be > > > interested in this protocol or using the protocol to give it a try, > have a > > > look at the example code, and provide any and all feedback they can. > The > > > biggest thing that this protocol needs is real world usage that we can > > > utilize to improve and update this protocol based on use-cases. > > > > > > --Matt > > > > > > On Tue, Apr 9, 2024 at 8:39 AM Jean-Baptiste Onofré <j...@nanthrax.net> > > > wrote: > > > > > >> Hi Matt, > > >> > > >> Sorry, I'm late on this vote and in the doc review. > > >> > > >> I like the idea. Thanks for the very detailed document. > > >> > > >> Just curious, what's the next step (if I can help in any way :) ) ? > > >> > > >> Regards > > >> JB > > >> > > >> On Tue, Feb 27, 2024 at 6:35 PM Matt Topol <zotthewiz...@gmail.com> > > >> wrote: > > >> > > > >> > Hey all, > > >> > > > >> > I'd like to propose a vote for us to officially adopt the protocol > > >> > described in the google doc[1] for Dissociated Arrow IPC Transports. > > >> This > > >> > proposal was originally discussed at [2]. Once this proposal is > > >> adopted, I > > >> > will work on adding the necessary documentation to the Arrow website > > >> along > > >> > with examples etc. > > >> > > > >> > The vote will be open for at least 72 hours. > > >> > > > >> > [ ] +1 Accept this Proposal > > >> > [ ] +0 > > >> > [ ] -1 Do not accept this proposal because... > > >> > > > >> > Thank you everyone! > > >> > > > >> > --Matt > > >> > > > >> > [1]: > > >> > > > >> > https://docs.google.com/document/d/1zHbnyK1r6KHpMOtEdIg1EZKNzHx-MVgUMOzB87GuXyk/edit#heading=h.38515dnp2bdb > > >> > [2]: > https://lists.apache.org/thread/tn5wt4p52f6kqjtx3tjxqd9122n4pf94 > > >> > > > >
