Re: Inquiry on Using RecordBatchStreamWriter/RecordBatchStreamReader for Network Transmission

2025-01-21 Thread Aldrin
Hello! Can I depend on these interfaces to leverage Arrow format as binary exchange mechanism over HTTP?Yes. You can see [1] for a bit of discussion and some github links. But, the short answer is that the stream writer and stream reader interfaces are convenient interfaces to data movement over

Re: Inquiry on Using RecordBatchStreamWriter/RecordBatchStreamReader for Network Transmission

2025-01-21 Thread David Li
See apache/arrow-experiments [1] where there are many examples in multiple languages of doing just that, and more complicated examples of things like mixing Arrow and JSON data in a stream. (1): Yes, that's the intent of Arrow IPC! (2): I don't think there's anything glaring off the top of my he

Inquiry on Using RecordBatchStreamWriter/RecordBatchStreamReader for Network Transmission

2025-01-21 Thread Amarinder Randhawa
Hello Arrow developers I’m exploring the use of RecordBatchStreamWriter and RecordBatchStreamReader to serialize/deserialize Arrow data, over HTTP, as binary data payload. My assumption is that these methods provide a machine-independent serialization mechanism, allowing the Arrow format over HTTP

Re: [DISCUSS] Monorepo GitHub workflow: allow one issue with multiple PRs

2025-01-21 Thread Jacob Wujciak
> I am pro resolving the 1:1 relation between PRs and issues dissolving* Am Mi., 22. Jan. 2025 um 00:02 Uhr schrieb Jacob Wujciak : > > Thanks for restarting this discussion. In general I am pro resolving > the 1:1 relation between PRs and issues. At the moment I don't have a > firm view on what

Re: [DISCUSS] Monorepo GitHub workflow: allow one issue with multiple PRs

2025-01-21 Thread Jacob Wujciak
Thanks for restarting this discussion. In general I am pro resolving the 1:1 relation between PRs and issues. At the moment I don't have a firm view on what should replace it as a source of truth for changelogs etc. Though I am leaning towards using PRs/commits for that with issues just being used

Re: [DISCUSS] Monorepo GitHub workflow: allow one issue with multiple PRs

2025-01-21 Thread Bryce Mecum
I think we should allow multiple PRs for any issue in order to reduce friction for contributors. I remembered this thread while looking at the issue for Decimal32/64 support [1] which I think is a good example of where filing separate issues for every patch doesn't add value and adds friction. Som