t after one request/response pair. Groonga
> can't return only the specified range result after the
> response is returned.
>
> > - recommendations about compression
>
> In the case that network is the bottleneck, LZ4 or Zstandard
> compression will improve total perfo
ecommendations about TCP receive window size
> - recommendation to open multiple TCP connections on very fast networks
> (e.g. >25 Gbps) where a CPU thread could be the throughput bottleneck
HTTP/3 may be better for these cases.
Thanks,
--
kou
In
"Re: [DISCUSS] Protocol for exc
Thank you Kou, Gavin, David, Antoine, and Raphael.
It sounds like there is agreement on the following:
- There is broad interest in the topic of how best to transfer Arrow data
over HTTP.
- Focusing only on REST-style APIs is too limiting; we should scope this to
be about HTTP more broadly. (This
I really like the idea of leveraging the mature ecosystem support for
IPC streams [1] to provide a set of conventions for sending and
receiving arrow data over plain HTTP.
For context, myself and my colleagues have run into a number of pain
points whilst working on FlightSQL:
- The additiona
I also agree that an informal spec "how to efficiently transfer Arrow
data over HTTP" makes sense.
Probably with several aspects:
- one-shot GET data
- streaming GET
- one-shot PUT or POST
- streaming POST
- non-Arrow prologue and epilogue (for example JSON-based metadata)
- conventions for w
I'm with Kou: what exactly are we trying to specify?
- The HTTP mapping of Flight RPC?
- A full, locked down RPC framework like Flight RPC, but otherwise unrelated?
- Something else?
I'd also ask: do we need to specify anything in the first place? What is
stopping people from using Arrow in thei
I know that myself and a number of folks I work with would be interested in
this.
gRPC is a bit of a barrier for a lot of services.
Having a spec for doing Arrow over HTTP API's would be solid.
In my opinion, it doesn't necessarily need to be REST-ful.
Something like JSON-RPC might fit well with
Hi Kou,
I think it is too early to make a specific proposal. I hope to use this
discussion to collect more information about existing approaches. If
several viable approaches emerge from this discussion, then I think we
should make a document listing them, like you suggest.
Thank you for the info
ferent schema.
* A client reads them by creating multiple record batch
stream readers. Python example:
https://github.com/hhatto/poyonga/blob/b8c2a2ba9fdbb8250d1d3d4db64137a9781fb7b7/poyonga/result.py#L97-L101
Thanks,
--
kou
In
"[DISCUSS] Protocol for exchanging Arrow da
Several recent discussions have highlighted the lack of an established
specification / protocol for sending Arrow-formatted data through REST
APIs. I would like to start a discussion here to gauge interest and gather
ideas about this.
For background:
Flight RPC provides a framework for building R
10 matches
Mail list logo
