I also think a set of best practices for Arrow over HTTP would be a
valuable resource for the community...even if it never becomes a
specification of its own, it will be beneficial for API developers and
consumers of those APIs to have a place to look to understand how
Arrow can help improve throughput/latency/maybe other things. Possibly
something like httpbin.org but for requests/responses that use Arrow
would be helpful as well. Thank you Ian for leading this effort!
It has mostly been covered already, but in the (ubiquitous) situation
where a response contains some schema/table and some non-schema/table
information there is some tension between throughput (best served by a
JSON response plus one or more IPC stream responses) and latency (best
served by a single HTTP response? JSON? IPC with metadata/header?). In
addition to Antoine's list, I would add:
- How to serve the same table in multiple requests (e.g., to saturate
a network connection, or because separate worker nodes are generating
results anyway).
- How to inline a small schema/table into a single request with other
metadata (I have seen this done as base64-encoded IPC in JSON, but
perhaps there is a better way)
If anybody is interested in experimenting, I repurposed a previous
experiment I had as a flask app that can stream IPC to a client:
https://github.com/paleolimbot/2023-11-21_arrow-over-http-scratchpad/pull/1/files
.
- recommendations about compression
Just a note that there is also Content-Encoding: gzip (for consumers
like Arrow JS that don't currently support buffer compression but that
can leverage the facilities of the browser/http library)
Cheers!
-dewey
On Mon, Nov 20, 2023 at 8:30 PM Sutou Kouhei <k...@clear-code.com> wrote:
Hi,
But how is the performance?
It's faster than the original JSON based API.
I implemented Apache Arrow support for a C# client. So I
measured only with Apache Arrow C# but the Apache Arrow
based API is faster than JSON based API.
Have you measured the throughput of this approach to see
if it is comparable to using Flight SQL?
Sorry. I didn't measure the throughput. In the case, elapsed
time of one request/response pair is important than
throughput. And it was faster than JSON based API and enough
performance.
I couldn't compare to a Flight SQL based approach because
Groonga doesn't support Flight SQL yet.
Is this approach able to saturate a fast network
connection?
I think that we can't measure this with the Groonga case
because the Groonga case doesn't send data without
stopping. Here is one of request patterns:
1. Groonga has log data partitioned by day
2. Groonga does full text search against one partition (2023-11-01)
3. Groonga sends the result to client as Apache Arrow
streaming format record batches
4. Groonga does full text search against the next partition (2023-11-02)
5. Groonga sends the result to client as Apache Arrow
streaming format record batches
6. ...
In the case, the result data aren't always sending. (search
-> send -> search -> send -> ...) So it doesn't saturate a
fast network connection.
(3. and 4. can be parallel but it's not implemented yet.)
If we optimize this approach, this approach may be able to
saturate a fast network connection.
And what about the case in which the server wants to begin sending
batches
to the client before the total number of result batches / records is
known?
Ah, sorry. I forgot to explain the case. Groonga uses the
above approach for it.
- server should not return the result data in the body of a response
to a
query request; instead server should return a response body that gives
URI(s) at which clients can GET the result data
If we want to do this, the standard "Location" HTTP headers
may be suitable.
- transmit result data in chunks (Transfer-Encoding: chunked), with
recommendations about chunk size
Ah, sorry. I forgot to explain this case too. Groonga uses
"Transfer-Encoding: chunked". But recommended chunk size may
be case-by-case... If a server can produce enough data as
fast as possible, larger chunk size may be
faster. Otherwise, larger chunk size may be slower.
- support range requests (Accept-Range: bytes) to allow clients to
request
result ranges (or not?)
In the Groonga case, it's not supported. Because Groonga
drops the result after one request/response pair. Groonga
can't return only the specified range result after the
response is returned.
- recommendations about compression
In the case that network is the bottleneck, LZ4 or Zstandard
compression will improve total performance.
- recommendations about TCP receive window size
- recommendation to open multiple TCP connections on very fast networks
(e.g. >25 Gbps) where a CPU thread could be the throughput bottleneck
HTTP/3 may be better for these cases.
Thanks,
--
kou
In <CANa9GTHuXBBkn-=uevmbr2edmiyquunc6qdqdvh7gpeps9c...@mail.gmail.com>
"Re: [DISCUSS] Protocol for exchanging Arrow data over REST APIs" on
Sat, 18 Nov 2023 13:51:53 -0500,
Ian Cook <ianmc...@apache.org> wrote:
Hi Kou,
I think it is too early to make a specific proposal. I hope to use this
discussion to collect more information about existing approaches. If
several viable approaches emerge from this discussion, then I think we
should make a document listing them, like you suggest.
Thank you for the information about Groonga. This type of
straightforward
HTTP-based approach would work in the context of a REST API, as I
understand it.
But how is the performance? Have you measured the throughput of this
approach to see if it is comparable to using Flight SQL? Is this
approach
able to saturate a fast network connection?
And what about the case in which the server wants to begin sending
batches
to the client before the total number of result batches / records is
known?
Would this approach work in that case? I think so but I am not sure.
If this HTTP-based type of approach is sufficiently performant and it
works
in a sufficient proportion of the envisioned use cases, then perhaps
the
proposed spec / protocol could be based on this approach. If so, then
we
could refocus this discussion on which best practices to incorporate /
recommend, such as:
- server should not return the result data in the body of a response
to a
query request; instead server should return a response body that gives
URI(s) at which clients can GET the result data
- transmit result data in chunks (Transfer-Encoding: chunked), with
recommendations about chunk size
- support range requests (Accept-Range: bytes) to allow clients to
request
result ranges (or not?)
- recommendations about compression
- recommendations about TCP receive window size
- recommendation to open multiple TCP connections on very fast networks
(e.g. >25 Gbps) where a CPU thread could be the throughput bottleneck
On the other hand, if the performance and functionality of this
HTTP-based
type of approach is not sufficient, then we might consider
fundamentally
different approaches.
Ian
