Hello there,
I have a java server (jetty, jersey) which produces a streaming arrow response (via a javax.ws.rs.core.StreamingOutput). The stream of arrow is produced ‘on the fly’ and occasionally something goes wrong once we’ve already begun streaming. In this case we throw an exception which propagates up to jersey, which then closes the response normally (there seems to be no good way to get jersey to hand the exception back up to jetty to cause an abort on the response stream). We’re sending whole arrow batches, so the client just sees a regular arrow stream (i.e. the stream up to and including the last entire arrow row batch we sent) and so has no way to detect that an error occurred. Maybe there is actually a way to get jetty to hang up the connection without proper termination (which the client can then detect) – although this does mean we have no ability to indicate why the error happened – but we have so far been unable to figure out how to do this. Another thought was having some way in a streaming arrow response to indicate this kind of error inline in the stream/be able to ‘poison’ an arrow stream – to indicate to the client that something exceptional happened part way through the streaming response. Maybe some way of sending an EOS with an error flag set. There are probably other, arguably more elegant ways to handle this kind of thing e.g. if we can assume http2 everywhere something like https://http2.github.io/http2-spec/#StreamErrorHandler but it seems currently the tooling isn’t really there/at any rate it could be nice to have a transport-independent way of indicating an error. I was wondering if anyone has dealt with anything similar to this/has any pointers/thoughts on whether adding to the streaming format makes sense here? Best, Ben
smime.p7s
Description: S/MIME cryptographic signature
