Hey David, I was actually testing test_flight.test_http_basic_auth(). But I think the same applies. The Java default implementation expects a handshake. More to the point it expects a BasicAuth protobuf which I believe is not exposed at all in python. Always returning true in BasicServerAuthHandler.authenticate() allows for the test implementations of Java and Python authentication to speak to each other.
Thanks for the below link, that really clarified things for me. I would add to the list that we should normalise the use of BasicAuth protobuf message between java and cpp. Apologies for the urgency yesterday, I am glad it was sorted and more my fault than Arrow's. Best, Ryan On Thu, Jul 4, 2019 at 11:29 AM David Li <li.david...@gmail.com> wrote: > Hmm, interesting. I assume you mean test_flight.test_token_auth() as > the client? The tests weren't written to be explicitly compatible, but > there's no reason you should get an indefinite stall. > > We don't use Handshake/ServerAuthHandler#authenticate, so that would > explain why we don't see issues. I commented on this in the initial > implementation: > https://github.com/apache/arrow/pull/4125#discussion_r273935691 > > > there is not a 1-1 mapping between connected clients and connected > peers, and so you can > > only know the identity of the peer at the moment it makes a call. Just > doing a handshake > > (Authenticate) isn't enough to identify who is on the other end of a > particular connection. > > the reason being that a layer 7 load balancer (e.g. Envoy) means one > gRPC connection can represent multiple clients. Conversely, > client-side load balancing (built into gRPC) means one client-side > "connection" can actually represent multiple servers. Of course, you > have to consciously deploy in this manner, so Handshake is still > useful if you know you won't ever need these features. > > As I see it, this means there's a few things to work on: > - Flight RPC feature compatibility needs to be tested, not just format > compatibility. > - We should start thinking about async APIs and/or timeouts in any > sort of API that makes a network call (there's already a JIRA: > https://issues.apache.org/jira/browse/ARROW-1009), since "never > returns" is a terrible failure mode > > Best, > David > > On 7/4/19, Ryan Murray <rym...@dremio.com> wrote: > > Hey David, > > > > I am curious to see what you are doing different from me. I am running > the > > Java ExampleFlightServer.java against the python auth flight tests and > they > > are not passing. The particular issue is that incoming.next() never > returns > > in BasicServerAuthHandler.java:56 > > > > It doesn't appear to be anything wrong w/ the auth piece specifically > > rather the server appears to not be getting the auth info to verify. I am > > still investigating my issue but I am glad that someone else has gotten > > this to work. > > > > Best, > > Ryan > > > > On Thu, Jul 4, 2019 at 9:13 AM Antoine Pitrou <anto...@python.org> > wrote: > > > >> > >> It may be worth opening a JIRA for the flaky tests if not already done. > >> > >> Regards > >> > >> Antoine. > >> > >> > >> Le 04/07/2019 à 18:11, David Li a écrit : > >> > I'm also curious as to what the issue was, as we've been doing > >> > Python-client-Java-server auth with development builds without > >> > trouble. > >> > > >> > Regardless - this does point out a need for more cross-language Flight > >> > testing (perhaps a Flight-specific integration suite?), and to get > >> > existing tests running more consistently in CI (Flight/Java in > >> > particular has a lot of flaky tests, though the auth tests are enabled > >> > in Travis). > >> > > >> > Best, > >> > David > >> > > >> > On 7/4/19, Jacques Nadeau <jacq...@apache.org> wrote: > >> >> Which is exactly why I was withholding a vote until there was more > >> >> information. > >> >> > >> >> On Thu, Jul 4, 2019, 7:25 AM Antoine Pitrou <solip...@pitrou.net> > >> wrote: > >> >> > >> >>> On Thu, 4 Jul 2019 09:04:34 -0500 > >> >>> Wes McKinney <wesmck...@gmail.com> wrote: > >> >>>> > >> >>>> That being said, with Ryan's issue, he is using a feature > >> >>>> (cross-language auth in Flight) that isn't being tested. The Flight > >> >>>> integration tests do not use authentication AFAIK so I'm not > >> >>>> surprised > >> >>>> to hear that there may be an issue with it. > >> >>> > >> >>> OTOH, it's a bit unlikely. Flight authentication is implemented is: > >> >>> - the Arrow codebase simply passes opaque tokens around > >> >>> - interpretation of tokens is handled by application code > >> >>> - marshalling of tokens is handled by Protocol Buffers > >> >>> > >> >>> So unless something silly is going on (such as "passing an empty > >> >>> string > >> >>> instead of the actual token") there's not much potential for > >> >>> auth interoperability issues in the core Flight codebase. > >> >>> > >> >>> Regards > >> >>> > >> >>> Antoine. > >> >>> > >> >>> > >> >>> > >> >> > >> > > > > > > -- > > > > Ryan Murray | Principal Consulting Engineer > > > > +447540852009 | rym...@dremio.com > > > > <https://www.dremio.com/> > > Check out our GitHub <https://www.github.com/dremio>, join our community > > site <https://community.dremio.com/> & Download Dremio > > <https://www.dremio.com/download> > > > -- Ryan Murray | Principal Consulting Engineer +447540852009 | rym...@dremio.com <https://www.dremio.com/> Check out our GitHub <https://www.github.com/dremio>, join our community site <https://community.dremio.com/> & Download Dremio <https://www.dremio.com/download> I
