Would you be able to share the server’s getStream() and getFlightInfo() implementations?
Note that getStream() needs should be written such that it doesn’t block the grpc thread. Get Outlook for Mac <https://aka.ms/GetOutlookForMac> From: Susmit Sarkar <susmitsir...@gmail.com> Date: Wednesday, April 30, 2025 at 2:59 AM To: David Li <lidav...@apache.org> Cc: nik.9...@gmail.com <nik.9...@gmail.com>, dev@arrow.apache.org <dev@arrow.apache.org> Subject: Re: Query on stuck Arrow Flight Client while interacting from local workstation (mac) Hi David Sharing the arrow client thread dump for reference. Strangely if we pass a dummy non existent s3 path we are getting proper error from server cef_flight_server.exceptions.S3AccessError: Failed to access S3: [Errno 2] Path does not exist 'bg0975-cef-ccmedev-data/pp/load_date=2024-11-21/part-00007.c008.snappy.parquet'. Detail: [errno 2] No such file or directory Which translates the server is reachable and we do see the logs in server as well It works fine if we call the client within a VM issue arises in local workstation, where its stuck indefinitely. Thanks, Susmit On Wed, Apr 30, 2025 at 12:54 PM David Li <lidav...@apache.org<mailto:lidav...@apache.org>> wrote: This is not specific to Flight; use jstack or your favorite instrumentation tool (VisualVM etc.) On Wed, Apr 30, 2025, at 15:41, NIKHIL RANJAN wrote: > Hi David, > > How to enable thread dump logs for both client and server code. > > As of now, I don't see any error on either client side or server side. It > just hangs/gets stuck. > > Thanks, > Nikhil > > On Thu, 24 Apr, 2025, 14:39 Susmit Sarkar, > <susmitsir...@gmail.com<mailto:susmitsir...@gmail.com>> wrote: > >> Hi Team, >> >> We are using this below code snippet in Scala to query the flight server, >> but seems to be stuck indefinitely, this issue is seen when we are testing >> from our local workstation (Mac to be precise) >> >> Another interesting thing, it's able to propagate the error message >> correctly but not the FlightStream data, the same code works fine when we >> run inside a linux VM. >> >> Do you folks see any issue in the code? >> >> def fetchDataStreamIterator(details: BaseDataAccessDetails): >> Iterator[FlightStream] = { >> logger.info<http://logger.info>(s"Fetching data for details: >> ${details.toString}") >> val ticketStr = buildTicketStr(details) >> logger.info<http://logger.info>(s"Generated ticket string: $ticketStr") >> >> val allocator = new RootAllocator(Long.MaxValue) >> val client = FlightClient.builder(allocator, >> Location.forGrpcInsecure(serverHost, serverPort)).build() >> >> try { >> val ticket = new Ticket(ticketStr.getBytes(StandardCharsets.UTF_8)) >> val stream = client.getStream(ticket) >> >> Iterator.continually { >> if (stream.next()) Some(stream) else { >> // Cleanup when no more batches >> close(stream, client) >> None >> } >> }.takeWhile(_.isDefined).flatten >> } catch { >> case e: FlightRuntimeException => >> logger.error(s"Error communicating with Flight server: >> ${e.getMessage}") >> throw new CefFlightServerException(s"Error communicating with Flight >> server: ${e.getMessage}", e) >> case e: Exception => >> logger.error(s"Failed to fetch data: ${e.getMessage}") >> throw new CefFlightServerException("Failed to fetch data from Flight >> Server", e) >> } >> } >> >> >> Thanks, >> >> Susmit >> >> Warning: The sender of this message could not be validated and may not be the actual sender.
