On Mon, Nov 13, 2017 at 6:23 AM, sam reddy <[email protected]> wrote:
> We are building a grpc service in python which has a bidirectional > streaming endpoint and also a unary endpoint. > > we want the stream to live forever so we have no timeouts and streams are > working as expected, we are using kubernates and docker for deployment. > > But we are facing issues with scaling the service, how to scale an > infinite streaming grpc server, we can't scale based on a number of > requests but there is only one request made and data is sent as frames. > > how can we scale this service? right now in worker thread pool has max > threads as 100. > > one quick solution is to give max worker threads a higher number and scale > based on CPU load and memory usage. > > is there any better way to do it. > (This response also posted over here <https://stackoverflow.com/questions/47263294/grpc-server-scaling-bidirectional-infinite-streaming/47318936#47318936> .) Right now we don't have a good answer: the thread-per-RPC assumption was baked into gRPC Python fairly early and deeply, well before we were aware of "just keep an open connection in case either side has anything to say" long-lived RPCs being a use case. We're working on better solutions but they'll likely be a while in coming. Increasing the number of worker threads definitely sounds like the right answer for the time being. I'd be very curious to hear how it works out since your threads will be mostly idle most of the time (right?). An option to maybe try that might work out well would be to design an object that implements the interface of futures.ThreadPoolExecutor but that actually does some sophisticated internal multiplexing to service a great many more RPCs. It's an idea that I've had on my mind for a while <https://github.com/grpc/grpc/issues/7632> but haven't gotten around to testing out myself. -Nathaniel -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAEOYnARQ2fi6xym08LZeVpnkZayn5ZS0j7DMfkfkhWVCMbk_Mw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
smime.p7s
Description: S/MIME Cryptographic Signature
