florinbabes commented on PR #2030: URL: https://github.com/apache/solr/pull/2030#issuecomment-1784170646
> Before proceeding with the code review, let me ask an additional question: How is the size affecting the current implementation? Zookeeper on heap/off-heap memory? Zookeeper disk? Apache Solr on heap/off-heap memory? Apache Solr query time performance? > > Depending on the answers to these questions I would could be in favour of forcing the compact storage (if it's beneficial for performance significantly). In terms of the configurability discussion I believe it could be an over-complication, if it's for human readability I suspect it would be quite a quick process for a human to get the JSON and put it on some online editor such as jsonlint.com? Let's also remember that the limit in Zookeeper is currently configurable, but I agree it often causes headaches. And it comes without saying, thank you very much for the contribution @holysleeper and thanks @cpoerschke for the heads up! Hello @alessandrobenedetti. The current implementation is affecting the off-heap Zookeeper memory size. Currently if we have 2 models each of 20MB and we want to add another one, zookeeper will use a lot of off-heap memory and it could go OOM (our container orchestrator will kill the container). Even if we have the file size limit jute.maxbuffer set to 536870912 and ZK heap to 4GB. The improvement in current PR will allow us to upload more models using less memory for ZK. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
