Jan writes: > Can you say something about the root cause for solr processes to > crash? Are they killed by Linux?
They are usually not crashing, I "just" see shards go into down/recovery state. Some of the time they recover without intervention, some of the time a shard or two stays down, and only recover if I restart the affected Solr instance. Usually the load on the affected server is high and following the log files does show more GC-activity when this happens. > Which version of Java did you run on 18.04? We were running Java 11 on Ubuntu 18.04 as well, so that would have been 11.0.17+8-1ubuntu2~18.04. > Other changes done at the same time, such as OS-level settings for > ulimits, vm.max_map_count, swappiness etc. No other changes - just everything that the Ubuntu 18.04 to 20.04 upgrade did, including the Linux kernel version. Which makes it a little hard to pin down, so I was fishing after known Ubuntu 20.04 gotchas :-) > If you have not fine-tuned your JVM settings, it is recommended to run > with the default JVM/GC settings. G1 in latest Java-11 shoud be fine. A long time ago we adjusted according to the suggestions on https://cwiki.apache.org/confluence/display/solr/ShawnHeisey#ShawnHeisey-G1(GarbageFirst)Collector So we are running with: # These GC settings have shown to work well for a number of common Solr workloads GC_TUNE=" \ -XX:+UseG1GC \ -XX:+ParallelRefProcEnabled \ -XX:MaxGCPauseMillis=250 \ -XX:+UseLargePages \ -XX:AutoBoxCacheMax=20000 \ -XX:BiasedLockingStartupDelay=500 \ -XX:G1HeapRegionSize=32m \ -XX:InitiatingHeapOccupancyPercent=75 \ -XX:+HeapDumpOnOutOfMemoryError \ -XX:HeapDumpPath=/var/log/solr \ " and SOLR_HEAP="12g". Best regards, Adam -- "Tears in waves Adam Sjøgren Lights on fire" a...@koldfront.dk
