Hi All, As an effort to enhance disaster recovery for solr, we have started a solr backup process on a daily basis. The backup runs for each replica one after the other, after which an integrity check is executed to check if the index is having no faults.
Although, throughout the backup, we experience high io wait on production servers as complete data of 25 gb is being read [size of each shard is ~25 gb]. The backup executes daily at night 3 AM [backup for each replica runs sequentially] and write is done on a separate disc, still response time takes a significant hit, thereby increasing the number of timeouts and 5xx. Is there a way to limit the io so that backup is done at a slower pace keeping the response time and other metrics intact?