Nikolay, This sounds like a philosophical question :)
By rigid I mean inability to change some properties without whole cluster restart, and I do think it's bad. I have a good example in mind - the number of threads running rebalance. Originally, we added a validation check for this configuration, so it was impossible to change it without cluster restart. There was a user who was running a cluster; the volume of data grew and afterwards they needed to increase rebalance speed. But they could not do this. Once we removed the validation and added proper handling of different rebalance threads number, the user managed to restart nodes one-by-one, wait for rebalance and keep the cluster up and running. So I think the general direction should be handle automatically as much as possible and make behavior consistent, rather than introduce additional restrictions.
