Hi, Rene. That sounds great, chaos engineering and controlled experiments are great tools, smoke tests improvements are also subject of discussion and I think that UAT (user acceptance tests) suite should be designed finally. It's important to a business to have one. It's important to the community to have one, otherwise quality can not be expected. I think the policy should be developed which forces and motivates movement toward the direction observed.
13 дек. 2017 г. 19:59 пользователь "Rene Moser" <m...@renemoser.net> написал: > Hi all > > On 12/13/2017 05:04 AM, Ivan Kudryavtsev wrote: > > Hello, devs, users, Rohit. Have a good day. > > > > Rohit, you intend to freeze 4.11 on 8 january and, frankly speaking, I > see > > risks here. A major risk is that 4.10 is too buggy and it seems nobody > uses > > it actually right now in production because it's unusable, unfortunately, > > so we are planning to freeze 4.11 which stands on untested 4.10 with a > lot > > of lacks still undiscovered and not reported. I believe it's a very > > dangerous way to release one more release with bad quality. Actually, > > marvin and units don't cover regressions I meet in 4.10. Ok, let's take a > > look at new one our engineers found today in 4.10: > > So, the point is, how do we (users, devs, all) improve quality? > > Marvin is great for smoke testing but CloudStack is dealing with many > infra vendor components, which are not covered by the tests. How can we > detect flows not covered by marvin? > > For me, I decided (independent of this discussion) to write integration > tests in a way one would not expect, not following the "happy path": > > Try to break CloudStack, to make a better CloudStack. > > Put a chaos monkey in your test infra: Shut down storage, kill a host, > put latency on storage, disable network on hosts, make load on a host. > read only fs on a cluster wide primary fs. shut down a VR, remove a VR. > > Things that can happen! > > Not surprisingly I use Ansible. It has an extensive amount of modules > which can be used to battle prove anything of your infra. Ansible > playbooks are fairly easy to write, even when you are not used to write > code. > > I will share my works when ready. > > René > > > > > >
