Hello elementary friends, As many of you know, devs are now in the process of developing the next release of elementary, Isis. During various conversations there’s this one subject that has risen quite often - how can testing be incorporated in the process.
For the impatient ones, jump right away to the instructions at [1]. Now, before some anxious folks jump ahead, let me stop you all and say it's NOT about TDD. We've been there, done the discussions, and for now it was decided that no one is doing that [2]. What I am going to talk about now is system/integration level testing, that is the curse called QE [3]. QE is somewhat unloved, seems dark and boring among and by many developers, especially in open source. Mainly due to the fact, that in open source we consider our users to be our testers. While it could be working well enough for various libraries, and different software packages, I claim it cannot work for users of the entire OS/DE, especially the one that aspires to provide the best and most polished experience for its users. We cannot expect high level of technical capabilities from our users to understand what exactly is the problem they experience, let alone to report bugs they find. As such I had few private discussions with members of the elementary dev teams, and came with to the following assumptions: 1. Nobody will be forced to do TDD. IF (and that's a big IF) someone wants to do it, they can do it on their own expense. 2. Most elementary applications are not completely new inventions. They are just new applications that are based on the mostly stable sets of libraries, which have their own projects, boards and testing methods. 3. External dependencies are just expected to work, and we/users will post bugs if they don't. 4. In system/integration 'testing' it is interesting to see that our applications work; that is the other technologies are integrated correctly and a good working interface is created. 5. Any type of automatic testing for the apps is fine, as long as it works. 6. System Testing can be done in any language. Finally, as I am mostly familiar with Python, I thought I'd check the autopilot framework, used by Ubuntu to test their entire system. Hereby I wanted to tell that first preliminary testing is done. I've successfully installed autopilot on my Isis dev build box, adopted one set of tests [4] and they now pass. That means, that technically it is possible to use autopilot for testing our applications. So... to read how to run the tests on your own, read the How-To [1] ==== What next? ==== So here we are now, with one part of elementary that can be tested automatically. I don't like to fool anyone nor myself with claims that now we have implemented testing. So here few possible tasks that I think should be done next to move with this stuff: 1. One possible task is to create more tests for pantheon-terminal. I'm sure its developers will be swayed with joy. 2. Another possible task is to create some small set of tests for another app. I'm sure that the elementary dev/users community can come up with awesome ideas as to what can be done with this. I'm all ears now and open to suggestions. So long, and remember, I talk too much, and I'm a crappy developer. Any complaints are welcome in sane limits. Cheers. [1] https://docs.google.com/document/d/1AHDT0hR8mjVcyhERsR7qC5Zs1GB1SNfUFHR6gFuJVTM/edit?usp=sharing [2] Unless someone does it, maintains it and keeps doing it until someone else does it too. [3] Quality Engineering (once called Quality Assurance, but that's not fare, as it somewhat equates quality engineers to robots. They are not. Actually, they are very talented engineers, and if one considers that their work is to do whatever there is in their power to break stuff, one could probably like it too). [4] Tests for pantheon-terminal were adopted from gnome-terminal and pass completely (9 tests). The code for the tests is at https://github.com/alourie/pantheon_terminal<http://unity.ubuntu.com/autopilot/faq/faq.html> <http://unity.ubuntu.com/autopilot/faq/faq.html> <http://unity.ubuntu.com/autopilot/faq/faq.html> -- Alex Lourie
-- Mailing list: https://launchpad.net/~elementary-dev-community Post to : elementary-dev-community@lists.launchpad.net Unsubscribe : https://launchpad.net/~elementary-dev-community More help : https://help.launchpad.net/ListHelp
