Re: Thoughts from a former KDE, currently Mozilla developer
On Wednesday 8 December 2010 02:51:46 Aaron J. Seigo wrote: > On Tuesday, December 7, 2010, Benoit Jacob wrote: > > * > > 3. QA, Unit-testing, continuous testing (KDE: very little, Mozilla: a > > lot) > > > > Let's face it, KDE has always been deficient in the QA department. In > > > > order to ensure good QA, it needs: > > * a lot more unit tests than it currently has > > agreed. care to donate some resources? :) > > > * unit tests must be automatically run, frequently. > > agreed; people are working on automated unit test set ups for kde. we have > > more variance than mozilla does however, which means this: > > In mozilla, > > > > everything is built and the unit tests run on 10 configs everytime you > > push to the repository, see http://tbpl.mozilla.org/ > > would be great. given our resources and the variety of our targets, highly > unlikely for now. in future, perhaps. > > > * regressions must never be tolerated. if a commit causes a > > > > regression, back it out. > > ah, blanket statements. as a rule of thumb: good idea. rigidly followed: > stupid, as it's an awesome way to discourage development contribution. Note that it piggy back to unit tests IMO. If a commit causes a regression caught by a unit test it should be reverted indeed. If that happens it's mainly a problem during the reviewing phase and so on. Going beyond than that could indeed discourage development contribution (unit tests are easy enough to run before you send the patch to be a good compromise). My 0.02€. Regards. -- Kévin Ottens, http://ervin.ipsquad.net KDAB - proud patron of KDE, http://www.kdab.com signature.asc Description: This is a digitally signed message part. >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
Re: Thoughts from a former KDE, currently Mozilla developer
On Wed, December 8, 2010 8:38 am, Kevin Ottens wrote: > On Wednesday 8 December 2010 02:51:46 Aaron J. Seigo wrote: >> On Tuesday, December 7, 2010, Benoit Jacob wrote: >> > * >> > 3. QA, Unit-testing, continuous testing (KDE: very little, Mozilla: a >> > lot) >> > >> > Let's face it, KDE has always been deficient in the QA department. In >> > >> > order to ensure good QA, it needs: >> > * a lot more unit tests than it currently has >> >> agreed. care to donate some resources? :) >> >> > * unit tests must be automatically run, frequently. >> >> agreed; people are working on automated unit test set ups for kde. we >> have >> >> more variance than mozilla does however, which means this: >> > In mozilla, >> > >> > everything is built and the unit tests run on 10 configs everytime you >> > push to the repository, see http://tbpl.mozilla.org/ >> >> would be great. given our resources and the variety of our targets, >> highly unlikely for now. in future, perhaps. >> >> > * regressions must never be tolerated. if a commit causes a >> > regression, back it out. >> >> ah, blanket statements. as a rule of thumb: good idea. rigidly followed: >> stupid, as it's an awesome way to discourage development contribution. > > Note that it piggy back to unit tests IMO. If a commit causes a regression > caught by a unit test it should be reverted indeed. If that happens it's > mainly a problem during the reviewing phase and so on. Going beyond than > that > could indeed discourage development contribution (unit tests are easy > enough to run before you send the patch to be a good compromise). For smaller commits, it seems reasonable to demand that they don't cause regressions. For larger commits, it won't always be possible to test sufficiently for regressions before committing - there may be too many affected apps if a change is made in the libraries, and a single developer can't be expected to have the time or knowledge to test multiple apps. This may apply much less to Mozilla than it does to KDE. Another important consideration is that major changes being developed will often have to play catch-up before they are committed, the patches having to be modified frequently as changes are made to the libraries and applications affected. To insist that no regressions occur would mean that such patches might never be committed, by part-time developers at least. As long as there is time for testing before the next KDE release, most regressions can be caught afterwards, and that's probably the best that can be hoped for. -- David Jarvie. KDE developer. KAlarm author - http://www.astrojar.org.uk/kalarm >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
Re: Thoughts from a former KDE, currently Mozilla developer
2010/12/7 Aaron J. Seigo : >> * >> 2. Can one tell volunteers contributors what they should be working >> on? (KDE: no, Mozilla: yes) > > a nice bit of philosophy. imho: > > should we communicate what is needed to be worked on? yes. > should we make it "cool" to work on what is needed by offering support and > positive reinforcement? yes. > should we tollerate those who refuse to do so? yes. > do we always know what is needed? no. > can we always know what is needed? no. > > several projects within kde do engage in both guidance as well as group agenda > setting. in that specific way, it's not quite the kde of 5 years ago. > > "However, KDE could do more to communicate what actually needs to be done" > > do you have some concrete applicable suggestions? Some ideas. You could introduce a notion of 'blocking bug' where if such a bug isn't fixed in time for the release, the buggy feature must be dropped. This allows to keep the time-based schedule. Example: if a feature makes an app crash on startup for a significant number of users, that should block it. If a feature is only half-baked, that fact is a bug (user disappointment) that should block it. You could then regularly advertise the remaining list of blocking bugs, and at some late point in the release process, you could accept only commits that fix blocking bugs. >> * >> 3. QA, Unit-testing, continuous testing (KDE: very little, Mozilla: a lot) >> >> Let's face it, KDE has always been deficient in the QA department. In >> order to ensure good QA, it needs: >> * a lot more unit tests than it currently has > > agreed. care to donate some resources? :) Sorry, I'm too busy to do anything but being a self-appointed advice giver. > and in some ways, Mozilla has a far, far easier task and so can benefit from > things that we can probably only dream about. :/ The only thing here (crash reports) that makes Mozilla's task far easier is that it distributes binaries. If really that's necessary to get good crash reports on linux, then it should still be possible for you to get the same by working with distros (they are your provider of binaries). >> top-class experience; but I hope very, very much that you keep KHTML >> around, possibly as an optional non-default-install component. Indeed, > > that isn't up to "us", that's up to the KHTML team. they continue to work on > it, and as long as they do then it will remain around, probably in future as > an optional component as you described. > >> you'll get new people into KDE development, second you might find that >> in several years KHTML is revived. I estimate that 50 dedicated >> developers people is the minimum to get a modern web engines going. > > you know what they call someone who does the same thing over and over > expecting the results to be different? I don't think that's doing the same thing over and over again. KHTML has been a neglected part of KDE for as long as WebKit started to look like the obvious choice. I realize that's up to the KHTML developers but was hoping to reach them too on this list. Guys, even if WebKit does displace KHTML as KDE's main/default/standard web engine, your own project remains a huge opportunity for prospective contributors. So why don't you blog and try to attract new contributors? Why keep passively waiting as KHTML gets more and more forgotten? Note: maybe I missed recent developments, I would love to be proven wrong. >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
Re: Thoughts from a former KDE, currently Mozilla developer
On Wednesday 8 December 2010 15:09:38 David Jarvie wrote: > On Wed, December 8, 2010 8:38 am, Kevin Ottens wrote: > > On Wednesday 8 December 2010 02:51:46 Aaron J. Seigo wrote: > >> On Tuesday, December 7, 2010, Benoit Jacob wrote: > >> > * > >> > 3. QA, Unit-testing, continuous testing (KDE: very little, Mozilla: a > >> > lot) > >> > > >> > Let's face it, KDE has always been deficient in the QA department. In > >> > > >> > order to ensure good QA, it needs: > >> > * a lot more unit tests than it currently has > >> > >> agreed. care to donate some resources? :) > >> > >> > * unit tests must be automatically run, frequently. > >> > >> agreed; people are working on automated unit test set ups for kde. we > >> have > >> > >> more variance than mozilla does however, which means this: > >> > In mozilla, > >> > > >> > everything is built and the unit tests run on 10 configs everytime you > >> > push to the repository, see http://tbpl.mozilla.org/ > >> > >> would be great. given our resources and the variety of our targets, > >> highly unlikely for now. in future, perhaps. > >> > >> > * regressions must never be tolerated. if a commit causes a > >> > > >> > regression, back it out. > >> > >> ah, blanket statements. as a rule of thumb: good idea. rigidly followed: > >> stupid, as it's an awesome way to discourage development contribution. > > > > Note that it piggy back to unit tests IMO. If a commit causes a > > regression caught by a unit test it should be reverted indeed. If that > > happens it's mainly a problem during the reviewing phase and so on. > > Going beyond than that > > could indeed discourage development contribution (unit tests are easy > > enough to run before you send the patch to be a good compromise). > > For smaller commits, it seems reasonable to demand that they don't cause > regressions. For larger commits, it won't always be possible to test > sufficiently for regressions before committing - there may be too many > affected apps if a change is made in the libraries, and a single developer > can't be expected to have the time or knowledge to test multiple apps. > [...] I didn't claim that. My point is that at least the unit tests shouldn't show any regressions, I never talked about getting to each application and test it manually. Really, running the automated tests before submitting is a low enough price to pay, but would improve the quality quite a bit. Regards. -- Kévin Ottens, http://ervin.ipsquad.net KDAB - proud patron of KDE, http://www.kdab.com signature.asc Description: This is a digitally signed message part. >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
Re: Thoughts from a former KDE, currently Mozilla developer
2010/12/8 Kevin Ottens : >> For smaller commits, it seems reasonable to demand that they don't cause >> regressions. For larger commits, it won't always be possible to test >> sufficiently for regressions before committing - there may be too many >> affected apps if a change is made in the libraries, and a single developer >> can't be expected to have the time or knowledge to test multiple apps. >> [...] > > I didn't claim that. My point is that at least the unit tests shouldn't show > any regressions, I never talked about getting to each application and test it > manually. > > Really, running the automated tests before submitting is a low enough price to > pay, but would improve the quality quite a bit. Also, you could have a Try-server: a Git repository to which people can freely push changesets, with a hook so that changes are automatically built and tested on various platforms, before they push to the actual 'for real' git repository. See: https://wiki.mozilla.org/ReleaseEngineering/TryServer and the results: http://tbpl.mozilla.org/?tree=MozillaTry Benoit >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
Re: Thoughts from a former KDE, currently Mozilla developer
2010/12/7 Andreas Pakulat : > On 07.12.10 14:28:57, Benoit Jacob wrote: >> If I want to get a list of all KDE crashes that happened in libGL.so*, >> how do I go about that? > > Thats more a problem of the tool we're using to track bugs, there's no > way to tell bugzilla that something is a backtrace and what format it is > in so that one can do more than a plain-text search on it. Right, so here we find a concrete thing to fix: crash reports should not land into bugzilla, nor should they land in any issue tracker, rather they should land into a specialized database with a specialized web interface. > Also, with > enough debug symbols you won't be able to search for libGL anyway as > the backtrace has references to source files only. How do you think that debuggers know the file? I think they just use the modules map for the process, e.g. on linux it's /proc/$PID/maps, that allows to find the object filename from the address. Try this: cat /proc/`pidof kwin`/maps > >> Regarding backtraces, as I said in my next email, it's not realistic >> to expect the user to do anything more than click OK, when an app >> crashed on him. > > I'm sorry, but I as a developer don't want to work with such users > anyway. If a user isn't able or willing to provide information what he > did for the crash to occur or tell me wether its reproduceable then I > don't want to spend time on helping him. On the other hand, if the user > is able/willing to do the above, he surely is able/willing to open his > package manager and select a package or two (where the name is suggested > by dr. konqi already) and then hit the reload button in dr. konqi. Even if the x % of users who do this give you enough crash reports to handle, you still lose the other aspect of crash reports: statistics. It's just very useful to know the top 100 crash locations in an app, and how crashiness changes over time. Benoit >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
Re: Thoughts from a former KDE, currently Mozilla developer
On Tue, Dec 7, 2010 at 3:01 PM, Thiago Macieira wrote: > On Tuesday, 7 de December de 2010 20:28:57 Benoit Jacob wrote: >> So do crash reports now automatically land in a database that's easy >> to do queries on? (See the web interface provided by Socorro) >> >> If I want to get a list of all KDE crashes that happened in libGL.so*, >> how do I go about that? >> >> Regarding backtraces, as I said in my next email, it's not realistic >> to expect the user to do anything more than click OK, when an app >> crashed on him. So I think that the -dbg packages rather need to be >> used by the server. > > Server-side debug symbols implies uploading the core file to the server, which > may contain a variety of interesting information like user passwords, credit > card numbers, etc. Imagine if kwalletd crashed even. > > Not to mention that some core files are quite big to upload and store. This may be totally infeasible, but what if you did client-side debugging, but used debugging symbols stored on a server? The debugger would grab the parts of the debugging symbols it needs from the server, do the analysis locally, then submit the results. That way there would be no security risk, but people wouldn't have to keep the debugging symbols installed. -Todd >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
Re: Thoughts from a former KDE, currently Mozilla developer
Em Terça-feira, 7 de Dezembro de 2010, às 11:22:47, Benoit Jacob escreveu: > * > 1. Issue tracker (Mozilla) vs. Mailing lists (KDE). > > KDE uses a Mailing list for most its development discussion, and only > uses trackers (bugzilla, reviewboard) for specific tasks (user bug > reports, patch review). > > Mozilla uses a (modern, well-configured) bugzilla for almost > everything including 90% of development discussion. > > My point here is not to advertise bugzilla specifically (though I love > it), but issue trackers in general. > > An issue tracker, contrary to a mailing list, allows to efficiently > keep track of what TODO items remain to be done, how they must be > prioritized, etc. It also is much more convenient for core > contributors because it allows them to handle issues later, so they > can reclaim control of how they spend their time, instead of having to > constantly handle the flow of emails. Hi Benoît Thanks for your suggestions. A couple of comments from me: The issue tracker and the mailing lists are not mutually exclusive things. The issue tracker is a very good tool for, well, tracking issues -- bug reports, suggestions, even feature planning, requirements, unrelated tasks, etc. In some projects, it's also used for controlling the integration system (like WebKit). This I support and I think KDE should use more often. One feature missing in Bugzilla is proper rich formatting, like a wiki. The mailing list is where discussions happen and decisions are made. Code reviews currently happen on mailing lists and that's how I prefer it. Some projects, apparently like Mozilla, do it all on the issue tracker. WebKit does it. For Qt, what I'm trying to find is a system that integrates the review system to the mailing list. Something like we have for kde-bugs-dist: two-way, with enough information so people can filter incoming emails. If I can't find such a system, I'll settle for a system with easy command-line access and mailing list logging (one-way only). > * > 2. Can one tell volunteers contributors what they should be working > on? (KDE: no, Mozilla: yes) > > A common bit of KDE wisdom is that you can't tell volunteers what they > should be working on, since they are volunteers. Mozilla is much more > proactive in deciding what needs more attention, and what is just a > waste of time, and that applies to everyone regardless of > volunteer/employee status. Here I'm not saying that Mozilla got it > right and KDE got it wrong, because the fact is that KDE has an order > of magnitude more volunteer developers than Mozilla has, and that > could partly be related to that. However, KDE could do more to > communicate what actually needs to be done, for the benefit of those > volunteers who are interested in investing their time in what's most > useful. It's nice making everyone feel that they can "be KDE" but the > truth is that you are much more KDE when you work on what KDE really > needs. There's a huge difference in that Mozilla is mostly developed by the Mozilla Corporation, which is owned by the Mozilla Foundation. The Corp. people are paid to work on what the Corp. wants them to, which in turn takes their cue from the community suggestions (among other sources). That said, however, one thing that KDE can do better is to collect the suggestions, trends, requests and organise them for the volunteer contributors. I've seen many a contributor who wants to do something but doesn't know where. A little guidance could do wonders. > * > 3. QA, Unit-testing, continuous testing (KDE: very little, Mozilla: a lot) > > Let's face it, KDE has always been deficient in the QA department. In > order to ensure good QA, it needs: > * a lot more unit tests than it currently has (Kate in SC 4.5.0 was > horribly buggy) > * unit tests must be automatically run, frequently. In mozilla, > everything is built and the unit tests run on 10 configs everytime you > push to the repository, see http://tbpl.mozilla.org/ > * this means investing in some build hardware. There are cloud > services for that, too. > * regressions must never be tolerated. if a commit causes a > regression, back it out. If a big set of changes need to go through a > bad intermediate state, develop it e.g. in a separate git repository. That's a laudable goal and one we do for Qt. We can't test every push, though, simply because that doesn't scale. I have some material I'm preparing that I'll share on this subject by the end of the year. KDE is much futher back in that. It's not reasonable to say jump from where KDE is to where Mozilla or Qt are. It needs to be a gradual process, with a plan. For us in Qt, we've had unit tests running for over 7 years. But it wasn't until last year that we had a system that tested the incoming changes and blocked them if a regression happened. Even then, the coverage is very low (around 60% on average) and there are many unstable tests. And a proper build farm can cost in the range of mill
Re: Thoughts from a former KDE, currently Mozilla developer
Em Quarta-feira, 8 de Dezembro de 2010, às 09:39:42, Benoit Jacob escreveu: > > Also, with > > enough debug symbols you won't be able to search for libGL anyway as > > the backtrace has references to source files only. > > How do you think that debuggers know the file? I think they just use > the modules map for the process, e.g. on linux it's /proc/$PID/maps, > that allows to find the object filename from the address. > > Try this: > cat /proc/`pidof kwin`/maps Not really. The debugger queries the information from libdl first, which has a list of everything it loaded and where. It also contains a list of (public, dynamic) symbols, which is how it can show that the address is in a certain function. That's the dladdr() function. And that's only after the debug symbol query failed. The map list is not always complete, due to anonymous maps. > >> Regarding backtraces, as I said in my next email, it's not realistic > >> to expect the user to do anything more than click OK, when an app > >> crashed on him. > > > > I'm sorry, but I as a developer don't want to work with such users > > anyway. If a user isn't able or willing to provide information what he > > did for the crash to occur or tell me wether its reproduceable then I > > don't want to spend time on helping him. On the other hand, if the user > > is able/willing to do the above, he surely is able/willing to open his > > package manager and select a package or two (where the name is suggested > > by dr. konqi already) and then hit the reload button in dr. konqi. > > Even if the x % of users who do this give you enough crash reports to > handle, you still lose the other aspect of crash reports: statistics. > It's just very useful to know the top 100 crash locations in an app, > and how crashiness changes over time. That is true. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Senior Product Manager - Nokia, Qt Development Frameworks PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part. >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
Re: Thoughts from a former KDE, currently Mozilla developer
On Wednesday, December 8, 2010, Thiago Macieira wrote: > One feature missing in Bugzilla is proper rich formatting, like a wiki. or threaded conversations. like a mailing list. :) -- Aaron J. Seigo humru othro a kohnu se GPG Fingerprint: 8B8B 2209 0C6F 7C47 B1EA EE75 D6B7 2EB1 A7F1 DB43 KDE core developer sponsored by Qt Development Frameworks signature.asc Description: This is a digitally signed message part. >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
Re: Thoughts from a former KDE, currently Mozilla developer
[ Comments below, in line ] On Wednesday 08 December 2010 at 12:54 pm, Aaron J. Seigo penned about "Re: Thoughts from a former KDE, currently Mozilla developer" > On Wednesday, December 8, 2010, Thiago Macieira wrote: > > One feature missing in Bugzilla is proper rich formatting, like a wiki. > > or threaded conversations. like a mailing list. :) > I kinda liked Usenet. ;) Cheers, -- Pablo Sanchez - Blueoak Database Engineering, Inc Ph:819.459.1926 Fax: 760.860.5225 (US) >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
Re: Thoughts from a former KDE, currently Mozilla developer
On 8 Dec 2010, at 8:55 AM, Thiago Macieira wrote: > Em Terça-feira, 7 de Dezembro de 2010, às 11:22:47, Benoit Jacob escreveu: >> * >> 1. Issue tracker (Mozilla) vs. Mailing lists (KDE). >> >> KDE uses a Mailing list for most its development discussion, and only >> uses trackers (bugzilla, reviewboard) for specific tasks (user bug >> reports, patch review). >> >> Mozilla uses a (modern, well-configured) bugzilla for almost >> everything including 90% of development discussion. >> >> My point here is not to advertise bugzilla specifically (though I love >> it), but issue trackers in general. >> >> An issue tracker, contrary to a mailing list, allows to efficiently >> keep track of what TODO items remain to be done, how they must be >> prioritized, etc. It also is much more convenient for core >> contributors because it allows them to handle issues later, so they >> can reclaim control of how they spend their time, instead of having to >> constantly handle the flow of emails. > > Hi Benoît > > Thanks for your suggestions. A couple of comments from me: > > The issue tracker and the mailing lists are not mutually exclusive things. > The > issue tracker is a very good tool for, well, tracking issues -- bug reports, > suggestions, even feature planning, requirements, unrelated tasks, etc. In > some projects, it's also used for controlling the integration system (like > WebKit). This I support and I think KDE should use more often. > > One feature missing in Bugzilla is proper rich formatting, like a wiki. > > The mailing list is where discussions happen and decisions are made. > > Code reviews currently happen on mailing lists and that's how I prefer it. > Some projects, apparently like Mozilla, do it all on the issue tracker. > WebKit > does it. > > For Qt, what I'm trying to find is a system that integrates the review system > to the mailing list. Something like we have for kde-bugs-dist: two-way, with > enough information so people can filter incoming emails. > > If I can't find such a system, I'll settle for a system with easy > command-line > access and mailing list logging (one-way only). > You might want to look at http://code.google.com/appengine/articles/rietveld.html as it does a fairly good job, and has a decent command line tool. -- >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
