A tangent, but a relevant one (to the issue of speed). Have you tried running the tests with Lucene/Solr code being in the RAM disk?
I found that compiling source on RAMDisk is a lot faster than even with SSD drive. Must be just frequency of access. It might be the same with tests. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 3 December 2014 at 00:47, Michael Wechner <[email protected]> wrote: > Hi Shawn > > Thanks very much for your feedback. Please see my comments inline below > > Am 03.12.14 um 00:10 schrieb Shawn Heisey: >> On 12/2/2014 3:31 PM, Michael Wechner wrote: >>> I am trying to setup a testing environment, which is running tests >>> incrementally based on changed files, instead of running whole builds. >>> >>> I would like to try this for Lucene/Solr, hence I am trying to >>> understand better the relationship between individual code changes and >>> associated tests. For example I have noticed to the following change: >>> >>> git diff --name-only 641ed06..cb4fc89 >>> lucene/core/src/java/org/apache/lucene/codecs/compressing/CompressingStoredFieldsFormat.java >>> >>> and if I understand correctly, then the following tests would have to be >>> executed: >>> >>> lucene/core/src/test/org/apache/lucene/codecs/compressing/TestCompressingStoredFieldsFormat.java >>> lucene/test-framework/src/java/org/apache/lucene/codecs/compressing/CompressingCodec.java >>> >>> is that correct? >> The test dependency tree for this particular class is *EXTENSIVE* and >> not very easy to track down. >> >> CompressingStoredFieldsFormat is the parent class for >> Lucene50StoredFieldsFormat. It is directly referenced by three classes >> in addition to the direct descendant. Here's a screenshot showing all >> direct references in branch_5x: >> >> https://www.dropbox.com/s/3pb4kljvdbd6xug/CompressingStoredFieldsFormat-eclipse.png?dl=0 >> >> Lucene50StoredFieldsFormat is used by a few classes, one of which is >> Lucene50Codec: >> >> https://www.dropbox.com/s/0ibnw87m6rveinv/Lucene50StoredFieldsFormat.png?dl=0 >> >> Lucene50Codec is extremely central to almost everything the unreleased >> Lucene 5.x does if it is using default settings, so a very large number >> of tests would be affected by a change in the compressed stored field >> format. Just for giggles, I grabbed a screenshot of the classes that >> directly reference Lucene50Codec: >> >> https://www.dropbox.com/s/eqq30uvckc8b19d/Lucene50Codec.png?dl=0 > > thank you very much for your insights. I did not realize that this > particular case has > such extensive dependencies >> >> The build system includes the ability to use Clover if it is present ... >> I don't know much about it, except that it's related to determining test >> coverage for parts of the software. Perhaps a clover report could give >> you the info you need? >> >> https://www.atlassian.com/software/clover/overview > > I have heard about Clover, but never used it. I will have a closer look > at it now >> >> Aside from that ... running the entire test suite multiple times >> whenever you change anything is the best option. The test system >> randomizes a large number of options on each test run, so running the >> tests multiple times will test many different combinations. A very >> large number of bugs have been discovered and fixed because of that >> randomization. > > > I think you are right that in certain cases, like for example in the > case of CompressingStoredFieldsFormat > it makes a lot of sense to run the entire test suit (even multiple times > considering randomization), but I would argue that there are probably > quite a few other cases where incremental testing would be sufficient, > at least as a first shot. > > I am saying this, because I think for larger projects running entire > test suits do not scale well and that it is rather unefficient and > inconvenient for developers to run the entire test suite every time, > before they commit/push a tiny change. > > Since I am not familiar enough with the code of Lucene/Solr I cannot say > whether all code is covered by tests, > but I would assume it is not. Also it would be interesting to know > whether developers do manual testing before they commit/push or whether > they just rely on running the entire test suit. I guess they are doing > manual tests as well, but probably only test things which they think are > related to the changes which they made. > > Hence I think it would help developers to have a system which supports > incremental testing. I really do not mean to say that it should replace > the entire suite testing approach, but it would be additionally. > > I will think about it some more :-) > > Thanks > > Michael > >> >> Thanks, >> Shawn >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
