Wow this is a delightful machine: I am jealous! Do you have any sense of whether the higher IOPs/throughput of NVMe SSDs vs "mere" SATA III matters for time to run Lucene's/Solr's tests?
Also, how did you parallelize the running of the tests? Just the normal top-level "ant test"? Or one "ant test" under lucene and one under solr, running at once? Mike McCandless http://blog.mikemccandless.com On Wed, May 27, 2015 at 9:29 AM, Steve Rowe <[email protected]> wrote: > SSD: http://www.newegg.com/Product/Product.aspx?Item=N82E16820167299 > CPU: http://www.newegg.com/Product/Product.aspx?Item=N82E16819117404 > M/B: http://www.newegg.com/Product/Product.aspx?Item=N82E16813132518 > RAM: http://www.newegg.com/Product/Product.aspx?Item=N82E16820231820 > > The mem wasn’t listed as supported by the mobo manufacturer, and it isn’t > detected at its full speed (2800MHz), so currently running at 2400 > (“overclocked” from detected 2100 I think). CPU isn’t overclocked from stock > 3GHz, but I got a liquid cooler, thinking I’d experiment (haven’t much yet). > Even without overclocking the fans spin faster when all the cores are busy, > and it’s quite the little space heater. > > I installed Debian 8, but had to fix the installer in a couple places because > it didn’t know about the new NVMe device naming scheme: > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=785147 > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=785149 > > I also upgraded to the 4.0 Linux kernel, since Debian 8 ships with the 3.16 > kernel, and 3.19 contains a bunch of NVMe improvements. > > And I turned “swappiness" down to zero (thanks to Mike: > <http://blog.mikemccandless.com/2011/04/just-say-no-to-swapping.html>) after > seeing a bunch of test stalls while running the Lucene monster tests with 4 > JVMs (takes about 2 hours, but I can get it down to 90 minutes or so by > splitting the two tests in Test2BSortedDocValues out into their own suites - > I’ll make an issue). > > Steve > >> On May 27, 2015, at 5:08 AM, Anshum Gupta <[email protected]> wrote: >> >> 8-real-core Haswell-E with 64G DDR4 memory and a NVMe 750-series SSD. >> >> Can run *all* of the Lucene and Solr tests in 10 minutes by running multiple >> ant jobs in parallel! >> >> On Wed, May 27, 2015 at 1:17 AM, Ramkumar R. Aiyengar >> <[email protected]> wrote: >> Curious.. sarowe, what's the spec? >> >> On 26 May 2015 20:41, "Anshum Gupta" <[email protected]> wrote: >> The last buch of fixes seems to have fixed this. The tests passed on a >> Jenkins that had failing tests earlier. >> Thanks Steve Rowe for lending the super-powerful machine that runs the >> entire suite in 8 min! >> >> I've seen about 20 runs on that box and also runs on Policeman Jenkins with >> no issues related to this test since the last commit so I've also >> back-ported this to 5x as well. >> >> On Tue, May 26, 2015 at 9:19 AM, Chris Hostetter <[email protected]> >> wrote: >> >> : Right, as I said, we weren't hitting this issue due to our Kerberos conf. >> : file. i.e. the only thing that was different on our machines as compared to >> : everyone else and moving that conf file got the tests to fail for me. It >> : now fails fairly consistently without the patch (from SOLR-7468) and has >> : been looking good with the patch. >> >> that smells like the kind of thing that sould have some "assume sanity >> checks" built into it. >> >> Given: >> * the test setups a special env before running the test >> * the test assumes the specific env will exist >> * the user's machine may already have env properties set before running ant >> that affect the expected special env >> >> therefore: before the test does the setup of the special env, it should >> sanity check that the users basic env doesn't have any properties that >> violate the "basic" situation. >> >> so, hypothetical example based on what little i understand the >> authentication stuff: if the purpose of a test is to prove that some >> requests work with (or w/o) kerberos authentication, then before doing any >> setup of a "mock" kerberos env (or before setting up a "mock" situation >> where no authentication is required), the test should verify that there >> isn't already an existing kerberos env. (or if possible: "unset" whatever >> env/properties define that env) >> >> >> trivial example of a similar situation is the script engine tests -- >> TestBadConfigs.testBogusScriptEngine: the purpose of the test is to >> ensure that a solrconfig.xml file that refers to a script engine (by >> name) which is not installed on the machine will produce an expeted error >> at Solr init. before doing the Solr init, we have an whitebox assume that >> asks the JVM directly if a script engine with that name already exists) >> >> >> >> -Hoss >> http://www.lucidworks.com/ >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> >> >> >> -- >> Anshum Gupta >> >> >> >> -- >> Anshum Gupta > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
