It's worth noting that this build uses CMake to generate the Makefiles from the project specification files in the prepare_to_build.bat step, but that should have little bearing on the resulting behavior.
The problem is that when run outside of jenkins, the build takes just a minute, and Task Manager clearly shows dozens of gcc.exe processes running in parallel. When the same steps as listed below are run from within Jenkins, even with the -j flag passed to make, we don't get more than one gcc.exe running at a time. If anyone has any ideas, please let us know! Thanks :) On Thu, Jun 14, 2012 at 7:11 AM, Chris Haverkate <chris.haverk...@gmail.com>wrote: > Here is a dump of all the details I can think of. If something is missing, > let me know. > > *Make:* > make -v > GNU Make 3.81 > Copyright (C) 2006 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. > There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A > PARTICULAR PURPOSE. > > This program built for Windows32 > > *Environment Variables (from Jenkins):* > > Name ↓ <http://10.5.69.54:8080/systemInfo#>Value > <http://10.5.69.54:8080/systemInfo#> > ALLUSERSPROFILEC:\ProgramDataAPPDATAC:\Users\[user]\AppData\RoamingBASEE:\Jenkins > CommonProgramFilesC:\Program Files (x86)\Common Files > CommonProgramFiles(x86)C:\Program Files (x86)\Common Files > CommonProgramW6432C:\Program Files\Common > FilesCOMPUTERNAME...ComSpecC:\Windows\system32\cmd.exe > FP_NO_HOST_CHECKNOHOMEDRIVEC:HOMEPATH \Users\[user]JENKINS_HOMEE:\Jenkins > LOCALAPPDATAC:\Users\[user]\AppData\LocalLOGONSERVER \\INDIA-ENGMAKEFLAGS > -IC:/QNX650/target/qnx6/usr/includeMOZ_PLUGIN_PATHC:\Program Files > (x86)\Foxit Software\Foxit Reader\plugins\ NUMBER_OF_PROCESSORS12OS > Windows_NTPathC:\Program Files (x86)\NVIDIA > Corporation\PhysX\Common;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program > Files (x86)\Microsoft SQL Server\100\Tools\Binn\;C:\Program Files\Microsoft > SQL Server\100\Tools\Binn\;C:\Program Files\Microsoft SQL > Server\100\DTS\Binn\;C:\QNX650\host\win32\x86\usr\bin;C:\Program Files > (x86)\QNX Software Systems\bin;C:\Program Files (x86)\GE > Energy\ToolboxST\V04.04.11C\PATHEXT > .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC > PROCESSOR_ARCHITECTUREx86 > PROCESSOR_ARCHITEW6432AMD64PROCESSOR_IDENTIFIERIntel64 > Family 6 Model 44 Stepping 2, GenuineIntel PROCESSOR_LEVEL6 > PROCESSOR_REVISION2c02ProgramDataC:\ProgramData ProgramFilesC:\Program > Files (x86)ProgramFiles(x86)C:\Program Files (x86)ProgramW6432C:\Program > Files PSModulePathC:\Windows\system32\WindowsPowerShell\v1.0\Modules\ > PUBLICC:\Users\PublicQNX_CONFIGURATION C:\Program Files (x86)\QNX > Software SystemsQNX_HOSTC:/QNX650/host/win32/x86QNX_JAVAHOMEC:\QNX650\_jvm > QNX_TARGETC:/QNX650/target/qnx6SSO204056987SystemDriveC: SystemRoot > C:\WindowsTEMPC:\Users\[user]\AppData\Local\TempTMP > C:\Users\[user]\AppData\Local\Temp USERDNSDOMAIN...USERDOMAIN...USERNAME > [user] USERPROFILEC:\Users\[user]VS100COMNTOOLSC:\Program Files > (x86)\Microsoft Visual Studio 10.0\Common7\Tools\windir C:\Windows > windows_tracing_flags3windows_tracing_logfile > C:\BVTBin\Tests\installpackage\csilogfile.log > *System Properties (from Jenkins): > *Name ↓ <http://10.5.69.54:8080/systemInfo#> Value > <http://10.5.69.54:8080/systemInfo#> > awt.toolkitsun.awt.windows.WToolkit executable-warE:\Jenkins\jenkins.war > file.encodingCp1252file.encoding.pkg sun.iofile.separator\ > hudson.diyChunkingtruehudson.lifecycle > hudson.lifecycle.WindowsServiceLifecyclejava.awt.graphicsenv > sun.awt.Win32GraphicsEnvironmentjava.awt.headless truejava.awt.printerjob > sun.awt.windows.WPrinterJobjava.class.pathE:\Jenkins\jenkins.warjava.class.version > 50.0java.endorsed.dirsE:\Jenkins\jre\lib\endorsedjava.ext.dirs > E:\Jenkins\jre\lib\ext;C:\Windows\Sun\Java\lib\ext java.homeE:\Jenkins\jre > java.io.tmpdirC:\Users\haverkch\AppData\Local\Temp\java.library.path > E:\Jenkins\jre\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\Program > Files (x86)\NVIDIA > Corporation\PhysX\Common;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program > Files (x86)\Microsoft SQL Server\100\Tools\Binn\;C:\Program Files\Microsoft > SQL Server\100\Tools\Binn\;C:\Program Files\Microsoft SQL > Server\100\DTS\Binn\;C:\QNX650\host\win32\x86\usr\bin;C:\Program Files > (x86)\QNX Software Systems\bin;C:\Program Files (x86)\GE > Energy\ToolboxST\V04.04.11C\;.java.runtime.nameJava(TM) SE Runtime > Environmentjava.runtime.version 1.6.0_26-b03java.specification.nameJava > Platform API Specificationjava.specification.vendor Sun Microsystems Inc. > java.specification.version1.6java.vendorSun Microsystems Inc.java.vendor.url > http://java.sun.com/java.vendor.url.bug > http://java.sun.com/cgi-bin/bugreport.cgi > java.version1.6.0_26java.vm.infomixed > modejava.vm.name Java HotSpot(TM) Client VMjava.vm.specification.nameJava > Virtual Machine Specification java.vm.specification.vendorSun > Microsystems Inc.java.vm.specification.version1.0java.vm.vendor Sun > Microsystems Inc.java.vm.version20.1-b02line.separator > os.archx86os.nameWindows 7os.version 6.1path.separator;sun.arch.data.model > 32sun.boot.class.path > E:\Jenkins\jre\lib\resources.jar;E:\Jenkins\jre\lib\rt.jar;E:\Jenkins\jre\lib\sunrsasign.jar;E:\Jenkins\jre\lib\jsse.jar;E:\Jenkins\jre\lib\jce.jar;E:\Jenkins\jre\lib\charsets.jar;E:\Jenkins\jre\lib\modules\jdk.boot.jar;E:\Jenkins\jre\classes > sun.boot.library.pathE:\Jenkins\jre\binsun.cpu.endianlittlesun.cpu.isalistpentium_pro+mmx > pentium_pro pentium+mmx pentium i486 i386 i86 > sun.desktopwindowssun.io.unicode.encodingUnicodeLittle > sun.java.commandE:\Jenkins\jenkins.war > --httpPort=8080sun.java.launcherSUN_STANDARDsun.jnu.encoding Cp1252 > sun.management.compilerHotSpot Client Compilersun.os.patch.levelService > Pack 1 svnkit.http.methodsDigest,Basic,NTLM,Negotiate > svnkit.ssh2.persistentfalseuser.country USuser.dirC:\Windows\system32 > user.homeC:\Users\[user]user.language enuser.namehaverkchuser.timezone > America/Los_Angelesuser.variant > > *Plugins:* > Name ↓ <http://10.5.69.54:8080/systemInfo#> Version > <http://10.5.69.54:8080/systemInfo#> > Enabled <http://10.5.69.54:8080/systemInfo#> Pinned > <http://10.5.69.54:8080/systemInfo#> > ant1.1true falsejavadoc1.0truefalsecvs1.6true falseezwall0.2truefalse > global-build-stats1.3 truefalsegreenballs1.12truefalsemaven-plugin1.467 > truefalseradiatorviewplugin1.13truefalsesectioned-view 1.16truefalse > ssh-slaves0.21truefalse subversion1.34truefalsetfs1.20truefalsetimestamper > 1.2.2truefalsetranslation > *Build Information:* > > QNX Build: > > - restricted to run on a node configured to be the same as the one > Jenkins is installed on (using the windows service) > - source code management = TFS > - Trigger = polling > - adding timestamps to console output (through a plugin) > - execute Windows batch commands... > - rm -rf Build_GT > - "C:\Program Files (x86)\Microsoft Visual Studio > 10.0\Common7\IDE\tf" get $/Framework/ROOT/Framework/source/Shared /force > - xcopy "%WORKSPACE%\Framework\ROOT\*" "%WORKSPACE%\Root\" /S /Y /R > /I > - cd Root/GT/Firmware > - call prepare_to_build.bat localbuildonly > - @echo on > - set SHELL=cmd.exe > - cd %WORKSPACE%/Build_GT/QNX_ppcbe_Release > - C:/QNX650/host/win32/x86/usr/bin/make.exe -j24 > > > > On Wed, Jun 13, 2012 at 12:51 PM, Slide <slide.o....@gmail.com> wrote: > >> What version of make is being used? You are going through cygwin? Can you >> explain a little more about your environment? >> >> slide >> >> >> On Wed, Jun 13, 2012 at 12:49 PM, chris <chris.haverk...@gmail.com>wrote: >> >>> ---------------------------- >>> So I realized that I did not respond to the list properly. Anyways, here >>> is an update. >>> >>> Still no luck. I've been playing with Jenkins more and my system seems >>> to briefly use more than one processor, but the utilization quickly goes >>> back to 0% - 10%. >>> >>> Worse yet, The builds for QNX have only been slowing down. They started >>> at about 6 minutes and have reached well over an hour per build now. >>> >>> ---------------------------- >>> >>> >>> Yeah, freestyle. >>> On Jun 1, 2012 4:48 PM, "Alex Earl" <slide.o....@gmail.com> wrote: >>> I can't think of anything that would restrict this in Jenkins. What type >>> of job are you using? Freestyle? >>> >>> Sent from my Windows Phone >>> ------------------------------ >>> From: Chris Haverkate >>> Sent: 6/1/2012 4:47 PM >>> To: Alex Earl >>> Subject: RE: Multi-process jobs in Windows >>> >>> Yeah, I explicitly called it using the full path. >>> On Jun 1, 2012 4:44 PM, "Alex Earl" <slide.o....@gmail.com> wrote: >>> You are sure the same make is being used in both cases? >>> >>> Sent from my Windows Phone >>> ------------------------------ >>> From: Chris Haverkate >>> Sent: 6/1/2012 4:43 PM >>> To: Alex Earl >>> Subject: RE: Multi-process jobs in Windows >>> >>> I'm using a version of make that is distributed with QNX. I've tried >>> "make -j12" with no luck. >>> >>> Outside of Jenkins the -j option works fine, with our without a number. >>> >>> On Friday, June 1, 2012 4:11:29 PM UTC-7, slide wrote: >>>> >>>> Are you running under cygwin or using a native win32 make? We run -j5 >>>> with no problems for several jobs which run under cygwin. >>>> >>>> Sent from my Windows Phone >>>> From: chris >>>> Sent: 6/1/2012 3:42 PM >>>> To: Jenkins Users >>>> Subject: Multi-process jobs in Windows >>>> Hey, >>>> >>>> I am trying to setup a new job that uses make as the build tool. I >>>> want to use the "-j" option to spawn several parallel jobs. The >>>> problem is that the "-j" doesn't seem to have any effect on the speed. >>>> Based on the job log file, it does not appear that multiple process >>>> are running in parallel. If I manually run "make -j" in the workspace, >>>> it's obvious that multiple jobs are running in parallel because the >>>> output is intermixed and the CPU utilization for all cores is pegged. >>>> >>>> I've looked through environment variables, tried a using both shells >>>> to run make, tried running the job in both slave and in Jenkins and >>>> looked online, but have not found any information on the issue. >>>> >>>> Does anyone have any ideas about running parallel jobs using make in >>>> Jenkins on Windows? >>>> >>>> Thanks. >>>> >>> >> >> >> -- >> Website: http://earl-of-code.com >> > > -- Aaron Ten Clay http://www.aarontc.com/
