Honestly it is pretty difficult. Given the difficulty, would it still make sense to do that change? (the one that sets the same number of workers/parallelism across different languages in testing)
On Mon, Dec 5, 2016 at 3:33 PM, Saikat Kanjilal <sxk1...@hotmail.com> wrote: > Hello again dev community, > > Ping on this, apologies for rerunning this thread but never heard from > anyone, based on this link: https://wiki.jenkins-ci.org/ > display/JENKINS/Installing+Jenkins I can try to install jenkins locally > but is that really needed? > > > Thanks in advance. > > > ------------------------------ > *From:* Saikat Kanjilal <sxk1...@hotmail.com> > *Sent:* Tuesday, November 29, 2016 8:14 PM > *To:* dev@spark.apache.org > *Subject:* Spark-9487, Need some insight > > > Hello Spark dev community, > > I took this the following jira item (https://github.com/apache/ > spark/pull/15848) and am looking for some general pointers, it seems that > I am running into issues where things work successfully doing local > development on my macbook pro but fail on jenkins for a multitiude of > reasons and errors, here's an example, if you see this build > output report: https://amplab.cs.berkeley.edu/jenkins//job/ > SparkPullRequestBuilder/69297/ you will see the DataFrameStatSuite, now > locally I am running these individual tests with this command: ./build/mvn > test -P... -DwildcardSuites=none > -Dtest=org.apache.spark.sql.DataFrameStatSuite. > It seems that I need to emulate a jenkins like environment locally, > this seems sort of like an untenable hurdle, granted that my changes > involve changing the total number of workers in the sparkcontext and if so > should I be testing my changes in an environment that more closely > resembles jenkins. I really want to work on/complete this PR but I keep > getting hamstrung by a dev environment that is not equivalent to our CI > environment. > > > > I'm guessing/hoping I'm not the first one to run into this so some > insights. pointers to get past this would be very appreciated , would love > to keep contributing and hoping this is a hurdle that's overcomeable with > some tweaks to my dev environment. > > > > Thanks in advance. >
