btw, this is a compilation error in the SBT build that only shows up on the ubuntu workers.
On Mon, Nov 5, 2018 at 5:07 PM shane knapp <skn...@berkeley.edu> wrote: > the maven build is quite happy: > > https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7-ubuntu-testing/ > > i've wiped all the cached deps from .ivy2 on one ubuntu worker, pinned the > build to that (research-jenkins-worker-08) and re-launched the sbt build. > let's see if it fails in a new, spectacular way (or just the same as > before). > > but again: why is this failing on the ubuntu nodes, vs centos? if it's a > poisoned cache, i will be sad, but that's easy to fix. if it's a bad dep > in the pom, then... ¯\_(ツ)_/¯ > > On Mon, Nov 5, 2018 at 5:00 PM Wenchen Fan <cloud0...@gmail.com> wrote: > >> Have you tried Maven instead of SBT? This looks like a Java dependency >> problem, e.g. a wrong version of Avro is picked. >> >> On Tue, Nov 6, 2018 at 8:30 AM shane knapp <skn...@berkeley.edu> wrote: >> >>> i'm really close (for real: really close!) on the ubuntu port... but >>> one build has been a thorn in my side and i was wondering if i could get >>> some extra eyes on this as i grind through the remaining few pieces of my >>> own personal system dependency hell. :) >>> >>> the job in question is: >>> >>> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.7-ubuntu-testing >>> >>> it's identical to the regular spark-master-test-sbt-hadoop-2.7 job, >>> except i'm building against a newer version of java (1.8.0_171 vs >>> 1.8.0_60). >>> >>> the centos job always passes on every worker. >>> >>> the ubuntu job fails on every ubuntu worker during the scala unidoc >>> generation w/the following error: >>> >>> """ >>> [error] >>> /home/jenkins/workspace/spark-master-test-sbt-hadoop-2.7-ubuntu-testing/core/src/main/scala/org/apache/spark/serializer/GenericAvroSerializer.scala:123: >>> value createDatumWriter is not a member of >>> org.apache.avro.generic.GenericData >>> [error] writerCache.getOrElseUpdate(schema, >>> GenericData.get.createDatumWriter(schema)) >>> [error] ^ >>> [info] No documentation generated with unsuccessful compiler run >>> [error] one error found >>> >>> """ >>> an example job w/this failure is here: >>> >>> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.7-ubuntu-testing/30/consoleFull >>> >>> thoughts? am i missing something obvious? i've checked and there are >>> no avro system packages installed on any of the workers (centos or ubuntu). >>> >>> thanks in advance, >>> >>> shane >>> -- >>> Shane Knapp >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>> https://rise.cs.berkeley.edu >>> >> > > -- > Shane Knapp > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu > -- Shane Knapp UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
