Thanks for that, Joshua. Late last night I moved my BigTable-related code (depending on beam:beam-sdks-java-io-google-cloud-platform) into a separate pom/project from GCS-related code (depending on google-cloud-storage).
pom A: +- org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.2.0:compile | +- com.google.api:gax-grpc:jar:0.20.0:compile | | +- io.grpc:grpc-protobuf:jar:1.2.0:compile | | +- com.google.api:api-common:jar:1.1.0:compile | | +- com.google.api:gax:jar:1.3.1:compile | | \- org.threeten:threetenbp:jar:1.3.3:compile pom B: | +- org.apache.beam:beam-sdks-java-core:jar:2.2.0:compile | \- org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.2.0:compile | +- com.google.api:gax-grpc:jar:0.20.0:compile | | +- io.grpc:grpc-protobuf:jar:1.2.0:compile | | \- org.threeten:threetenbp:jar:1.3.3:compile ... +- com.google.cloud:google-cloud-storage:jar:1.4.0:compile | +- com.google.cloud:google-cloud-core:jar:1.4.0:compile | | +- org.json:json:jar:20160810:compile | | +- com.google.api:api-common:jar:1.1.0:compile | | +- com.google.api:gax:jar:1.5.0:compile Jacob On Wed, Jan 31, 2018 at 3:24 AM, Joshua Fox <[email protected]> wrote: > The problem does not occur for me now following a dependency update. > > Here are the parts of mvn dependency:tree that involve gax > > | +- org.apache.beam:beam-sdks-java-io-google-cloud-platform: > jar:2.2.0:compile > | | +- com.google.api:gax-grpc:jar:0.20.0:compile > > ... > > +- com.google.code.gson:gson:jar:2.8.2:compile > \- com.google.api:gax:jar:1.15.0:compile > > Those that involve Beam > > +- org.apache.beam:beam-runners-google-cloud-dataflow-java: > jar:2.2.0:compile > | +- org.apache.beam:beam-sdks-java-core:jar:2.2.0:compile > | +- org.apache.beam:beam-sdks-java-extensions-google-cloud- > platform-core:jar:2.2.0:compile > | +- org.apache.beam:beam-sdks-common-runner-api:jar:2.2.0:compile > ... > > | +- org.apache.beam:beam-sdks-java-io-google-cloud-platform: > jar:2.2.0:compile > | | +- com.google.api:gax-grpc:jar:0.20.0:compile > > > Here are the parts of my pom that involve gax and Beam > > > <dependency> > <groupId>org.apache.beam</groupId> > <artifactId>beam-runners-google-cloud-dataflow-java</artifactId> > <version>2.2.0</version> > </dependency> > > > <dependency> > <groupId>com.google.api</groupId> > <artifactId>gax</artifactId> > <version>1.15.0</version> > </dependency> > > > > > On Wed, Jan 31, 2018 at 2:55 AM, Jacob Marble <[email protected]> wrote: > >> Josh, what did you do to work around this? >> >> This suddenly crept up on a production pipeline yesterday, without >> anything changing on our side (we do rebuild at every run). >> >> Jacob >> >> On Fri, Dec 8, 2017 at 6:46 PM, Chamikara Jayalath <[email protected]> >> wrote: >> >>> Created https://issues.apache.org/jira/browse/BEAM-3321 to update >>> the gax-grpc dependency of Beam. >>> >>> - Cham >>> >>> On Friday, December 8, 2017 at 6:12:20 PM UTC-8, Harsh Vardhan wrote: >>>> >>>> +chamikara@ >>>> >>>> >>>> On Friday, December 8, 2017 at 1:31:00 AM UTC-8, Joshua Fox wrote: >>>>> >>>>> I use Cloud Datastore API to check for Kinds in the Datastore, then >>>>> use Dataflow -- now upgrading to Beam -- to copy one Datastore to >>>>> another.. >>>>> >>>>> After adding beam-sdks-java-io-google-cloud-platform to my pom, I >>>>> start getting this when initializing the Cloud Datastore API >>>>> >>>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>>> com/google/api/gax/retrying/ResultRetryAlgorithm >>>>> at com.google.cloud.datastore.DatastoreOptions$DefaultDatastore >>>>> Factory.create(DatastoreOptions.java:51) >>>>> at com.google.cloud.datastore.DatastoreOptions$DefaultDatastore >>>>> Factory.create(DatastoreOptions.java:45) >>>>> at com.google.cloud.ServiceOptions.getService(ServiceOptions.java:426) >>>>> >>>>> It is caused by gax dependencies. >>>>> >>>>> Specifically, before I add beam-sdks-java-io-google-cloud-platform there >>>>> is this gax dependency >>>>> >>>>> +- com.google.cloud:google-cloud-datastore:jar:1.12.0:compile >>>>> | +- com.google.cloud:google-cloud-core:jar:1.12.0:compile >>>>> | | +- com.google.api:gax:jar:1.15.0:compile >>>>> >>>>> >>>>> >>>>> and after I add it there is this >>>>> >>>>> +- org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar: >>>>> 2.2.0:compile >>>>> | +- com.google.api:gax-grpc:jar:0.20.0:compile >>>>> | | +- com.google.api:gax:jar:1.3.1:compile >>>>> >>>>> >>>>> If I add gax 1.15.0 to my pom explicitly, I get >>>>> >>>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>>> com/google/api/gax/retrying/ExceptionRetryAlgorithm >>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>> .. >>>>> at com.google.cloud.BaseService.<clinit>(BaseService.java:48) >>>>> at com.google.cloud.datastore.DatastoreOptions$DefaultDatastore >>>>> Factory.create(DatastoreOptions.java:51) >>>>> at com.google.cloud.datastore.DatastoreOptions$DefaultDatastore >>>>> Factory.create(DatastoreOptions.java:45) >>>>> at com.google.cloud.ServiceOptions.getService(ServiceOptions.java:426) >>>>> >>>>> Clearly Datastore and Beam should work together. Yet there have been >>>>> dependency problems between the two for a while. See this discussion >>>>> <https://stackoverflow.com/questions/40830727/> from 1 year ago. >>>>> >>>>> How can I resolve this? >>>>> >>>> >> > > > -- > > > *JOSHUA FOX* > Principal Software Architect | Freightos > > > > *T (Israel): *+972-545691165 <+972%2054-569-1165> | *T (US)*: + > 1-3123400953 <(312)%20340-0953> > Smooth shipping. > > > >
