Hi. As I had mentioned, upgrading to the latest version of the library is not as simple as I thought. Google Cloud team introduced many breaking changes. Many of the APIs (classes & methods) have been scrapped/replaced/modified/refactored/renamed.
In short, a simple change of version may demand changes on the processor's code, especially on the AbstractProcessors (AbstractGCS, AbstractGCP) which may pose backward compatibility issues, I'm afraid. Thanks, Sivaprasanna On Thu, May 3, 2018 at 9:26 PM, Joe Witt <joe.w...@gmail.com> wrote: > Sivaprasanna > > I might not completely follow but is there a 3rd option to upgrade to > a more recent library and solve the use of the proper jars > problem(smaller nar)? > > Thanks > > On Thu, May 3, 2018 at 11:51 AM, Sivaprasanna <sivaprasanna...@gmail.com> > wrote: > > Hi > > > > I've started the initial works on implementing Google Cloud Pub/Sub > > processors. The associated Jira ID is NIFI-5133 > > <https://issues.apache.org/jira/browse/NIFI-5133>. This will go to the > > existing GCP bundle which currently has only the storage processors. Upon > > some inspection, I noticed the following: > > > > - As of now, the bundle uses google-cloud > > <http://mvnrepository.com/artifact/com.google.cloud/google-cloud> as > its > > dependency which is like uber/fat jar that contains most of the Google > > Cloud's client library SDKs including storage, bigquery, pubsub, etc. > The > > main point is it is using a very older version (0.8.0) > > - I thought of using google-cloud-bom > > <http://mvnrepository.com/artifact/com.google.cloud/google-cloud-bom> > in > > the bundle's POM > > <https://github.com/apache/nifi/blob/master/nifi-nar- > bundles/nifi-gcp-bundle/pom.xml> > > and then use the required artifacts in the processor's POM > > <https://github.com/apache/nifi/blob/master/nifi-nar- > bundles/nifi-gcp-bundle/nifi-gcp-processors/pom.xml>. > > The benefit is, it will help us reduce the overall size of the NAR. > > > > When I tried to do #2, I realized this is not a simple version change > but a > > change that brings backward compatibility issues. Ex: Some APIs used in > the > > older version i.e. 0.8.0 have now been entirely scrapped and moved to > > different library. We can do either two things: > > > > 1. User the Pub/Sub APIs from the older version but the problem is > it's > > very old and the problem of upgrading would soon catchup with us. > > 2. Or we can continue to use the older version of google-cloud-storage > > only for the storage processors and introduce the #2 mentioned above > but I > > don't think then the new processors can't properly extend the existing > > AbstractGCPProcessor > > <https://github.com/apache/nifi/blob/master/nifi-nar- > bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/ > org/apache/nifi/processors/gcp/AbstractGCPProcessor.java>. > > > > > > A quick glance on the processor code and the POM would help you > understand > > my concern. > > > > I'm stuck up here so any help & guidance in this regard is very much > > appreciated. :) > > > > Thanks, > > > > Sivaprasanna >
