Hello Matthew, The main reason we don't use nexus is that we were unaware of it and as Raúl said the Java contributors adapted the R workflow that publishes to nightlies.a.o Looking at your slack conversation with infra it seems that the nexus repository is a bit tight on space currently so it like would not be the best to move our ~450MB/day nightlies there.
But as I am not a Java developer I would be open to changes if there is a clear benefit for the devs using the nightlies. Best Jacob On Wed, Jan 25, 2023 at 11:56 AM Matthew Benedict de Detrich <matthew.dedetr...@aiven.io.invalid> wrote: > From my understanding as a general design perspective, if you are uploading > Java/JVM jars with the intention of them being resolved as a library you > should be putting them into the Apache Nexus repo (in this case the > snapshots repo). Of course nothing is stopping you from uploading it to a > remote folder via rsync (as you are doing) but the whole idea behind Nexus > is it gives tools for managing JVM libraries/dependencies (mainly for the > host of the repository, less so for users uploading to it). In short there > isn't a very strong technical reason (if there was presumably you wouldn't > be using it as a solution right now), but as far as I can tell it is an > expectation to use it for JVM library jars judging by the response at > https://the-asf.slack.com/archives/CBX4TSBQ8/p1674497939153149 (I would > say > that if you want more details then just ask for more detail in that slack > channel). One example is that you had to implement snapshot expiry > yourself, whereas Nexus automatically handles this for everyone (there is a > job in Nexus which does snapshot expiry so as a user you don't even have to > worry about it). > > As a Java/JVM developer, personally I would also say that putting library > jars via rsync into a remote folder is highly unorthodox given that an > Apache Nexus repository exists (if it didn't that would be another > story). The impression I have is that the nightlies folder (at least from > the other projects that use it) is mainly for the result of builds (e.g. > projects that are "run" which produce executables/fatjars i.e. > applications, databases), docs, microsites, etc etc. > > On Wed, Jan 25, 2023 at 11:03 AM Raúl Cumplido <raulcumpl...@gmail.com> > wrote: > > > Hi Matthew, > > > > Uploading java jars to nightlies.apache.org was implemented on the > > following PR: https://github.com/apache/arrow/pull/13328 > > > > From what I remember the solution of uploading to nightlies.apache.org > was > > to follow the same workflow as our R nightly builds. More info on when > the > > R nightly builds were first implemented on this JIRA: > > https://issues.apache.org/jira/browse/ARROW-16400 > > > > I don't remember investigating the Apache Nexus snapshot repository when > we > > added the Java jars. We followed the the same workflow we already had. > > > > Do you know if there is a technical reason for preferring the Nexus > > Snapshot Repository for JVM artifacts or is it for consistency with other > > projects? > > > > Thanks, > > Raúl > > > > > > > > El mié, 25 ene 2023 a las 10:38, Matthew Benedict de Detrich > > (<matthew.dedetr...@aiven.io.invalid>) escribió: > > > > > Hello Arrow community! > > > > > > I am coming from the Apache Pekko community where we are trying to > > > bootstrap the newly approved incubator project. Currently we are > working > > on > > > snapshots/nightlies, specifically where to upload them and also the > > process > > > of generating the snapshots (i.e. nightly, after merge into main etc > > etc). > > > > > > As of now we are uploading into Apache nightlies (i.e. > > > https://nightlies.apache.org/ with PR > > > https://github.com/apache/incubator-pekko/pull/60) and the method of > > > uploading was actually derived from Apache Arrow. However after further > > > cursory investigation I found out that Apache Arrow seems to be the > > > only Apache project that seems to upload snapshots into > > > https://nightlies.apache.org/. Every other project uploads snapshots > > into > > > Apache's Nexus snapshot repository (i.e. > > > https://repository.apache.org/content/groups/snapshots/) and after > > > discussion with Apache's #askinfra slack channel, they also recommend > > that > > > any JVM library snapshot artifacts be published to this Nexus Snapshot > > > Repository. > > > > > > So the question is, is there a reason why this project is uploading > > > snapshot jars into the nightlies? The only technical reason I can come > up > > > with is that with nightlies you have control over the expiry (i.e. > > setting > > > it to 30 days), whereas with Apache's Snapshot Repo there is a default > > > global job that cleans snapshots. Pekko is currently considering > > uploading > > > to Apache's Nexus Snapshot Repo instead of nightlies but I want to make > > > sure we haven't missed anything, for context of the discussion in pekko > > see > > > https://lists.apache.org/thread/p5s8ysypyd2l2slb3o9f2v5vrf8dgsx8. > > > > > > Many thanks > > > > > > -- > > > > > > Matthew de Detrich > > > > > > *Aiven Deutschland GmbH* > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > *m:* +491603708037 > > > > > > *w:* aiven.io *e:* matthew.dedetr...@aiven.io > > > > > > > > -- > > Matthew de Detrich > > *Aiven Deutschland GmbH* > > Immanuelkirchstraße 26, 10405 Berlin > > Amtsgericht Charlottenburg, HRB 209739 B > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > *m:* +491603708037 > > *w:* aiven.io *e:* matthew.dedetr...@aiven.io >
