> > We can either > store files as is (as Cassandra does, see link below) in something > that is called Generic Repository — this way we manage directory layouts > ourselves and have more control over what is published and how > or > store files in prepared RPM / DEB repositories where we only upload > packages and everything else repository does itself — this way we rely on > Bintray’s implementations of corresponding repositories and have lower > control over directory layout, but the overall process seems to be simpler.
Petr, he I would lean towards your experience and opinion. Personally, I can't see any significant advantages or disadvantages of either approach. This suggests me that we should go for the simplest (2nd) way. -- Denis On Tue, Mar 27, 2018 at 11:59 PM, Petr Ivanov <mr.wei...@gmail.com> wrote: > > > On 28 Mar 2018, at 00:18, Denis Magda <dma...@gridgain.com> wrote: > > Petr, thanks for the update. > > >> 1. I’ve finished preliminary developing of Stage II version of RPM >> packages [1]. Main “new feature” is — split design. Also I’ve added >> package.sh script for automating package building process which will help >> organise corresponding builds in TC as well as simplify process for >> developers who wishes to have custom packages. >> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache >> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer. > > > *Anton*, would you be able to check up Petr's pull-request? > > >> corresponding to each repository type (RPM and DEB) and manage this >> layout (repodata, distributions, versions, etc.) by ourselves, having more >> control over repositories but lacking some simplicity of deploying new >> releases. WDYT? Should we try Cassandra approach? They are storing their >> DEB packages as I described above [5]. > > > Petr, do bintray repos come for free for ASF project? Are there any > limitations in regards total repository or package size? > > > As I see, ASF has premium free account at Bintray with, I guess, virutally > limitless posiiblilities (the only I stumbled across is limit to uploading > files more that 250Mb through UI — upload is possible through API). Yet, > I’ll ask INFRA specifically about this. > > > > WDYT? Should we try Cassandra approach? They are storing their DEB >> packages as I described above [5]. > > > I'm not sure I got the question. Could you show examples of both > approaches? > > > We can either > store files as is (as Cassandra does, see link below) in something > that is called Generic Repository — this way we manage directory layouts > ourselves and have more control over what is published and how > or > store files in prepared RPM / DEB repositories where we only upload > packages and everything else repository does itself — this way we rely on > Bintray’s implementations of corresponding repositories and have lower > control over directory layout, but the overall process seems to be simpler. > > > > Also — a question arose while I was working on this issue: which OSes (and >> which versions of each) are we going to support (if we are going) in terms >> of step-by-step list? Currently RPM packages are tested only with latest >> CentOS (and, respectively — RHEL), but there are a lot more RPM-based >> distributives [6] some of which are more o less popular among OS community >> (ALT, Fedora, openSUSE, etc.). > > > I would test RHEL and Ubuntu for sure, of the last versions. As for the > other distributions, I'd like to hear community opinion. > > > There is minor to none necessity of testing packages on RHEL — CentOS is > binary clone. > Ubuntu is out of scope because it uses DEB packages and currently we are > discussing RPM only. > > > > -- > Denis > > > On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <mr.wei...@gmail.com> wrote: > >> Hi, Igniters! >> >> >> Here are some news on our RPM packages initiative. >> >> 1. I’ve finished preliminary developing of Stage II version of RPM >> packages [1]. Main “new feature” is — split design. Also I’ve added >> package.sh script for automating package building process which will help >> organise corresponding builds in TC as well as simplify process for >> developers who wishes to have custom packages. >> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache >> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer. >> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4] >> repositories on Apache Bintray. Though they are already prepared for >> hosting RPM and DEB packages respectively, and there is a way of linking >> them to apache.org/dist/ignite page, there is possible alternative in >> storing there only plain directory layout corresponding to each repository >> type (RPM and DEB) and manage this layout (repodata, distributions, >> versions, etc.) by ourselves, having more control over repositories but >> lacking some simplicity of deploying new releases. WDYT? Should we try >> Cassandra approach? They are storing their DEB packages as I described >> above [5]. >> >> Also — a question arose while I was working on this issue: which OSes >> (and which versions of each) are we going to support (if we are going) in >> terms of step-by-step list? Currently RPM packages are tested only with >> latest CentOS (and, respectively — RHEL), but there are a lot more >> RPM-based distributives [6] some of which are more o less popular among OS >> community (ALT, Fedora, openSUSE, etc.). >> >> >> [1] https://issues.apache.org/jira/browse/IGNITE-7647 >> [2] https://github.com/apache/ignite/pull/3703 >> [3] https://bintray.com/apache/ignite-rpm >> [4] https://bintray.com/apache/ignite-deb >> [5] https://bintray.com/apache/cassandra/debian#files/ >> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions >> >> >> >> >> > On 15 Mar 2018, at 22:15, Petr Ivanov <mr.wei...@gmail.com> wrote: >> > >> > I suppose that most everything if not all from libs/options will go to >> OPTIONAL (I’d call it simply ‘apache-ignite-libs'). >> > More precise lib selection (if something from optional would better to >> have in core package) will be discussed right after preliminary split >> architecture agreement. >> > >> > >> > >> >> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dpavlov....@gmail.com> wrote: >> >> >> >> I like idea of keeping simple system of modules, so +1 from me. >> >> >> >> Where optional libs (e.g Direct IO plugin) would be included, would it >> be >> >> core or optional? >> >> >> >> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dma...@apache.org>: >> >> >> >>>> >> >>>>> >> >>>>> How big would be a final core module? >> >>>> Around 30M. Can be shrinked to ~15M if separate Visor and create >> it’s own >> >>>> package. >> >>> >> >>> >> >>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and >> >>> propose the simplest modular system: >> >>> >> >>> - core module that includes basic Ignite capabilities including SQL, >> >>> compute grid, service grid, k/v >> >>> - optional module hosts the rest - ML, streamers integration (kafka, >> >>> flink), kubernetes, etc. >> >>> >> >>> What do you think? >> >>> >> >>> -- >> >>> Denis >> >>> >> >>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr.wei...@gmail.com> >> wrote: >> >>> >> >>>> *DEB package >> >>>> >> >>>> >> >>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr.wei...@gmail.com> wrote: >> >>>>> >> >>>>> Considering that DEV package for now is almost platform independent >> >>> (its >> >>>> a java application more or less), that package will work almost on >> any >> >>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc. >> >>>>> The only restriction is existence of systemctl (systemd) service >> >>> manager >> >>>> — we are dependent on it. >> >>>>> >> >>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’ >> and >> >>>> package has no arch or dist suffix — it will work on CentOS, RHEL, >> >>> Fedora, >> >>>> etc. with presence of aforementioned systemd. >> >>>>> >> >>>>> >> >>>>> >> >>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <dsetrak...@apache.org >> > >> >>>> wrote: >> >>>>>> >> >>>>>> Will Debian package work for Ubuntu? >> >>>>>> >> >>>>>> D. >> >>>>>> >> >>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr.wei...@gmail.com> >> >>>> wrote: >> >>>>>> >> >>>>>>> Not a problem, rather nuisance. Also, when we will move to >> official >> >>>>>>> repositories, there can be a problem from OS community. >> >>>>>>> >> >>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB >> package >> >>>> build >> >>>>>>> (package layout / install scripts) for speeding up things and >> >>> excluding >> >>>>>>> possible duplication and desynchronisation, so its a matter of >> ’sit >> >>>> and do’ >> >>>>>>> rather then some technical research. Thats why I rose discussion >> >>> about >> >>>>>>> future package architecture, so that after agreement I'm be able >> to >> >>>> pack >> >>>>>>> both RPM and DEB identically. >> >>>>>>> >> >>>>>>> Yet, if you insist, I can create DEB package according to current >> RPM >> >>>>>>> layout in no time. >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan < >> dsetrak...@apache.org> >> >>>>>>> wrote: >> >>>>>>>> >> >>>>>>>> Peter, >> >>>>>>>> >> >>>>>>>> I don't think the package size of 280M is going to be a problem >> at >> >>>> all, >> >>>>>>> but >> >>>>>>>> what you are suggesting can be an improvement down the road. >> >>>>>>>> >> >>>>>>>> In the mean time, I think our top priority should be to provide >> >>>> packages >> >>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough. >> >>>>>>>> >> >>>>>>>> Agree? >> >>>>>>>> >> >>>>>>>> D. >> >>>>>>>> >> >>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr.wei...@gmail.com> >> >>> wrote: >> >>>>>>>> >> >>>>>>>>> Hi, Igniters! >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd >> >>> like >> >>>> to >> >>>>>>>>> move >> >>>>>>>>> forward to further improve and widen AI delivery through >> packages. >> >>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about >> 280M+ >> >>>> and, >> >>>>>>> to >> >>>>>>>>> improve usability and significantly reduce required download >> >>> sizes, I >> >>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as >> >>>> follows: >> >>>>>>>>> - CORE >> >>>>>>>>> - bin >> >>>>>>>>> - config >> >>>>>>>>> - libs (!optional) >> >>>>>>>>> - OPTIONAL LIBS >> >>>>>>>>> - BENCHMARKS >> >>>>>>>>> - DOCS (?) >> >>>>>>>>> - EXAMPLES >> >>>>>>>>> - .NET PLATFORM FILES >> >>>>>>>>> - C++ PLATFORM FILES >> >>>>>>>>> >> >>>>>>>>> This architecture, as I assume, will add flexibility (no reason >> to >> >>>>>>> download >> >>>>>>>>> all 280M+ of binaries where you are to run only core node >> >>>> functionality) >> >>>>>>>>> and >> >>>>>>>>> maintainability (you are in full control of what is installed on >> >>> your >> >>>>>>>>> system). >> >>>>>>>>> >> >>>>>>>>> After successful architecture choice, same scheme are planned >> to be >> >>>>>>> used in >> >>>>>>>>> DEB packages as well. >> >>>>>>>>> >> >>>>>>>>> WDYT? >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> -- >> >>>>>>>>> Sent from: http://apache-ignite-developer >> s.2346864.n4.nabble.com/ >> >>>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>> >> >>>> >> >>>> >> >>> >> > >> >> > >
