This is another area where I think our thoughts align David. I've also wanted to split his into even smaller units for a while.. a clear example is our move to background jobs. I see those as prime targets for horizontal scaling.. but as you've highlighted, at the moment to get the library of core code you may want to write a job you need to install the whole app.. koha-libs would certainly help with that
On Tue, 20 Apr 2021, 2:12 am , <dc...@prosentient.com.au> wrote: > Hi Mason, > > [FYI this started as a response and turned into a > stream-of-consciousness.][Also really keen to hear from Tomas and Agustin > about their work with Docker/Kubernetes/Helm/etc.] > > The idea with "koha-core" was to separate out the third-party systems away > from the core application. In this case, "koha-libs" would be about > separating the core libraries away from the core application. > > Consider that the "koha-core" package contains the full Koha application > (including code, templates, images, other assets, etc) and its application > dependencies (e.g. at, cron, daemon, starman, sudo, unzip, xmlstarlet, yaz, > mysql-client). My latest koha-core package alone was 38MB. A "koha-libs" > package would probably be measured in terms of KB. > > I'm thinking about the absolute minimum required to run things like the > SIP server or some other service/microservice (e.g. z3950-responder, REST > service, etc). The koha-libs package would need to contain the Koha .pm > files and koha-libs would depend on koha-perldeps. > > In theory, over time, some of the libraries and dependencies would > probably move out of koha-libs and koha-perldeps to live in the individual > services, but I don't have a full vision for that yet. > > You could think about a koha-libs package as a Perl API rather than a HTTP > API, which could be used by Koha's existing background services, and it > could be used by new separate services. > > I have a vision for creating a service that depends on koha-libs, points > at a koha-conf.xml file, and then implements its own functionality separate > from the main Koha codebase. > > It could allow for a lot of power and flexibility. > > -- > > That said, maybe I'm thinking too small and too Debian-focused. Maybe a > CPAN distribution of Koha/ modules makes more sense (or both koha-libs and > a CPAN distribution). Apparently we could use the cpanfile to specify the > dependencies for the CPAN distribution too. Then you could install it on > any suitable OS/container. In a container environment, you could always > install the Koha/ modules and dependencies on a volume, and then share that > volume with your containers. That would have some pros and cons. > > I'm mostly just brainstorming. Thinking about how we can re-organize > things to take advantage of modern technology. But then to what end? > > One advantage of Docker is to run different versions of the same software > on the same virtual machine... > > While I was thinking about 1+ containers per Koha library, maybe we do > just go with a 20.05 Koha container, a 20.11 Koha container, a 21.05 Koha > container. I suppose each of those versions could have multiple containers > though. For example, a SIP server container, a Z39.50 responder container, > a Zebra indexer container, a Zebra server container, etc. Technically > speaking, if we planned the volumes right, the Koha "etc" config and > "share" and "var" volumes could all be shared across those containers. Of > course, that means that all the containers have to be running on the same > host. At the scale of most Koha libraries, that would probably be fine. I > suppose the main change here would be to the debian scripts and koha-common > init script... > > -- > > Overall, I suppose though most vendors would win from improved > multi-tenancy. Having 100 Starman instances for 50 Koha libraries where > only 3 libraries might have requests at a time is massive overhead. > > I imagine improved multi-tenancy coupled with multiple versions per host > via Docker... that would be a nicer setup. > > Preloading Koha::REST::V1 and getting its work done in the master starman > process rather than the child processes would be better too... > > Ok, that's enough ranting for me for now heh. Too many ideas and never > enough time. > > David Cook > Software Engineer > Prosentient Systems > Suite 7.03 > 6a Glen St > Milsons Point NSW 2061 > Australia > > Office: 02 9212 0899 > Online: 02 8005 0595 > > -----Original Message----- > From: Koha-devel <koha-devel-boun...@lists.koha-community.org> On Behalf > Of Mason James > Sent: Friday, 16 April 2021 3:50 PM > To: koha-devel@lists.koha-community.org > Subject: Re: [Koha-devel] Packaging idea (for decoupling systems, Docker, > etc) > > hi David > > i think we may already have this functionality with the 'koha-core' > package? > > > On 15/04/21 3:32 pm, dc...@prosentient.com.au wrote: > > > > Hey all, > > > > What do people think about creating a “koha-libs” package which just > contains Koha’s libraries (ie C4/, Koha/, etc)? > > > > I’m not as familiar with DEB packaging as I am with RPM packaging, but I > recently did it with a RPM-based project I manage. It allowed me to easily > create other services with the same dependencies. I created these other > services in other Docker containers, and I was able to use my core > application libraries without having to install the entire application in > every container. I share the application configuration file between them > and that’s it. > > > > It’s not a perfect solution. Really what I want is app-service1, > app-service2, app-service3, and app-common with app-common containing the > shared libraries. But I like it as an intermediate step. > > > > And I would be lying if it didn’t have pros and cons. One con is > updating a library in app-libs, when I need a change in app-service1, and > then making sure that all my deployments have the right updated library. > More overhead than just having a monolithic application. On the other hand, > if I want to make a change just to app-service1, I can just update it > without having to affect any other parts of the application. That added > overhead also gives incentive for writing cleaner testable code which does > the right thing in the first place. > > > > Anyway, just food for thought. > > > > David Cook > > > > Software Engineer > > > > Prosentient Systems > > > > Suite 7.03 > > > > 6a Glen St > > > > Milsons Point NSW 2061 > > > > Australia > > > > Office: 02 9212 0899 > > > > Online: 02 8005 0595 > > > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel@lists.koha-community.org > > https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : https://www.koha-community.org/ git : > > https://git.koha-community.org/ bugs : > > https://bugs.koha-community.org/ > > _______________________________________________ > Koha-devel mailing list > Koha-devel@lists.koha-community.org > https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : https://www.koha-community.org/ git : > https://git.koha-community.org/ bugs : https://bugs.koha-community.org/ > > > _______________________________________________ > Koha-devel mailing list > Koha-devel@lists.koha-community.org > https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : https://www.koha-community.org/ > git : https://git.koha-community.org/ > bugs : https://bugs.koha-community.org/ >
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/