Hi,
I'd like to share some experiences of our installations and what we have
done and we haven't.
We've used many servers for apache part, and galera cluster for Mariadb
part. We used nginx for load balancing and I shared a new entry in the
wiki about our conf:
https://wiki.koha-community.org/wiki/Nginx_conf_sample_for_load_balancing_with_many_servers
The main problem always occured from Zebra site as you said David.
Indexing for more millions records was horrible of course.
After indexing it was easy to copy data and create new server while
working in a huge VM environment. Not only indexing data but also with
many requests from users also used a lot of resources of hardwares.
Since Zebra part was the weak point, we preferred to power database
servers and change a lot of code just to search directly from database
instead of using Zebra. This gave us quite big flexibility in our
installation.
In some problems, if we need to do re-indexing Zebra, we used nigths of
the day to do that and used some extra servers for indexing and when
finished we copied the data.
I personally spend a lot of time for Zebra things and I really hate its
structure. I hope Elastic search will be good for cluster environments.
We also thought Dockerized environment for a scale for large
installations but we could not find necessary money and time to finish.
Best regards
Mengu
Devinim Ltd.
On 11.06.2020 11:03, Ere Maijala wrote:
Hi,
This is a welcome discussion, thanks for starting it!
A couple of quick comments:
1. If you're running Elasticsearch and Z3950 Responder, I can't really
see a need to run Zebra at all anymore. Is there something that it's
really needed for?
3. Z3950 Responder is pretty lightweight, and it'll be the search index
doing most of the work. But I understand the need from fault tolerance
point of view. You could use YAZ Proxy or Metaproxy as the load balancer
for multiple Z3950 Responders. However, that's still a single point
managing the sessions.
--Ere
dc...@prosentient.com.au kirjoitti 11.6.2020 klo 10.52:
Hi all,
For a long time, I've been thinking about how Koha could scale better, and
I'm wondering what other people are thinking about this topic. I think Mengu
is probably the best expert on this topic that we have on Earth, so
especially curious about his input.
It's relatively easy to add more Apache web servers, more MySQL nodes, and
more Elasticsearch nodes. but the same might not be said for other parts of
Koha like the following:
1. Zebra
a. In theory, you could load balance queries to a cluster of Zebra
listeners
b. But indexing updates to many Zebra databases could be tricky (I
don't know if Zebra's database can support many readers). I suppose you
could alter rebuild_zebra.pl to update multiple databases in a variety of
different ways. I imagine Mengu could speak to this, as I recall him talking
once about needing to do work to make Zebra scale?
2. I'm not familiar enough with the SIP server to say
a. Off the top of my head, I think a TCP load balancer would be good
enough here
3. Z3950 Responder?
a. I'm not familiar with this daemon
4. What about cronjobs? Only 1 instance should execute the cronjobs.
a. I suppose a person could disable cronjobs on all Koha nodes
except for 1? It wouldn't be very robust in terms of availability but it
would technically work most of the time.
Any other thoughts?
David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
_______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
_______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
