Hi Andre, *, On Thu, Apr 7, 2011 at 1:37 PM, Andre Schnabel <andre.schna...@gmx.net> wrote:
> Please leave the technical discussion aside for a minute. No, cannot do that, as this is directly related to your point: > What *we* as a team of localizers need is a working and performant > setup to do our translation work. Yes, and I fully agree, and that of course is also my highest measuremen/goal. It is my true belief that the current setup will handle this just fine. > - we had no full localization on the server (but we will have to provide > this for 3.4 localization) See the other messages. What is timeconsuming (and no memory will help here, as it is is CPU bound) is importing of the new data, and time until it is ready after a server-restart. Nothing to do about it, apart from the admins actually doing the import using differently formed comments that make use of all of the assigned CPUs. > - hardly any team did full translation on the server, we "only" did > bugfixes See my post. I'm /sure/ the server can handle the full online-translation within pootle. I'm sure that the server will not get more than 10 apache requests per seconds on average, and the server can handle about 200 > There were some good reasons to have a rather good equipped server for > pootle. A memory leak/wasteful setup is not a good reason for this. > We knew, that pootle might be not the perfect solution regarding > memory usage, multithreading ... That we now cannot use the initialy > planned setup is very unfortunate. But we still need a system that > provides similar performance. Again: If performance is not satisfactory for translation work, I'll reconsider. But again: I don't see any evidence that performance could be increased by assigning more RAM to the VM. >> I clearly explained why I'm convinced that it is a memory leak. It >> doesn't free the memory, even when the machine is idling for hours. It >> doesn't need that memory, since when the worker is replaced by a fresh >> one, the functionality still works. > > No - it is just that this is totally irrelevant in the current situation. No it is not. > Even if pootle has a memory leak, we won't fix this within the next few > days. But we need to start translating asap! Again: It is /perfeclty working/ when restarting the worker threads. And the translator will not notice that the worker-thread has been replaced. The translator will not notice whether the few milliseconds he has to wait are * because the keep-alive expired and a new http-connection has to be negotiated * a server process expired and thus is restarted * seeking through the actual file for next/previous match took a little longer. > So - if there is any way to provide more ressources (memory) we should do > this and analyze the root cause later (I'm sure, pootle developers are > interested to help with this). /IF/ there are memory related problems, I'll assign more RAM. But again: everything I experienced so far says: More memory won't help at all. >> Again: >> * I don't have a problem with pootle taking half a day to import the >> files for a new release (one time thing, no big deal) >> * I don't have a problem with pootle taking very long for the very >> first hit of the frontpage after a reboot (system is not rebooted that >> often, also no big deal) >> * I don't have a problem with restarting pootle server-processes to >> workaround the memory-leak (whether you call it leak or not, I >> definitely call it leak). It limits the amount of concurrent worker >> processes, but that is not a problem since even with just the two as >> in the VM, you can easily serve 50 concurrent requests per second >> while benchmarking (resulting in being able to handle about 200 >> request/second of the frontpage, thus much reserves available. No >> problem) > > Maybe you don't have, but the translators will have a problem with this. > So either we can provide more ressources to the VM or we need a > different solution. Again: Those stuff that takes ages are /NOT/ solvable by assigning more ressources to the VM. Those are CPU-bound, and all of them are single-threaded. If the process maxes out a single CPU, that is as much speed as you can get, no matter how many RAM is sitting idle. * Pootle has a very, very poor first-start performance. → not a problem as the server will not be rebooted every other day. And in case this wasn't clear: Restarting the worker-processes will /not/ have the same effect, the user will not notice anything, * Pootle has poor performance when generating the zips for download (fist trigger per language and project) → This again is CPU bound, and again: More RAM will not help. This is the only case where the user can have a problem (and is part of the problem). It doesn't help to click multiple download links on one page to get the zips faster, on the contrary. Click one, wait for the processing to be done, and when you got the first one, feel free to download all the remaining ones. When multiple users request huge zips at the same time, all server processes are busy. Currently there are two, can be increased to 4, but that's not a big difference. It is CPU bound (and also does a little disk i/o) and I repeat myself once again: MORE RAM WILL NOT HELP!!! If you want a dedicated pootle server, than order a 64 core system. With that, you can hide pootle's weak spots. It will be idle 99% of the time, but for the point of time where 10+ people get the idea to download the zips at the same time, you won't run out of CPUs. > All other discussion is very academic at the moment. No - it is just very tiring that people still stick with their guesswork. Again: The only problem is creation of zips for download. This takes (in terms of web-responsiveness) /ages/, and blocks the server's process while it is performed. It is CPU-bound, and thus the amount of workers one can have is very limited. Pootle must be reconfigured to limiit these actions, or (more simple to implement as a workaround just create all the zips once per day in a cronjob (or better distributed during the day, as creating them all at once would again create the CPU-bottleneck) and only allow to download those. Thus you don't get 100%up-to-date versions of the file. But when you download the zips, you're not interested in the current state anyway, as just after you download someone else could have edited in pootle thus creating the same issue. Again: * Don't request additional zips of a project before the first one is delivered to you. ciao Christian -- Unsubscribe instructions: E-mail to l10n+h...@libreoffice.org Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.libreoffice.org/www/l10n/ All messages sent to this list will be publicly archived and cannot be deleted
