On Mon, Jun 25, 2018 at 6:49 PM, Rasto Rickardt <[email protected]> wrote:
> Hello, > > If there is no more RAM available, i would test sqlite3 plugin, as > sqlite3 is more suited for limited resources usage. Youl will possibly > need to change your workflow to export sqlite3 files and load it > somewhere else, but it should be a lot cheaper memory wise. > > Thanx. Seems that I have to stick with mysql at least at this phase. This one is swapping pretty heavily, i would suggest to set to around 10 > for 4GB RAM. > > sysctl vm.swappiness=10 so it will start using swap when available RAM > is around 400MB. > This is already done. > r. > > On 06/25/2018 05:35 PM, Alex K wrote: > > Thanks Dariush. Appreciate your feedback. > > > > I was testing several stripped down kernels by compiling and removing > > most of unused modules. The gain I had was in the range of few MB. > > Seems that I have to find a more elegant approach at uacctd > > configuration since with current setup I am loading several mysql > > plugins so as to be able to filter traffic with directions, networks and > > ports. > > I was testing an edge case where these plugins may rise up to 210 > > instances thus leading to this memory usage. Normally this will not be > > done so the normal case will be using 2.5 GB of RAM instead of 3.5 GB. > > Thus in normal cases seems that I am ok. > > > > Cheers > > > > > > On Mon, Jun 25, 2018 at 6:07 PM, Dariush Marsh-Mossadeghi > > <[email protected] <mailto:[email protected]>> wrote: > > > > OK, so it’s effectively an embedded system scenario, with a fixed > > config hardware, or similar > > No silver bullets or one-liner fixes here :-\ > > > > You’ve a number of options to slim down your memory footprint > > - Strip Debian of all the packages you don’t need, learn a lot about > > kernel tuning, and tune the kernel to your needs. > > - Start looking at one of the more embedded systems oriented > > distros, rather than OOTB Debian. Although Debian is skinnier than > > most, there’s a lot that can be done to minimise footprint. The fact > > that you’ve got a 64bit Debian9 running on it makes me fairly > > optimistic that you could run pretty much any Debian derivative. > > - Profiling your userspace software stack and see if there are any > > dependencies you don’t need, strip them out. Whether you can do this > > will very much depend on the modularity of the components. > > > > All of the above lead you down the path of maintaining forks and > > compiling your own packages, to a greater or lesser extent. That’s a > > maintenance overhead you want to avoid unless you have to. > > Having said all that, if you’re looking at deploying thousands of > > units, the economics of software maintenance may stack up for you. > > > > HTH > > Dariush > > > > > >> On 25 Jun 2018, at 14:32, Alex K <[email protected] > >> <mailto:[email protected]>> wrote: > >> > >> Let me change the posting style... :) > >> > >> On Mon, Jun 25, 2018 at 3:51 PM, Dariush > >> Marsh-Mossadeghi <[email protected] > >> <mailto:[email protected]>> wrote: > >> > >> OK, so we’re moving from bottom-posting to top-posting… > >> that’ll make it interesting for other readers ;-) > >> > >> The output of free doesn’t look desperate, but it is starting > >> to look a bit tight. > >> You’ve got about a gig of buffer/cache, which the kernel will > >> evict if it needs it. > >> You’ve got 200M of genuinely free memory. > >> What is potentially a little concerning is the gig of swap in > >> use, that may or may not be a problem depending on what else > >> is running on the box and how it’s memory use varies over time. > >> > >> I’m not intimatley familiar with the internals of uacctd’s > >> mysql plugins, but my advice would be: > >> > >> - Monitor the memory usage. If it doesn’t vary much over time > >> you could go on for years and be just fine. Keep a close watch > >> on swap usage, if that varies a lot or grows over time you’ll > >> want to do something about it. > >> > >> Monitoring of last 2 weeks is showing that free mem is fluctuating > >> form 50M to 220 MB and swaps remains steady. Thats why I am trying > >> to find a solution. > >> > >> > >> - Read up on OOM Killer, what it does and how it behaves. If > >> you see OOM Killer entries in your logs, it’s time to think > again > >> > >> I am aware of that. thanx > >> > >> > >> - Can you put some more memory in the box? 8GB of RAM will > >> cost you about £60. How much is your time worth ? > >> > >> This is not an option. What I have is 4 GB. This is not just a > >> personal project. There are going to be thousands of such > >> installations > >> > >> HTH > >> Dariush > >> > >> > >>> On 25 Jun 2018, at 12:32, Alex K <[email protected] > >>> <mailto:[email protected]>> wrote: > >>> > >>> Thanx for the reply. > >>> > >>> The output of free is the following: > >>> > >>> free > >>> total used free shared > >>> buff/cache available > >>> Mem: 4046572 2832576 152012 784240 > >>> 1061984 204248 > >>> Swap: 3906556 1086080 2820476 > >>> > >>> While is as below when stopped: > >>> > >>> free > >>> total used free shared > >>> buff/cache available > >>> Mem: 4046572 520352 3223884 14916 > >>> 302336 3286952 > >>> Swap: 3906556 485040 3421516 > >>> > >>> Seems that mysql plugins are reserving quite some memory as they > >>> list first in htop when sorted with memory. > >>> > >>> Thanx, > >>> Alex > >>> > >>> > >>> On Mon, Jun 25, 2018 at 2:22 PM, Dariush > >>> Marsh-Mossadeghi <[email protected] > >>> <mailto:[email protected]>> wrote: > >>> > >>> > >>>> On 25 Jun 2018, at 11:54, Alex K <[email protected] > >>>> <mailto:[email protected]>> wrote: > >>>> > >>>> Hi all, > >>>> > >>>> I have a setup with uacctd monitoring traffic of several > >>>> interfaces through NFLOG. > >>>> With uacctd stopped I see that the server (a relatively > >>>> small device with 4 GB of RAM) consumes 450MB of RAM. Once I > >>>> start uacctd the mem usage goes up to 3.5 GB. I am using > >>>> mysql plugin and this is running on Debian9 64 bit. > >>>> > >>>> Is there any tweaks I can use to put a limit on the memory > >>>> usage of uacctd. > >>>> > >>>> Thanx, > >>>> Alex > >>>> _______________________________________________ > >>>> pmacct-discussion mailing list > >>>> http://www.pmacct.net/#mailinglists > >>>> <http://www.pmacct.net/#mailinglists> > >>> > >>> > >>> tldr; post the output of the free command when uacctd is > >>> running and I’ll do my best to interpret it for you :-) > >>> > >>> Is it _really_ using it, or is it just not completely unused? > >>> > >>> The linux kernel generally does pretty good job of keeping > >>> stuff that might be useful in memory, but getting rid of it > >>> very very quickly if the space is needed for something else. > >>> The challenge you face is that most of the userspace tools > >>> come with a long list of caveats about what they appear to > >>> report vs what’s really happening, this is mainly due to the > >>> way the kernel shares memory between processes, and its use > >>> of as much memory as possible for various caches. > >>> > >>> Some threads worth reading if you’re not familiar with the > >>> in’s and out’s of linux kernel memory management... > >>> > >>> https://www.linuxatemyram.com/ > >>> https://stackoverflow.com/questions/4802481/how-to-see- > top-processes-sorted-by-actual-memory-usage > >>> <https://stackoverflow.com/questions/4802481/how-to-see- > top-processes-sorted-by-actual-memory-usage> > >>> https://stackoverflow.com/questions/3784974/want-to- > know-whether-enough-memory-is-free-on-a-linux-machine-to-deploy-a-new-app/ > >>> <https://stackoverflow.com/questions/3784974/want-to- > know-whether-enough-memory-is-free-on-a-linux-machine-to-deploy-a-new-app/ > > > >>> > >>> HTH > >>> Dariush > >> > > > > > > _______________________________________________ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > > <http://www.pmacct.net/#mailinglists> > > > > > > > > > > _______________________________________________ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > > > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists >
_______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
