On 02/16/2015 08:00 PM, Eliezer Croitoru wrote:
Hey Yuri,
OK I have seen something...
Now we might need also the virtual memory which might be vsz.
And the cachemgr output is not from squidview..
The last image I have seen from cachemgr was much helpful(with 10 helpers).
From what I have seen until now squidGuard uses about 13 MB of ram constantly.
If this is what it costs to run a squidGuard helper you should consider that it
has some peak usage time and lower bound usage time and you will need to
calculate how much helpers you need to provide
smooth surfing on peak usage time.
You should consider this peak usage as the required memory from squidGuard for
smooth operation.
Since not you and not will be able to change the basic memory
footprint(required) for squidGuard you will need to somehow decide if you
require a much efficient software or to adjust your system
accordingly.
I would agree that 100 * 13 MB of ram means about 1.3 GB of ram usage and might not be
acceptable for a "simple" filtering mechanism.
It is because squidGuard uses Berkeley DB which uses a default cache size of
256 KB for each URL table.
And each squidGuard process does this. And of course the OS keeps a full copy
in the file system buffer cache.
So with 100 redirectors and N URL tables, N * 100 * 256K is used by buffers for
Berkeley DB alone.
The main reasons for the unknown need for 100 helpers might be since it was not
designed to be used this way.
squidGuard does not support the Squid feature 'concurrency' for
url_rewrite_children. ufdbGuard does.
With concurrency, latency goes down and the number of processes can also be
reduced.
I must say that squidGuard is a very good piece of software but it lacks couple
things which might make it un-usable for some if not many systems and maybe
including yours.
Indeed in your case there is a chance that if you will even install a full
blown DB that will be stored all in ram and you will write a helper that will
mimic squidguard functionality\logic with
concurrency support you will get much less memory consumption with much more
efficient request handling.
It is much smarter to write a helper which will have a overall efficiency mark
on it then to add into squid something that might not be even needed in the
first place.
ufdbGuard has all that you need. It holds one copy of the database in memory
without allocating extra memory.
The ufdbGuard URL redirectors are leightweight processes using very little
memory and
the redirectors support concurrency so less are required.
Off list, Yuri asked help for compilation on Solaris and I made a fix for
ufdbGuard which is availabe for Yuri.
If there are no further complaints from Yuri, I will release the patch in one
or two days.
Marcus
For now before writing any helper etc I will want to see the cachemgr interface output
for the "redirector" option in two cases:
- startup
- after midnight when there are lots of helpers running under squid
You can see that squid since version 3.2 squid offers access to the squid cache
manager interface using a simple url such as:
"http://filter:3128/squid-internal-mgr/redirector";
(using the visible_hostname and the forward proxy port of squid)
All The Bests,
Eliezer
On 16/02/2015 21:56, Yuri Voinov wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Now:
2015/02/16 23:10:23 kid1| store_swap_size = 29826351.50 KB
2015/02/16 23:10:24 kid1| storeLateRelease: released 0 objects
2015/02/16 23:15:01 kid1| Starting new redirector helpers...
2015/02/16 23:15:01 kid1| helperOpenServers: Starting 1/100
'squidGuard' processes
2015/02/17 01:40:15 kid1| Starting new redirector helpers...
2015/02/17 01:40:15 kid1| helperOpenServers: Starting 1/100
'squidGuard' processes
^Croot @ cthulhu / # ps -e -o user,pid,rss,comm
_______________________________________________
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
