Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-08 Thread Steve Hill
On 08.10.14 15:05, Amos Jeffries wrote: New patch added to bug 4088. Please see if it resolves the external_acl_type leak. Seems to fix the problem - thank you! -- - Steve Hill Technical Director Opendium Limited http://www.opendium.com Direct contacts: Instant messager: xmpp:s

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-08 Thread Amos Jeffries
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 8/10/2014 4:40 a.m., Steve Hill wrote: > On 30.09.14 16:13, Amos Jeffries wrote: > >>> I'm trying to figure out if there's a way of convincing >>> valgrind to dump info about all the currently allocated memory >>> while the program is still running

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-07 Thread Steve Hill
On 30.09.14 16:13, Amos Jeffries wrote: I'm trying to figure out if there's a way of convincing valgrind to dump info about all the currently allocated memory while the program is still running - there would be a lot of legitimate stuff in the report, but hopefully a few hundred MB of memory tha

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-06 Thread Steve Hill
On 01.10.14 13:54, Amos Jeffries wrote: I recently opened a bug about this, that I will update now: http://bugs.squid-cache.org/show_bug.cgi?id=4088 Thank you for the reminder. I will start work on this next. I'm afraid the patch you added to that bug report doesn't work for me (in fact, i

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-03 Thread Victor Sudakov
Rafael Akchurin wrote: > > Tell me please, after implementing your guide, do you enjoy Kerberos > > proxy authentication in Firefox, or does it fall back to Basic auth? > I believe I do (but you made me doubt:) Can you please doublecheck which method your Firefox is using? All the howtos I have

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-03 Thread Rafael Akchurin
users@lists.squid-cache.org Subject: Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7. Rafael Akchurin wrote: > > Does Kerberos proxy authentication work with Firefox (Windows) at all? > > Success stories and recipes, anyone? > > If you see 'received type 1 NTLM token' m

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-03 Thread Victor Sudakov
Rafael Akchurin wrote: > > Does Kerberos proxy authentication work with Firefox (Windows) at all? > > Success stories and recipes, anyone? > > If you see 'received type 1 NTLM token' message it means your IE was > not able to use the Kerberos auth and have chosen NTLM instead. Any ideas how I can

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-03 Thread Rafael Akchurin
ds, Rafael From: squid-users on behalf of Victor Sudakov Sent: Friday, October 3, 2014 1:51 PM To: Amos Jeffries Cc: squid-users@lists.squid-cache.org Subject: Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7. Amos Jeffries wrote: > There are two thin

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-03 Thread Victor Sudakov
Amos Jeffries wrote: > There are two things you can do to further improve performance: > 1) converting from NTLM to Kerberos authentication. Does Kerberos proxy authentication work with Firefox (Windows) at all? Success stories and recipes, anyone? NTLM did work both with Firefox and MSIE and e

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-03 Thread Victor Sudakov
Amos Jeffries wrote: [dd] > > Bingo! After setting "ident_access deny all" squid does not grow > > infinitely any more. However, it remains a major CPU hog. > > > > Yay. Any news on the bug patch? Will try during the weekend. I can live without IDENT lookups for a while, they are not very imp

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-01 Thread Steve Hill
On 01.10.14 13:19, Michele Bergonzoni wrote: I have an external ACL defined as: external_acl_type preauth cache=0 children-max=1 concurrency=100 ttl=0 negative_ttl=0 %SRC %>{User-Agent} %URI %METHOD /usr/sbin/squid-preauth It is well known that external ACLs with ttl=0 and cache=0 leak RAM: I h

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-01 Thread Amos Jeffries
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2/10/2014 1:19 a.m., Michele Bergonzoni wrote: >> I have an external ACL defined as: external_acl_type preauth >> cache=0 children-max=1 concurrency=100 ttl=0 negative_ttl=0 %SRC >> %>{User-Agent} %URI %METHOD /usr/sbin/squid-preauth > > It is well

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-01 Thread Michele Bergonzoni
I have an external ACL defined as: external_acl_type preauth cache=0 children-max=1 concurrency=100 ttl=0 negative_ttl=0 %SRC %>{User-Agent} %URI %METHOD /usr/sbin/squid-preauth It is well known that external ACLs with ttl=0 and cache=0 leak RAM: I had this problem and discovered the reason onl

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-01 Thread Steve Hill
On 29.09.14 10:04, Steve Hill wrote: I _think_ I have narrowed it down to something ICAP related Looks like I was wrong - it actually seems to be external ACL related. I have an external ACL defined as: external_acl_type preauth cache=0 children-max=1 concurrency=100 ttl=0 negative_ttl=0 %SR

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-01 Thread Amos Jeffries
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 1/10/2014 8:45 p.m., Victor Sudakov wrote: > Amos Jeffries wrote: > > You have 200 MB of RAM locked up in IDENT lookups. > > Probably http://bugs.squid-cache.org/show_bug.cgi?id=3803 I have temporarily disabled IDENT-rel

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-10-01 Thread Victor Sudakov
Amos Jeffries wrote: > >>> > >>> You have 200 MB of RAM locked up in IDENT lookups. > >>> > >>> Probably http://bugs.squid-cache.org/show_bug.cgi?id=3803 > >> > >> I have temporarily disabled IDENT-related acls. Squid still grows > >> in memory, but more slowly. So IDENT was certainly one of the

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-30 Thread Amos Jeffries
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 1/10/2014 3:59 a.m., Steve Hill wrote: > > On 30.09.14 15:13, Amos Jeffries wrote: > >> IIRC the valgrind report is strangely at the end of mgr:info >> rather than mgr:mem. > > In that case the wiki is wrong. :) Anyway, it doesn't show up on > ei

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-30 Thread Amos Jeffries
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 30/09/2014 6:33 p.m., Victor Sudakov wrote: > Victor Sudakov wrote: >> Amos Jeffries wrote: >>> >>> You have 200 MB of RAM locked up in IDENT lookups. >>> >>> Probably http://bugs.squid-cache.org/show_bug.cgi?id=3803 >> >> I have temporarily disa

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-30 Thread Steve Hill
On 30.09.14 15:13, Amos Jeffries wrote: IIRC the valgrind report is strangely at the end of mgr:info rather than mgr:mem. In that case the wiki is wrong. :) Anyway, it doesn't show up on either of them for me so I guess you might be right that it isn't "real" leaked memory. (I still thought

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-30 Thread Steve Hill
On 29.09.14 13:39, Eliezer Croitoru wrote: Hey Steve, Can you share the basic cache manager requests statistics and the up time for the service? (mgr:info) This is with 8 workers and was restarted this morning, about 6 hours ago. As you can see, it's using about 5GB at the moment - as mentio

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-30 Thread Amos Jeffries
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 29/09/2014 10:04 p.m., Steve Hill wrote: > On 28.09.14 08:34, Eliezer Croitoru wrote: > >> Also to minimize the leak source try to share more information >> about your usage. Are you using SMP workers or not? What is the >> mem and info options in

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-30 Thread Eliezer Croitoru
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 About the memory page size: getconf PAGESIZE getconf PAGE_SIZE One of the above should provide the basic information. Waiting for some more info. Eliezer On 09/30/2014 04:10 PM, Amos Jeffries wrote: > The unusual one you should expect to be first i

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-30 Thread Amos Jeffries
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 1/10/2014 1:24 a.m., Victor Sudakov wrote: > Eliezer Croitoru wrote: >>> Attaching two cachemgr reports: right after squid restart and >>> several hours later (grown to 816M in SIZE). >> I am not very good at calculating what I do not know how to y

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-30 Thread Victor Sudakov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Eliezer Croitoru wrote: > > Attaching two cachemgr reports: right after squid restart and > > several hours later (grown to 816M in SIZE). > I am not very good at calculating what I do not know how to yet! > What I do understand is that "Cumulative all

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-30 Thread Eliezer Croitoru
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/30/2014 08:33 AM, Victor Sudakov wrote: > Attaching two cachemgr reports: right after squid restart and > several hours later (grown to 816M in SIZE). I am not very good at calculating what I do not know how to yet! What I do understand is that "

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-30 Thread Eliezer Croitoru
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/30/2014 05:11 AM, Victor Sudakov wrote: >> >> Can you share the basic cache manager requests statistics and the >> up time for the service? (mgr:info) >> >> This would give us a basic idea of the load\requests needed to >> reproduce it. > > I

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-29 Thread Victor Sudakov
Amos Jeffries wrote: > >>> > >>> I have tried looking at the cachemgr.cgi memory counters but > >>> they are too numerous and cryptic for me. > >> > >> Would you mind posting a copy of that mgr report? perhapse we can > >> see something unusual. > >> > > > > I am attaching two reports. The firs

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-29 Thread Victor Sudakov
> > Can you share the basic cache manager requests statistics and the up > time for the service? > (mgr:info) > > This would give us a basic idea of the load\requests needed to reproduce it. I am not Steve but in case you care to look at my statistics as well, I am attaching it. -- Victor Sud

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-29 Thread Eliezer Croitoru
Hey Steve, Can you share the basic cache manager requests statistics and the up time for the service? (mgr:info) This would give us a basic idea of the load\requests needed to reproduce it. Eliezer On 09/29/2014 12:04 PM, Steve Hill wrote: For what its worth, I'm also seeing a big memory le

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-29 Thread Steve Hill
On 28.09.14 08:34, Eliezer Croitoru wrote: Also to minimize the leak source try to share more information about your usage. Are you using SMP workers or not? What is the mem and info options in the cache manager page data? Did you tried "cache deny all" acl to minimize the source of the leak?

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-29 Thread Amos Jeffries
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 29/09/2014 8:16 p.m., Victor Sudakov wrote: > Amos Jeffries wrote: >>> >>> I have tried looking at the cachemgr.cgi memory counters but >>> they are too numerous and cryptic for me. >> >> Would you mind posting a copy of that mgr report? perhapse

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-28 Thread Victor Sudakov
Amos Jeffries wrote: > If I have to stick to some old unsupported version, I'd > choose 2.7. It has worked flawlessly until it was deleted > from the ports tree and I had to migrate to squid3. > > > > > > Also, are you using HTTPS interception? there are known memory > > leaks in O

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-28 Thread Amos Jeffries
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 29/09/2014 1:40 p.m., Amos Jeffries wrote: > On 29/09/2014 1:41 a.m., Victor Sudakov wrote: >> Eliezer Croitoru wrote: >>> On 09/28/2014 12:18 PM, Victor Sudakov wrote: If I have to stick to some old unsupported version, I'd choose 2.7. It

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-28 Thread Victor Sudakov
Victor Sudakov wrote: > > > Did you tried "cache deny all" acl to minimize the source of the leak? > > I suppose this somehow would defeat the purpose of the Squid cache. I have tried "cache deny all" and squid is still growing. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN sip:suda...@sibptus.tom

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-28 Thread Amos Jeffries
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 29/09/2014 1:41 a.m., Victor Sudakov wrote: > Eliezer Croitoru wrote: >> On 09/28/2014 12:18 PM, Victor Sudakov wrote: >>> If I have to stick to some old unsupported version, I'd choose >>> 2.7. It has worked flawlessly until it was deleted from the

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-28 Thread Eugene M. Zheganin
Hi. On 28.09.2014 21:14, Victor Sudakov wrote: How do I figure this out? I compiled squid from the FreeBSD ports collection, don't remember any tunable option concerning SMP workers. They're off by default. Plus, it's a 3.x feature. If you ported your config from your 2.7 installation, they're

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-28 Thread Victor Sudakov
Eliezer Croitoru wrote: > > Which is the place for squid related bugs and specially for the stable > versions. > The needed information is also here: > http://wiki.squid-cache.org/SquidFaq/BugReporting > > Since it's squid 3.4.7 or 8 you will have manage interface via plain > http so take a loo

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-28 Thread Victor Sudakov
Eliezer Croitoru wrote: > On 09/28/2014 12:18 PM, Victor Sudakov wrote: > > If I have to stick to some old unsupported version, I'd choose 2.7. > > It has worked flawlessly until it was deleted from the ports tree and > > I had to migrate to squid3. > > Victor, > Please start by declaring the envi

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-28 Thread Eliezer Croitoru
On 09/28/2014 12:18 PM, Victor Sudakov wrote: If I have to stick to some old unsupported version, I'd choose 2.7. It has worked flawlessly until it was deleted from the ports tree and I had to migrate to squid3. Victor, Please start by declaring the environment in the form of: is it a intercept

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-28 Thread Victor Sudakov
Eugene M. Zheganin wrote: > > squid 3.4.8 and 3.4.7 are leaking memory at the rate of several Mbytes > > per minute on FreeBSD. Squid 3.4.8 leaks faster than 3.4.7. > > > > The settings are more than modest: > > > > cache_mem 128 MB > > cache_dir ufs /webcache/cache 512 16 256 > > memory_pools off

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-28 Thread Eugene M. Zheganin
Hi. On 28.09.2014 12:28, Victor Sudakov wrote: squid 3.4.8 and 3.4.7 are leaking memory at the rate of several Mbytes per minute on FreeBSD. Squid 3.4.8 leaks faster than 3.4.7. The settings are more than modest: cache_mem 128 MB cache_dir ufs /webcache/cache 512 16 256 memory_pools off # neit

Re: [squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-28 Thread Eliezer Croitoru
Well you can start with: http://bugs.squid-cache.org/ Which is the place for squid related bugs and specially for the stable versions. The needed information is also here: http://wiki.squid-cache.org/SquidFaq/BugReporting Since it's squid 3.4.7 or 8 you will have manage interface via plain ht

[squid-users] leaking memory in squid 3.4.8 and 3.4.7.

2014-09-27 Thread Victor Sudakov
Colleagues, squid 3.4.8 and 3.4.7 are leaking memory at the rate of several Mbytes per minute on FreeBSD. Squid 3.4.8 leaks faster than 3.4.7. The settings are more than modest: cache_mem 128 MB cache_dir ufs /webcache/cache 512 16 256 memory_pools off # neither "on" nor "off" have any effect on