Re: [squid-users] Squid cpu usage 100% from few days ago !!

2016-08-13 Thread Marcus Kool
It seems that squid is doing a lot of calls to vfprintf. The first thing that comes to mind is that you have debugging on. What is the setting for debug_options ? Marcus On 08/13/2016 04:18 AM, Omid Kosari wrote: Hello, Recently 2 different squid boxes grows from ~40% cpu usage to 100% without

[squid-users] clarifying Features/SslPeekAndSplice on wiki + fake CONNECT

2016-08-21 Thread Marcus Kool
The ssl-bump peek/splice/bump feature is now maturing and many are using it but there are still some issues with the wiki page that I like to clarify. wiki: http://wiki.squid-cache.org/Features/SslPeekAndSplice section "processing steps" Can action "none" be removed from step 1? Step 1. what i

Re: [squid-users] clarifying Features/SslPeekAndSplice on wiki + fake CONNECT

2016-08-22 Thread Marcus Kool
Thanks for your reply. I will start changing the wiki page. When I think I am done, I will let you know for a review. What is left is my desire to get a fake CONNECT with FQDN (see below). Marcus On 08/22/2016 04:20 PM, Alex Rousskov wrote: On 08/21/2016 06:46 AM, Marcus Kool wrote: there

Re: [squid-users] clarifying Features/SslPeekAndSplice on wiki + fake CONNECT

2016-08-23 Thread Marcus Kool
On 08/23/2016 12:44 AM, Alex Rousskov wrote: On 08/22/2016 08:14 PM, Marcus Kool wrote: Thanks for your reply. I will start changing the wiki page. When I think I am done, I will let you know for a review. It is best to commit all your intended changes at once (if at all) rather than to use

Re: [squid-users] clarifying Features/SslPeekAndSplice on wiki + fake CONNECT

2016-08-23 Thread Marcus Kool
On 08/23/2016 11:26 AM, Alex Rousskov wrote: On 08/23/2016 07:59 AM, Marcus Kool wrote: On 08/23/2016 12:44 AM, Alex Rousskov wrote: On 08/22/2016 08:14 PM, Marcus Kool wrote: When I think I am done, I will let you know for a review. It is best to commit all your intended changes at once

Re: [squid-users] clarifying Features/SslPeekAndSplice on wiki + fake CONNECT

2016-08-24 Thread Marcus Kool
On 08/24/2016 02:43 AM, Alex Rousskov wrote: On 08/23/2016 08:34 AM, Marcus Kool wrote: ok, I suggest that you review what is done already. I have made a few corrections and improvements, trying to document every change (and some suggestions for future work) in the commit messages. The

Re: [squid-users] Skype+intercept+ssl_bump

2016-08-27 Thread Marcus Kool
On 07/30/2016 04:21 PM, Alex Rousskov wrote: *snip* Update: The question still stands, but we now know more about what happens if the on_unsupported_protocol bug (in code and/or documentation, depending on how you look at it) discussed above is fixed: Squid then starts tunneling traffic as it

Re: [squid-users] More host header forgery pain with peek/splice

2016-08-30 Thread Marcus Kool
Do I understand it correctly that Squid in normal proxy mode allows malware to do a CONNECT to any destination, while in transparent proxy mode does extra security checks which causes some regular (non-malware) clients to fail? And philosophical questions: is Squid the right tool to stop malware?

Re: [squid-users] Skype+intercept+ssl_bump

2016-09-01 Thread Marcus Kool
On 08/27/2016 02:20 PM, Marcus Kool wrote: On 07/30/2016 04:21 PM, Alex Rousskov wrote: *snip* Update: The question still stands, but we now know more about what happens if the on_unsupported_protocol bug (in code and/or documentation, depending on how you look at it) discussed above is

Re: [squid-users] More host header forgery pain with peek/splice

2016-09-04 Thread Marcus Kool
about the implications. Thanks Marcus On 09/04/2016 01:12 PM, Amos Jeffries wrote: On 31/08/2016 5:25 a.m., Marcus Kool wrote: Do I understand it correctly that Squid in normal proxy mode allows malware to do a CONNECT to any destination, while in transparent proxy mode does extra security ch

Re: [squid-users] subnet forward

2016-09-07 Thread Marcus Kool
On 09/07/2016 10:05 AM, Pol Hallen wrote: Hello all :-) I'm sorry if this couldn't squid problem.. honestly I don't know.. I've a small lan: dsl<-WAN_NIC0_192.168.5.0/30->lan1_192.168.10.0/24 (NIC1)<-->switch+AP lan2_192.168.1.0/24 (NIC2)<--->switch+AP I've squi

Re: [squid-users] subnet forward

2016-09-07 Thread Marcus Kool
to be sure that the link speed and duplex is OK, you need to look at both sides. Marcus On 09/07/2016 01:01 PM, Pol Hallen wrote: Since you have an ancient version of Squid I am assuming that you also have ancient hardware. :-) NIC are not so ancient :-) hw also.. Settings for eth0: Sup

Re: [squid-users] ssl bump certificate question

2016-09-07 Thread Marcus Kool
On 09/07/2016 05:58 PM, Antony Stone wrote: On Wednesday 07 September 2016 at 22:55:06, Yuri Voinov wrote: 08.09.2016 2:25, erdosain9 пишет: Hi. A query. Sslbump is possible without installing the certificate, machine by machine ??? Bump impossible. Splice - possible. Is there any way th

Re: [squid-users] ssl bump certificate question

2016-09-07 Thread Marcus Kool
On 09/07/2016 05:58 PM, Antony Stone wrote: On Wednesday 07 September 2016 at 22:55:06, Yuri Voinov wrote: 08.09.2016 2:25, erdosain9 пишет: Hi. A query. Sslbump is possible without installing the certificate, machine by machine ??? Bump impossible. Splice - possible. Is there any way th

Re: [squid-users] Web Whatsapp, Dropbox... problem

2016-09-12 Thread Marcus Kool
On 09/12/2016 11:14 AM, Yuri Voinov wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Oooops, acl must be: acl excludeSSL ssl::server_name_regex web\.whatsapp\.com why a regex? why not the following ? acl excludeSSL ssl::server_name web.whatsapp.com Marcus ___

Re: [squid-users] Web Whatsapp, Dropbox... problem

2016-09-12 Thread Marcus Kool
--- Hash: SHA256 Because ssl :: server_name_regex works reliably. As shown by my personal practice. But in general it is by op's choice. 12.09.2016 20:38, Marcus Kool пишет: > > > On 09/12/2016 11:14 AM, Yuri Voinov wrote: >>

Re: [squid-users] Best solution for content filtering using squid?

2015-05-07 Thread Marcus Kool
Hi, What is it that makes you want to go to a better solution ? did you look at ufdbGuard? Marcus PS: Beware! I am biased since I wrote ufdbGuard. On 05/07/2015 03:52 PM, Bob Cochran wrote: Hi, What is the best solution with squid for content filtering using lists of domains that should b

Re: [squid-users] Best solution for content filtering using squid?

2015-05-07 Thread Marcus Kool
On 05/07/2015 04:49 PM, Bob Cochran wrote: On 05/07/2015 02:57 PM, Marcus Kool wrote: did you look at ufdbGuard? Thank you. I did look at it briefly and moved on when I saw that a paid license was required if a commercial product made use of it. Perhaps I'm wrong about this? I

Re: [squid-users] squidGuard configuration test - echo test

2015-06-07 Thread Marcus Kool
The URL director interface was changed with Squid 3.4, see also http://wiki.squid-cache.org/Features/Redirectors The latest version of squidguard is 1.5 beta from 2010 and squidGuard does not support the new interface of Squid. ufdbGuard is also a URL redirector and since it has regular updates

Re: [squid-users] squidGuard configuration test - echo test

2015-06-08 Thread Marcus Kool
Helmut, you can download ufdbGuard here: https://www.urlfilterdb.com/downloads/software_doc.html and here: http://sourceforge.net/projects/ufdbguard/ ufdbGuard is just like Squid free Open Source Software. The trial license on www.urlfilterdb.com is about the URL database. Best regards,

Re: [squid-users] Memory usage question

2015-06-19 Thread Marcus Kool
What is the physical memory size ?? You might want to read the faq on memory: http://wiki.squid-cache.org/SquidFaq/SquidMemory Marcus On 06/19/2015 07:19 AM, Alex Samad wrote: Hi I recently push my squid VM memory up to 65G i pushed up squid usage (i thought) to 40G squid.conf cache_mem 4096

[squid-users] ufdbGuard 1.31-13 released

2015-06-26 Thread Marcus Kool
ufdbGuard, the fastest and free URL filter for Squid, has a new patch release. Patch 13 resolves: + new installation procedure for Solaris 10 and 11 - with much appreciated help from Yuri Voinov + various overblocking/underblocking issues with complex ACLs + redirection of URLs with HTTPS on Squ

Re: [squid-users] acl for redirect

2015-06-30 Thread Marcus Kool
I suggest to read this: https://support.google.com/websearch/answer/186669 and look at option 3 of section 'Keep SafeSearch turned on for your network' Marcus On 06/30/2015 05:48 PM, Mike wrote: Scratch that (my previous email to this list), google disabled their insecure sites when used as

Re: [squid-users] acl for redirect

2015-07-01 Thread Marcus Kool
office, so changing from a proxy to a DNS server is not an option, since we would also be required to change all several thousand of our customers DNS settings. On 6/30/2015 17:30 PM, Marcus Kool wrote: I suggest to read this: https://support.google.com/websearch/answer/186669 and look at option

Re: [squid-users] Squid and ufdbGuard, display blocked URL on client browser address bar

2015-07-10 Thread Marcus Kool
On 07/10/2015 12:54 AM, Amos Jeffries wrote: On 10/07/2015 9:51 a.m., David Touzeau wrote: Hi ikna This can be done, but you need to forget the ufdbgclient and create yourself a new one that is able to connect to the ufdbguard server in order to get ufdbguard results. In this case, you have w

Re: [squid-users] AUFS vs. DISKS

2015-07-15 Thread Marcus Kool
On 07/15/2015 11:39 AM, Amos Jeffries wrote: On 16/07/2015 1:51 a.m., Stakres wrote: Hi Fred, tests from my side: DISKD with TCP_HIT objects: 564KB/s with wget, the same url you have tested. AUFS with TCP_HITS objects: 47.8M/s, same wget, same squid, same url, same all. Wget with AUFS: Length

Re: [squid-users] ufdbGuard cannot check ssl certs

2015-07-15 Thread Marcus Kool
Hi Stan, ufdbGuard probably logs more error messages before "Cannot perform mandatory check of SSL certificates" What are they ? ufdbGuard then calls abort() which causes a core dump since it found something terribly wrong. Please reply to me or the ufdbGuard list at http://sourceforge.net/p

Re: [squid-users] AUFS vs. DISKS

2015-07-15 Thread Marcus Kool
On 07/15/2015 11:59 AM, Yuri Voinov wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Amos, I think, auds queue must be buffered more better and smoother. On some OS (I've tested) peak loads leads performance degradation. Periodically. That is why I'm not using aufs. This makes sense

Re: [squid-users] Squid 3.5.5 ssl_bump and ufdbGuard

2015-07-20 Thread Marcus Kool
First an introduction in blocking HTTPS: HTTPS is a protocol that is designed to be non-interceptable, and if it is intercepted, the browser will notify the user about this interception. This is very different from HTTP which can easily be intercepted and the interceptor can redirect a browser u

Re: [squid-users] Squid3: 100 % CPU load during object caching

2015-07-23 Thread Marcus Kool
I am not sure if it is relevant, maybe it is: I am developing an ICAP daemon and after the ICAP server sends a "100 continue" Squid sends the object to the ICAP server in small chunks of varying sizes: 4095, 5813, 1448, 4344, 1448, 1448, 2896, etc. Note that the interval of receiving the chunks i

Re: [squid-users] Squid3: 100 % CPU load during object caching

2015-07-23 Thread Marcus Kool
*Gesendet:* Donnerstag, 23. Juli 2015 um 13:29 Uhr *Von:* "Marcus Kool" *An:* "Jens Offenbach" , "Eliezer Croitoru" , "Amos Jeffries" , squid-users@lists.squid-cache.org *Betreff:* Re: [squid-users] Squid3: 100 % CPU load during object caching I am not sure i

Re: [squid-users] Squid3: 100 % CPU load during object caching

2015-07-24 Thread Marcus Kool
On 07/24/2015 03:25 AM, Jens Offenbach wrote: I have made a quick test of Squid 3.3.8 on Ubuntu 15.04 and I get the same problem: 100 % CPU usage, 500 KB/sec download rate. Gesendet: Freitag, 24. Juli 2015 um 07:54 Uhr Von: "Jens Offenbach" An: "Marcus Kool" , "E

Re: [squid-users] Squid3: 100 % CPU load during object caching

2015-07-24 Thread Marcus Kool
: Freitag, 24. Juli 2015 um 14:33 Uhr Von: "Marcus Kool" An: "Jens Offenbach" , squid-users@lists.squid-cache.org Betreff: Re: [squid-users] Squid3: 100 % CPU load during object caching On 07/24/2015 03:25 AM, Jens Offenbach wrote: I have made a quick test of Squid 3.3.8 on Ub

Re: [squid-users] Squid3: 100 % CPU load during object caching

2015-07-25 Thread Marcus Kool
memory is plentyful, just make sure that the OS has a large file system cache. So reduce mem_cahce of Squid a little and tune the OS with vm.swappiness=10 in /etc/sysctl.conf Best regards Marcus Have a nice weekend! Regards, Jens Gesendet: Freitag, 24. Juli 2015 um 19:01 Uhr Von: "Marcus

Re: [squid-users] squid centos and osq_lock

2015-07-31 Thread Marcus Kool
osq_lock is used in the kenel for the implementation of a mutex. It is not clear which mutex so we can only guess. Which version of the kernel and distro do you use? Since mutexes are used by Squid SMP, I suggest to switch for now to Squid non-SMP. What is the value of cpu_affinity_map in all

Re: [squid-users] squid centos and osq_lock

2015-08-01 Thread Marcus Kool
g the sweet spot between those trends is something else to tune for. <http://wiki.squid-cache.org/MultipleInstances#Tips> 2015-07-31 14:53 GMT+02:00 Marcus Kool: osq_lock is used in the kenel for the implementation of a mutex. It is not clear which mutex so we can only guess. Which versi

Re: [squid-users] ssl_bump updates coming in 3.5.8

2015-08-21 Thread Marcus Kool
I do not want to spoil things, but did you already read my latest addition to bug 4303 ? Marcus On 08/21/2015 04:28 AM, Amos Jeffries wrote: Hi all, Christos has managed (we think) to resolve a fairly major design issue that has been plaguing the 3.5 series peek-and-splice feature so far.

Re: [squid-users] Squid 3.5.7, cache_swap_high, bug or not bug ?

2015-08-29 Thread Marcus Kool
On 08/28/2015 08:53 PM, FredT wrote: Hi Amos, We have applied the patch with the client on the squid in prod a coule of hours ago... We can see now a real aggressive objects cleaning I can confirm a 200 obj/sec is a minimal number with huge traffic, you could fix the value a bit higher (250-30

Re: [squid-users] refresh_pattern and same objects

2015-09-01 Thread Marcus Kool
On 09/01/2015 05:14 AM, FredB wrote: More precisely I reduced the ttl of the first line refresh_pattern -i \.(htm|html|xml|css)(\?.*)?$ 10080 100% 10080 #All File 30 days max refresh_pattern -i \.(3gp|7z|ace|asx|bin|deb|divx|dvr-ms|ram|rpm|exe|inc|cab|qt)(\?.*)?$ 43200 100% 43200 ignore-no-

Re: [squid-users] refresh_pattern and same objects

2015-09-01 Thread Marcus Kool
And, of course, universal rule for store_id_access. I think that this works well for trackers gifs but not for other gifs with parameters. Store ID is powerful instrument for deduplication cache story. Which permits not to use terabytes disks. 02.09.15 0:00, Marcus Kool пишет: On 09/01

Re: [squid-users] refresh_pattern and same objects

2015-09-01 Thread Marcus Kool
victims, like the few gifs that actually have a different image depending on the parameter. 02.09.15 0:16, Marcus Kool пишет: > > On 09/01/2015 03:08 PM, Yuri Voinov wrote: >> > Better to write store-id rule which cut off parameters and store gif. > > Something li

Re: [squid-users] HTTPS URL Rewrite

2015-09-01 Thread Marcus Kool
When a browser requests https://www.example.com/index.html, Squid with ssl-bump sends two requests to the URL rewriter: 1. CONNECT www.example.com:443 2. GET https://www.example.com/index.html The URL rewriter must _not_ block the first and send an alternative URL for the second. Caveat: thi

Re: [squid-users] stoping after rotate

2015-09-05 Thread Marcus Kool
On Linux, an important sysctl parameter that determines how Linux behaves with respect to VM allocation is vm.overcommit_memory (should be 0). And vm.swappiness is important to tune servers (should be 10-15). Which version of Linux do you have and what is the output of sysctl -a | grep -e vm.

Re: [squid-users] stoping after rotate

2015-09-08 Thread Marcus Kool
mentioned that the swap is 32 GB. What is the size of the physical memory ? Did you already increase the swap ? Marcus 2015-09-05 15:08 GMT-03:00 Marcus Kool mailto:marcus.k...@urlfilterdb.com>>: On Linux, an important sysctl parameter that determines how Linux behaves with respect

Re: [squid-users] stoping after rotate

2015-09-08 Thread Marcus Kool
MT-03:00 Marcus Kool mailto:marcus.k...@urlfilterdb.com>>: On 09/08/2015 08:11 AM, Jorgeley Junior wrote: Thank you all, this is the output: vm.overcommit_memory = 0 vm.swappiness = 60 I have a Redhat 6.6 The value of vm.overcommit_memory is OK.

Re: [squid-users] stoping after rotate

2015-09-08 Thread Marcus Kool
20:25 GMT-03:00 Marcus Kool mailto:marcus.k...@urlfilterdb.com>>: On 09/08/2015 10:39 AM, Jorgeley Junior wrote: I have 8GB physical memory and my swap is 32GB. I didn't increase the swap yet, should I? You must start with reading the memory FAQ: http:/

Re: [squid-users] stoping after rotate

2015-09-09 Thread Marcus Kool
ok, I'll do it 2015-09-08 21:30 GMT-03:00 Marcus Kool mailto:marcus.k...@urlfilterdb.com>>: On 09/08/2015 09:23 PM, Jorgeley Junior wrote: ok, read that already, i set cache_mem to 5GB, so is not ok? No. Squid will use more

Re: [squid-users] after changed from 3.4.13 to 3.5.8 sslbump doesn't work for the site https://banking.postbank.de/

2015-09-17 Thread Marcus Kool
I just tried accessing https://banking.postbank.de/ using Squid 3.5.8 and Chrome. I also got the ERR_CONNECTION_CLOSED error. What is weird is that Squid sends a "CONNECT banking.postbank.de" 21 times to the URL rewriter. Then I changed the Squid configuration and added ".postbank.de" in our li

Re: [squid-users] after changed from 3.4.13 to 3.5.8 sslbump doesn't work for the site https://banking.postbank.de/

2015-09-26 Thread Marcus Kool
On 09/26/2015 03:03 PM, Dieter Bloms wrote: Hallo Marcus, On Thu, Sep 17, Marcus Kool wrote: I just tried accessing https://banking.postbank.de/ using Squid 3.5.8 and Chrome. I also got the ERR_CONNECTION_CLOSED error. thank you for testing, so I think the fault is not my config. May it

Re: [squid-users] On what methods does url filtering needs to apply?

2015-09-28 Thread Marcus Kool
"content filtering" may filter only content while a generic filter may filter anything including malware that uses PUT, OPTION and/or HEAD to upload credit card data. So it depends on what you want to filter. If it is downloadable content only, you can stick with filtering GET POST CONNECT. Ma

Re: [squid-users] ICAP and HTTPS

2015-10-06 Thread Marcus Kool
On 10/06/2015 06:05 PM, Rafael Akchurin wrote: Hello Paul, Eliezer, Alex, We (diladele ICAP) have an open bug /feature requests for this: https://github.com/ra-at-diladele-com/qlproxy_external/issues/731 https://github.com/ra-at-diladele-com/qlproxy_external/issues/726 As Alex

Re: [squid-users] Ssl-Bump and revoked server certificates

2015-10-06 Thread Marcus Kool
On 10/06/2015 07:18 PM, Jason Haar wrote: On 06/10/15 23:21, Walter H. wrote: Hello, can you please provide an example of how to use this in squid.conf #create external acl checker that returns "ERR" or "OK" based on cert data sent to it external_acl_type checkIfHTTPS children-max=20 concur

Re: [squid-users] Squid 3.5.10 Performance

2015-10-07 Thread Marcus Kool
On 10/07/2015 09:00 AM, FredB wrote: Just FI With high load system (and exactly the same configuration of course) the load average is significantly reduced by the use of the latest release in comparison with the previous 3.5.x versions diskd, digest auth, basic auth, delay pools, some acls,

Re: [squid-users] ssl bump and url_rewrite_program (like squidguard)

2015-11-03 Thread Marcus Kool
I suspect that the problem is that you redirect a HTTPS-based URL to an HTTP URL and Squid does not like that. Marcus On 11/03/2015 08:48 PM, Edouard Gaulué wrote: Hi community, I've followed http://wiki.squid-cache.org/ConfigExamples/Intercept/SslBumpExplicit to set my server. It looks re

Re: [squid-users] ssl bump and url_rewrite_program (like squidguard)

2015-11-04 Thread Marcus Kool
/http' Marcus On 11/04/2015 10:55 AM, Edouard Gaulué wrote: Le 04/11/2015 11:00, Amos Jeffries a écrit : On 4/11/2015 12:48 p.m., Marcus Kool wrote: I suspect that the problem is that you redirect a HTTPS-based URL to an HTTP URL and Squid does not like that. Marcus To give it a try in that d

Re: [squid-users] ssl bump and url_rewrite_program (like squidguard)

2015-11-05 Thread Marcus Kool
ps://ad.doubleclick.net"; message. But, I don't get the squid message anymore regarding http/https. It may be that rewrite_rule_program come after peek and splice stuff leading squid to an unpredictable situation. Is there a way to play on order things happen in squid? Regards, EG

Re: [squid-users] ssl bump and url_rewrite_program (like squidguard)

2015-11-12 Thread Marcus Kool
direction so any redirection by Squid or an other proxy is an attempt to break the SSL protocol. Redirection with HTTP is simple because the HTTP protocol has a built-in mechanism for redirection that proxies can use. Marcus I can also provide squid logs, but tell me what because I've got a lo

Re: [squid-users] ssl bump and url_rewrite_program (like squidguard)

2015-11-12 Thread Marcus Kool
I cannot make much of the logs and expect that information is missing. But using just logic, it seems that Squid has a problem with the redirect to a CONNECT. I suggest to set debug all,9 and to look closely at what happens with the redirection. Marcus On 11/12/2015 10:02 AM, Edouard Gaulué w

Re: [squid-users] Block google pictures

2015-11-27 Thread Marcus Kool
You can force Google safesearch, even with HTTPS. Google only needs that you put a CNAME entry in your DNS server for www.google.com. See https://support.google.com/websearch/answer/186669?hl=en Option 3 for more information. Marcus On 11/26/2015 12:27 PM, Funke, Martin wrote: Im using squid

Re: [squid-users] Squid memory leak on ubuntu 14.04

2015-11-27 Thread Marcus Kool
I do not have the detail of Ubuntu 14.04 but most likely 12.04 and 14.04 have a different version of malloc (see "man malloc") which allocates gigabytes of virtual memory. Most likely you see in top that the resident memory is what you expect that Squid uses (comparable as on 12.04) and the vir

Re: [squid-users] Peek and splice without replacing the certificates

2015-12-14 Thread Marcus Kool
On 12/14/2015 06:43 AM, Парфенович Н.А. wrote: Hello! Show you how to use Squid in transparent mode for tracking HTTPS without replacing the certificates? My squid.conf: http://pastebin.ru/AWU8LXvK. If such a configuration file to use version 3.5.8 squid compiled using Libressl, everything wor

Re: [squid-users] Using subordinate CA for SSL Bump

2015-12-14 Thread Marcus Kool
On 12/14/2015 09:16 PM, Amos Jeffries wrote: With all that looking hopeful, and the certs identified as the secondary chain being attached (everything except the firstprimary/signing cert). I'm not actually finding anywhere sending the actual signing certificate itself during the bumping steps

Re: [squid-users] Host header forgery affects pure splice environment too?

2015-12-28 Thread Marcus Kool
On 12/28/2015 01:33 AM, Jason Haar wrote: On 28/12/15 14:34, Amos Jeffries wrote: [...] I think we know what the problem is: TOR is making TLS connections (I don't know if they're HTTPS) on port 443 and uses SNI names that aren't real? peeking on tor-proxy-2.cypherpunks.to shows a certifica

Re: [squid-users] problem with squidGuard redirect page after upgrading squid

2015-12-28 Thread Marcus Kool
On 12/28/2015 08:46 PM, George Hollingshead wrote: I've had squid3.0 running with squidGuard on my old ubuntu 10.04 system with no problems for a few months now. I just recently was enlightened by Yuri how to compile using a local copy of openssl so i could upgrade to latest squid. This was

Re: [squid-users] problem with squidGuard redirect page after upgrading squid

2016-01-07 Thread Marcus Kool
On 01/07/2016 12:31 AM, Jason Haar wrote: On 06/01/16 00:04, Amos Jeffries wrote: Yes. Squid always has been able to given enough RAM. Squid stores most ACLs in memory as Splay trees, so entries are sorted by frequency of use which is dynamically adapted over time. Regex are pre-parsed and agg

Re: [squid-users] problem with squidGuard redirect page after upgrading squid

2016-01-08 Thread Marcus Kool
On 01/07/2016 06:48 PM, Jason Haar wrote: On 08/01/16 01:56, Marcus Kool wrote: Can you explain what the huge number of regexes is used for ? malware urls. I'm scraping them from publicly available sources like phishtank, malwaredomains.com. Ironically, they don't need to be reg

Re: [squid-users] URL Rewrite for https via Squidguard

2016-01-09 Thread Marcus Kool
On 01/09/2016 05:07 AM, Darren wrote: Hi I am trying to hack squidguard to allow me to redirect users attempts to connect to blocked https enabled sites. Some sites are allowed and the bulk are not. Currently I can see the Connect details being handed to SG for processing and if I change th

Re: [squid-users] URL Rewrite for https via Squidguard

2016-01-09 Thread Marcus Kool
hope and a possible way forward. regards Darren B. Sent from Mailbird <http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm_campaign=sent-from-mailbird> On 9/01/2016 5:46:36 PM, Marcus Kool wrote: On 01/09/2016 05:07 AM, Darren wrote: > Hi > &

Re: [squid-users] Rock datastore, CFLAGS and a crash that (may be) known

2016-02-16 Thread Marcus Kool
On 02/16/2016 12:32 PM, Jester Purtteman wrote: ./configure CFLAGS="-march=core2 -mcx16 -msahf -mno-movbe -mno-aes -mno-pclmul -mno-popcnt -mno-sse4 -msse4.1" CXXFLAGS="${CFLAGS}" --with-pthreads --prefix=/usr --localstatedir=/var --libexecdir=/usr/lib/squid--srcdir=. --datadir=/usr/s

Re: [squid-users] High cpu usage by re_search_internal

2014-10-04 Thread Marcus Kool
I suspect that the language setting is causing it. If $LANG is different from "C" it may have a huge impact on the performance of regular expression evaluation (not only in Squid but also awk, sed etc.) Try this: LANG=C /etc/init.d/squid start and see if Squid improves. Marcus > Hi, > > I have

Re: [squid-users] High cpu usage by re_search_internal

2014-10-06 Thread Marcus Kool
> I don't know i am correct or not but in /etc/init/squid3.conf i see > following > lines > env CONFIG="/etc/squid3/squid.conf" > env SQUID_ARGS="-YC" > > so i have added following line > env LANG=C > > correct ? does not need double qotation mark ? That should do it. Marcus > > > -- > View this

Re: [squid-users] Best way to deny access to URLs in Squid 3.3.x?

2014-10-14 Thread Marcus Kool
With every set of requirements, there is an other "best way". To selectively block websites and also block SSH tunnels, VPNs, proxies and remote software (some of which are detected on the fly) you can also use ufdbGuard. Your mileage varies with which URL database you use. Marcus On 10/14/2

Re: [squid-users] SSL bump fails accessing .gov.uk servers

2014-10-31 Thread Marcus Kool
With OpenSSL 1.0.1e-fips : openssl s_client -connect www.taxdisc.service.gov.uk:443 fails (tries TLS1.2) openssl s_client -connect www.taxdisc.service.gov.uk:443 -ssl3 works The webmail server of my ISP works like this: it uses only TLS1.0, so no TLS1.1 or TLS1.2, but when with op

Re: [squid-users] SSL bump fails accessing .gov.uk servers

2014-10-31 Thread Marcus Kool
On 10/31/2014 10:12 PM, Amos Jeffries wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 1/11/2014 12:09 p.m., Marcus Kool wrote: With OpenSSL 1.0.1e-fips : openssl s_client -connect www.taxdisc.service.gov.uk:443 fails (tries TLS1.2) openssl s_client -connect

Re: [squid-users] Squid 3.5.0.2 beta is available

2014-11-04 Thread Marcus Kool
The release notes have a link for ecap documentation that points to http://wiki.squid-cache.org/Features/BLAH which does not exist. The squid wiki refers to an old version of ecap so I tried searching for "ecap" on the home page but it throws an error: Not Found The requested URL /cgi-bin/swish-

Re: [squid-users] High CPU-Usage with squid 3.4.9 (and/or 3.4.4)

2014-11-10 Thread Marcus Kool
during our last tests (with 3.4.x) we also tried the worker option. it does not matter if workers are enabled or not. with more workers the cpu rise seems to be somewhat slower. so it is not connected to (smp)workers. it is the external auth helper - although the squid process and not the helper

Re: [squid-users] Centralized Squid - design and implementation

2014-11-16 Thread Marcus Kool
Let me start to say that I am biased since I am the author of ufdbGuard. If you have worked with squidGuard than you will find that ufdbGuard is an excellent replacement since ufdbGuard was forked in 2005 from squidGuard and has since gained many features. And I suggest to apply for a trial lice

Re: [squid-users] WARNING: there are more than 100 regular expressions

2014-11-27 Thread Marcus Kool
how many REs do you have ? and do you intend to use REs for blacklisting? Marcus On 11/27/2014 08:33 AM, Helmut Hullen wrote: Hallo, navari.lore...@gmail.com, Du meintest am 27.11.14: "Consider using less REs ..." is not possible. Then try something like "squidguard" with lots of user defi

Re: [squid-users] WARNING: there are more than 100 regular expressions

2014-11-27 Thread Marcus Kool
blocking facebook and twitter can be done with ACLs based on dstdomain. they are much faster than REs. Marcus On 11/27/2014 10:01 AM, navari.lore...@gmail.com wrote: ok i don't intend to use REs for blacklisting but only for blocking some sites like facebook twitter... In the other file i have

Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect

2015-01-05 Thread Marcus Kool
Much of the discussion so far has been about bumping traffic on port 443, bumping SSL-encapsulated HTTP traffic and not bumping (allowing) other traffic. Since port 443 is used for many protocols, it is in many cases dangerous to allow non-bumpable traffic: SSH tunnels using port 443 are common,

Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect

2015-01-05 Thread Marcus Kool
+ HTTP filter + Cisco/DPI + tcputils + sniffer + manual maintenance of ACLs/exclude list 05.01.2015 17:51, Marcus Kool пишет: Much of the discussion so far has been about bumping traffic on port 443, > bumping SSL-encapsulated HTTP traffic and not bumping (allowing) > other traffic.

Re: [squid-users] Squid 3 SSL bump: Google drive application could not connect

2015-01-05 Thread Marcus Kool
ice with the Squid development team but there is currently no sponsor to implement a new protocol to filter non-HTTP data in Squid. Marcus On Mon, Jan 5, 2015 at 9:10 AM, Marcus Kool mailto:marcus.k...@urlfilterdb.com>> wrote: On 01/05/2015 11:11 AM, Yuri Voinov wrote:

[squid-users] tcp_outgoing_address and ICAP server

2015-01-21 Thread Marcus Kool
I am using Squid 3.4.9 and have an issue with tcp_outgoing_address. The Squid server is connceted to the internet with multiple NICs and uses tcp_outgoing_address a.public.IP.address and also want to use an ICAP server on the same host using icap_service reqmod_urlfilterdb reqmod_precache

Re: [squid-users] tcp_outgoing_address and ICAP server

2015-01-24 Thread Marcus Kool
On 01/24/2015 10:15 AM, Amos Jeffries wrote: On 22/01/2015 10:11 a.m., Marcus Kool wrote: I am using Squid 3.4.9 and have an issue with tcp_outgoing_address. The Squid server is connceted to the internet with multiple NICs and uses tcp_outgoing_address a.public.IP.address and also want

Re: [squid-users] tcp_outgoing_address and ICAP server

2015-01-25 Thread Marcus Kool
On 01/24/2015 11:24 PM, Amos Jeffries wrote: On 25/01/2015 9:39 a.m., Marcus Kool wrote: On 01/24/2015 10:15 AM, Amos Jeffries wrote: On 22/01/2015 10:11 a.m., Marcus Kool wrote: I am using Squid 3.4.9 and have an issue with tcp_outgoing_address. The Squid server is connceted to the

Re: [squid-users] tcp_outgoing_address and ICAP server

2015-01-25 Thread Marcus Kool
On 01/25/2015 01:12 PM, Amos Jeffries wrote: On 25/01/2015 11:43 p.m., Marcus Kool wrote: On 01/24/2015 11:24 PM, Amos Jeffries wrote: On 25/01/2015 9:39 a.m., Marcus Kool wrote: On 01/24/2015 10:15 AM, Amos Jeffries wrote: On 22/01/2015 10:11 a.m., Marcus Kool wrote: I am using Squid

Re: [squid-users] tcp_outgoing_address and ICAP server

2015-01-25 Thread Marcus Kool
On 01/25/2015 02:33 PM, Amos Jeffries wrote: On 26/01/2015 4:59 a.m., Marcus Kool wrote: The debug trace starts with: Xaction.cc(133) openConnection: *Adaptation::Icap::OptXact* opens connection to 10.10.0.6:1344 and then comm.cc(549) comm_openex: comm_openex: Attempt open socket for

Re: [squid-users] Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2015-02-03 Thread Marcus Kool
Hi Omid, The I/O requirements can be estimated well if you tell more about the environment. If you know the number of requests/second that Squid prcoesses you can add a percentage to increase performance and calculate the desired I/Os per second (IOPS). When you have the desired IOPS, you can ca

Re: [squid-users] Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2015-02-03 Thread Marcus Kool
On 02/03/2015 12:56 PM, Omid Kosari wrote: Squidbox1: Average HTTP requests per minute since start:16000 Squidbox2: Average HTTP requests per minute since start:11000 16000 request/min = 266 requests/sec. With a well-tuned Squid system I estimate that the disk I/O is less than

Re: [squid-users] Hypothetically comparing SATA\SAS to NAS\SAN for squid.

2015-02-04 Thread Marcus Kool
On 02/04/2015 04:24 AM, Omid Kosari wrote: The only reason for extend is more capacity . Currently there is no problem with current setup except capacity . I can replace each SSD with new 500GB which doubles the capacity and it is not enough . and old SSDs will be unusable . So i prefer a long

Re: [squid-users] Is Squid can shutdown unused idle redirector's children?

2015-02-12 Thread Marcus Kool
Yuri, I suggest to consider using ufdbGuard instead of squidGuard. Besides being faster is has a different structure: the redirector that squid starts is a small lightweight process that forwards requests to ufdbguardd, a multithreaded daemon which has the URL database in memory. The database is

Re: [squid-users] Is Squid can shutdown unused idle redirector's children?

2015-02-12 Thread Marcus Kool
d cnanging free redirector to commercial one is not an option. ufdbGuard is not a commercial redirector, but is free and works with any free database or your own database/blacklist. It has an additional option to use a commercial database. 13.02.15 2:06, Marcus Kool пишет: Yuri, I suggest to

Re: [squid-users] Is Squid can shutdown unused idle redirector's children?

2015-02-16 Thread Marcus Kool
On 02/16/2015 08:00 PM, Eliezer Croitoru wrote: Hey Yuri, OK I have seen something... Now we might need also the virtual memory which might be vsz. And the cachemgr output is not from squidview.. The last image I have seen from cachemgr was much helpful(with 10 helpers). From what I have see

Re: [squid-users] Is Squid can shutdown unused idle redirector's children?

2015-02-17 Thread Marcus Kool
On 02/17/2015 08:21 AM, Yuri Voinov wrote: squidGuard does not support the Squid feature 'concurrency' for url_rewrite_children. ufdbGuard does. With concurrency, latency goes down and the number of processes can also be reduced. The lack of concurrency is main disadvantage of squidGuard. O

Re: [squid-users] Is Squid can shutdown unused idle redirector's children?

2015-02-17 Thread Marcus Kool
On 02/16/2015 11:43 PM, Amos Jeffries wrote: PS. Marcus, perhapse you should go on search around to find distro maintainers who are publishing SG and convince them to replace the defaults with ufdbguard. I have to do that periodically to clear up old Squid versions being forced on users. It hel

Re: [squid-users] Is Squid can shutdown unused idle redirector's children?

2015-02-17 Thread Marcus Kool
On 02/17/2015 11:30 AM, Yuri Voinov wrote: Also, gents. ufdbGuard is cool, but: - Where is good documentation? I found only one connon PDF. No performance recommendations, no administrator's guide - this good piece of software not so trivial as squidGuard, i.e., I don't know, how to support

Re: [squid-users] Refresh ACL list only

2015-03-17 Thread Marcus Kool
On 03/17/2015 02:59 PM, Samuel Anderson wrote: Unfortunately thats not really an option for me. I've already built everything just using squid. It works great and does everything I need it to do with the exception of refreshing the ACL lists. I just need to find a way to refresh those single

Re: [squid-users] Refresh ACL list only

2015-03-17 Thread Marcus Kool
On 03/17/2015 04:32 PM, Brendan Kearney wrote: On Tue, 2015-03-17 at 16:13 -0300, Marcus Kool wrote: it has a configuration option to respond with 'allow all' during a reconfiguration. a Fail-Open policy can be a security gap, and should be considered carefully before implemen

Re: [squid-users] Integrate Squid Kerberos auth and Squidguard ldapusersearch into AD

2015-04-15 Thread Marcus Kool
On 04/15/2015 11:38 AM, tchristin wrote: Hi all, I'm having trouble with Squid Kerberos auth and the Squidguard ldapusersearch that I use to apply ACLs by Active Directory groups membership. The problem is : - Squid and Squidguard see my user as : 'user@domain.local' so the '%s' variable of s

Re: [squid-users] HTTPS Filtering and SSL-Bump

2015-04-23 Thread Marcus Kool
On 04/23/2015 05:52 PM, Jonathan Chretien wrote: Hi all. I'm trying to implement the filtering of https content for a particular url. The only thing that I'm trying to do it's to unlock corporate video on the Youtube website. I do not want to unlock everything on Youtube but only our corpor

<    1   2   3   >