Hey Acid, First you should try 4.x latest. a reload of 50 domains every 10 second doesn't make sense. I don't understand the config and the setup. for 50 sites you just need a basic script and even one in bash will work dor you with grep. I wrote an example in ruby a while ago, I will try to find it in the next week. Maybe the server is overloaded.
I will try to give you an example later on. Eliezer בתאריך יום ג׳, 30 במרץ 2021, 20:41, מאת acidflash acidflash < acidfla...@linuxmail.org>: > > Hi Eliezer, > > I hope your doing ok. Thanks for the reply. Yeah currently what I am doing > is: > > include /etc/squid/blockedsites.list > and adding the ACL's and the denies in the list file. What version do you > recommend I upgrade to, and is this a known issue? The list is actually > pretty small, probably no more than 50 sites or so, and thats split across > 4 or 5 groups (different ACL's). I'll look into ufdbguard and the other > projects as well, does this sound familiar though? If you think that the > best path forward is to alleviate the burden off of squid to some external > tool, I could probably think up a few hacks to for that, but would > obviously prefer to keep it all within squid. Is this occurance common with > squid -k reconfigure and dstdomain matching? Thanks for your time. Stay > safe. > > *Sent:* Sunday, March 28, 2021 at 4:42 AM > *From:* "Eliezer Croitoru" <ngtech1...@gmail.com> > *To:* squid-users@lists.squid-cache.org > *Subject:* Re: [squid-users] [SPAM] Squid stops serving requests after > squid -k reconfigure > > Hey Acid, > > > > Haven’t seen you here for a very long time. > > The first thing is to upgrade squid if possible… > > > > It’s better that you won’t use squid -kreconf for big blacklists. > > Instead you should use some external software to match the blacklists. > > The most recommended software these days is ufdbguard. > > Depends on the size of your blacklist your might need to find the right > solution. > > The best solution would be to store the list in ram somehow. > > Have you tried some kind of rbl server? > > > > At the time I wrote some code to and some of it was merged into: > > https://github.com/looterz/grimd > > > > It has a reload url so you can update the files on disk and send a reload. > > > > Another service I am using is: > > https://github.com/andybalholm/redwood > > > > Which has a “Classification Service” function. > > It’s pretty easy to write a json http client that can run queries against > this classification service. > > > > Also you’d better use a file In the dstdomain ac ie: > > acl Blacklist dstdomain “/var/blacklists/xyx.list” > > http_access deny Blacklist > > > > and inside the xyx.list file just add lines of domains like > > .blacklisted-domain.com > > .example.com > > > > Etc.. > > > > > > All The Bests, > > Eliezer > > > > ---- > > Eliezer Croitoru > > Tech Support > > Mobile: +972-5-28704261 > > Email: ngtech1...@gmail.com > > Zoom: Coming soon > > > > > > *From:* squid-users <squid-users-boun...@lists.squid-cache.org> *On > Behalf Of *acidflash acidflash > *Sent:* Saturday, March 27, 2021 10:55 AM > *To:* squid-users@lists.squid-cache.org > *Subject:* [SPAM] [squid-users] Squid stops serving requests after squid > -k reconfigure > > > > I have gone through the forums, and I haven't found an answer to the > question, although it has been asked more than once. > > I am running squid 3.5.X on Centos 7, the compile options are: > "configure options: '--build=x86_64-redhat-linux-gnu' > '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' > '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' > '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' > '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' > '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' > '--infodir=/usr/share/info' '--disable-strict-error-checking' > '--exec_prefix=/usr' '--libexecdir=/usr/lib64/squid' '--localstatedir=/var' > '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' > '--with-logdir=$(localstatedir)/log/squid' > '--with-pidfile=$(localstatedir)/run/squid.pid' > '--disable-dependency-tracking' '--enable-eui' > '--enable-follow-x-forwarded-for' '--enable-auth' > '--enable-auth-basic=DB,LDAP,MSNT-multi-domain,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB,SMB_LM,getpwnam' > '--enable-auth-ntlm=smb_lm,fake' > '--enable-auth-digest=file,LDAP,eDirectory' > '--enable-auth-negotiate=kerberos' > '--enable-external-acl-helpers=file_userip,LDAP_group,time_quota,session,unix_group,wbinfo_group,kerberos_ldap_group' > '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' > '--enable-delay-pools' '--enable-epoll' '--enable-ident-lookups' > '--enable-linux-netfilter' '--enable-removal-policies=heap,lru' > '--enable-snmp' '--enable-ssl-crtd' '--enable-storeio=aufs,diskd,rock,ufs' > '--enable-wccpv2' '--enable-esi' '--enable-ecap' '--with-aio' > '--with-default-user=squid' '--with-dl' '--with-openssl' '--with-pthreads' > '--disable-arch-native' 'build_alias=x86_64-redhat-linux-gnu' > 'host_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -fpie' 'LDFLAGS=-Wl,-z,relro -pie -Wl,-z,relro -Wl,-z,now' 'CXXFLAGS=-O2 > -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions > -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches > -m64 -mtune=generic -fpie' > 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'" > > > > I have a service which adds domains to a blacklist file, and then calls > squid -k reconfigure. > Instead of writing to the file, this service updates the file completely > with new rules, > by deleting the old file, and creating a new one in its place, and then > calling squid -k reconfigure. > After doing this, on odd occasions, squid will stop serving traffic > completely, > until you do a squid stop, and squid start. After shutting down squid, > and starting squid up with the same rules, squid will continue to work > normally. > Its probably worth mentioning that during the time that these events are > taking place, > the server is under quite a bit of load, and clients don't stop sending > requests via the server. > What these directives look like: > > acl Porn dstdomain .xnxx.com .sex.com > acl Drugs dstdomain .drugs.com .silkroad.eu > http_access deny Porn > http_access deny Drugs > > > > This also seems to be amplified when there are several squid workers > (child processes) running. > In regards to order, these ACL's are above any other ACL's in the list. We > have a very basic squid conf file that looks like this: > ## START OF FILE > http_port 3128 > cache deny all > # > access_log /var/log/squid/access.log > cache_store_log none > cache_log /dev/null > logfile_rotate 4 > # > auth_param basic program /usr/lib64/squid/basic_db_auth --dsn > "DBI:mysql:host=XX.XX.XX.XX;port=XXXX;database=XXXXX" --user XXXXXX > --password XXXXXXXX --plaintext --persist > # > acl db-auth proxy_auth REQUIRED > auth_param basic credentialsttl 2 hours > auth_param basic casesensitive on > # > connect_timeout 55 minutes > # > request_header_access Allow allow all > request_header_access Authorization allow all > request_header_access WWW-Authenticate allow all > request_header_access Proxy-Authorization allow all > request_header_access Proxy-Authenticate allow all > request_header_access Cache-Control allow all > request_header_access Content-Encoding allow all > request_header_access Content-Length allow all > request_header_access Content-Type allow all > request_header_access Date allow all > request_header_access Expires allow all > request_header_access Host allow all > request_header_access If-Modified-Since allow all > request_header_access Last-Modified allow all > request_header_access Location allow all > request_header_access Pragma allow all > request_header_access Accept allow all > request_header_access Accept-Charset allow all > request_header_access Accept-Encoding allow all > request_header_access Accept-Language allow all > request_header_access Content-Language allow all > request_header_access Mime-Version allow all > request_header_access Retry-After allow all > request_header_access Title allow all > request_header_access Connection allow all > request_header_access Proxy-Connection allow all > request_header_access User-Agent allow all > request_header_access Cookie allow all > request_header_access All deny all > dns_v4_first on > via off > forwarded_for off > follow_x_forwarded_for deny all > dns_nameservers 8.8.8.8 8.8.4.4 > ## END OF FILE > > > > Your help is greatly appreciated, maybe there has been some insight into > this issue after 10+ years since the last time it was asked. > > > _______________________________________________ squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users > _______________________________________________ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users >
_______________________________________________ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
