17.11.2017 21:27, Vieri пишет: > ________________________________ > From: Alex Rousskov <rouss...@measurement-factory.com> >> 1. Your "works" and "does not work" setups currently differ in at least >> three variables: user agent name, slash after the user agent name, and >> acl negation in http_access. Find out which single variable is >> responsible for the breakage by eliminating all other differences. >> >> 2. Post two ALL,2 cache.logs, each containing a single transaction, one >> for the "works" case and one for the "does not work" case polished as >> discussed in #1. > > > I can't really do anything about #1 except maybe leave out the forward slash. > That's because my 2 examples are trying to achieve the opposite. > Let me just rephrase everything so it's crystal clear. > > My goal is to deny all client traffic from browsers that DO NOT have a > specific user-agent string. So this is a negated statement. One of the things > I can't do in Squid is define an ACL with a negated lookahead such as > (?!useragentname). I hope you listen about browser extensions for UA spoofing? > > So I set up two examples. > > Common to both: > > acl allowed_useragent browser MyAllowedUAstring > acl denied_useragent browser MyDeniedUAstring > > # example 1: > http_access deny denied_useragent > http_reply_access deny denied_useragent > deny_info > http://proxy-server1/proxy-error/?a=%a&B=%B&e=%e&E=%E&H=%H&i=%i&M=%M&o=%o&R=%R&T=%T&U=%U&u=%u&w=%w&x=%x&acl=denied_useragent > denied_useragent > > I then run this from my test client: > > # curl --insecure --user-agent MyAllowedUAstring https://www.gentoo.org > -> works as expected (I see the web site). I guess you don't need to see > cache.log here. > > Now I run this: > > # curl --insecure --user-agent MyDeniedUAstring https://www.gentoo.org > -> works as expected (I'm denied access and I see Squid's error page). > I guess there's no need for the full log here either. It boils down to this > anyway: > 2017/11/17 13:24:26.937 kid1| 28,2| RegexData.cc(73) match: > aclRegexData::match: match '(MyDeniedUAstring)' found in 'MyDeniedUAstring' > 2017/11/17 13:24:26.937 kid1| 85,2| client_side_request.cc(745) > clientAccessCheckDone: The request GET https://www.gentoo.org/ is DENIED; > last ACL checked: denied_useragent > > I'm done with example 1. That's because I cannot make a consistent list of > all user agents I want to actively block. Instead, I want to "deny everyone > except one or two". > > Also, since negative lookaheads are not supported in regular expressions, I > change my example 1 to: > > # example 2: > http_access deny !allowed_useragent > http_reply_access deny !allowed_useragent > deny_info > http://proxy-server1/proxy-error/?a=%a&B=%B&e=%e&E=%E&H=%H&i=%i&M=%M&o=%o&R=%R&T=%T&U=%U&u=%u&w=%w&x=%x&acl=denied_useragent > allowed_useragent > > Then I run this from the client: > > # curl --insecure --user-agent MyAllowedUAstring https://www.gentoo.org > -> I was expecting to be allowed access since Squid denies "everything that's > not" MyAllowedUAstring. Well, at least I should have passed the "deny" line > in example 2. > However, I'm being blocked right there. This is the full log: > > 2017/11/17 13:30:42.216 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New > connection on FD 88 > 2017/11/17 13:30:42.216 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection > on local=[::]:3229 remote=[::] FD 88 flags=25 > 2017/11/17 13:30:42.216 kid1| 33,2| client_side.cc(3943) > httpsSslBumpAccessCheckDone: sslBump needed for local=89.16.167.134:443 > remote=10.215.144.48 FD 8 flags=17 method 4 > 2017/11/17 13:30:42.216 kid1| 11,2| client_side.cc(2372) parseHttpRequest: > HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17 > 2017/11/17 13:30:42.216 kid1| 11,2| client_side.cc(2373) parseHttpRequest: > HTTP Client REQUEST: > --------- > CONNECT 89.16.167.134:443 HTTP/1.1 > Host: 89.16.167.134:443 > > > ---------- > 2017/11/17 13:30:42.216 kid1| 85,2| client_side_request.cc(745) > clientAccessCheckDone: The request CONNECT 89.16.167.134:443 is DENIED; last > ACL checked: allowed_useragent > 2017/11/17 13:30:42.216 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 13:30:42.216 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 13:30:42.216 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 13:30:42.226 kid1| 83,2| client_side.cc(3843) clientNegotiateSSL: > clientNegotiateSSL: New session 0x125e030 on FD 8 (10.215.144.48:65262) > 2017/11/17 13:30:42.226 kid1| 11,2| client_side.cc(2372) parseHttpRequest: > HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17 > 2017/11/17 13:30:42.226 kid1| 11,2| client_side.cc(2373) parseHttpRequest: > HTTP Client REQUEST: > --------- > GET / HTTP/1.1 > Host: www.gentoo.org > User-Agent: MyAllowedUAstring > Accept: */* > > > ---------- > 2017/11/17 13:30:42.227 kid1| 28,2| RegexData.cc(73) match: > aclRegexData::match: match '(MyAllowedUAstring)' found in 'MyAllowedUAstring' > 2017/11/17 13:30:42.227 kid1| 88,2| client_side_reply.cc(2073) > processReplyAccessResult: The reply for GET https://www.gentoo.org/ is > ALLOWED, because it matched denied_mimetypes_rep > 2017/11/17 13:30:42.227 kid1| 11,2| client_side.cc(1409) sendStartOfMessage: > HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17 > 2017/11/17 13:30:42.227 kid1| 11,2| client_side.cc(1410) sendStartOfMessage: > HTTP Client REPLY: > --------- > HTTP/1.1 307 Temporary Redirect > Server: squid > Mime-Version: 1.0 > Date: Fri, 17 Nov 2017 12:30:42 GMT > Content-Type: text/html;charset=utf-8 > Content-Length: 0 > Location: > http://proxy-server1/proxy-error/?a=-&B=&e=0&E=%5BNo%20Error%5D&H=89.16.167.134&i=10.215.144.48&M=CONNECT&o=&R=/&T=Fri,%2017%20Nov%202017%2012%3A30%3A42%20GMT&U=https%3A%2F%2F89.16.167.134%2F*&u=89.16.167.134%3A443&w=IT%40mydomain.org&x=&acl=denied_useragent > X-Squid-Error: 403 Access Denied > X-Cache: MISS from proxy-server1 > X-Cache-Lookup: NONE from proxy-server1:3227 > Connection: close > > Note that I have these defaults in my squid conf file: > > acl CONNECT method CONNECT > http_access deny CONNECT !SSL_ports > > Let's try another one: > > # curl --insecure --user-agent MyDeniedUAstring https://www.gentoo.org > -> This is as expected, I guess. > > Full log: > > 2017/11/17 13:30:10.365 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New > connection on FD 88 > 2017/11/17 13:30:10.365 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection > on local=[::]:3229 remote=[::] FD 88 flags=25 > 2017/11/17 13:30:10.365 kid1| 33,2| client_side.cc(3943) > httpsSslBumpAccessCheckDone: sslBump needed for local=89.16.167.134:443 > remote=10.215.144.48 FD 8 flags=17 method 4 > 2017/11/17 13:30:10.365 kid1| 11,2| client_side.cc(2372) parseHttpRequest: > HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17 > 2017/11/17 13:30:10.365 kid1| 11,2| client_side.cc(2373) parseHttpRequest: > HTTP Client REQUEST: > --------- > CONNECT 89.16.167.134:443 HTTP/1.1 > Host: 89.16.167.134:443 > > > ---------- > 2017/11/17 13:30:10.365 kid1| 85,2| client_side_request.cc(745) > clientAccessCheckDone: The request CONNECT 89.16.167.134:443 is DENIED; last > ACL checked: allowed_useragent > 2017/11/17 13:30:10.365 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 13:30:10.365 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 13:30:10.365 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 13:30:10.385 kid1| 83,2| client_side.cc(3843) clientNegotiateSSL: > clientNegotiateSSL: New session 0xdbdc70 on FD 8 (10.215.144.48:65237) > 2017/11/17 13:30:10.386 kid1| 11,2| client_side.cc(2372) parseHttpRequest: > HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17 > 2017/11/17 13:30:10.386 kid1| 11,2| client_side.cc(2373) parseHttpRequest: > HTTP Client REQUEST: > --------- > GET / HTTP/1.1 > Host: www.gentoo.org > User-Agent: MyDeniedUAstring > Accept: */* > > > ---------- > 2017/11/17 13:30:10.386 kid1| 88,2| client_side_reply.cc(2073) > processReplyAccessResult: The reply for GET https://www.gentoo.org/ is > DENIED, because it matched allowed_useragent > 2017/11/17 13:30:10.386 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 13:30:10.386 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 13:30:10.386 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 13:30:10.386 kid1| 88,2| client_side_reply.cc(2073) > processReplyAccessResult: The reply for GET https://www.gentoo.org/ is > ALLOWED, because it matched allowed_useragent > 2017/11/17 13:30:10.386 kid1| 11,2| client_side.cc(1409) sendStartOfMessage: > HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17 > 2017/11/17 13:30:10.386 kid1| 11,2| client_side.cc(1410) sendStartOfMessage: > HTTP Client REPLY: > --------- > HTTP/1.1 302 Found > Server: squid > Mime-Version: 1.0 > Date: Fri, 17 Nov 2017 12:30:10 GMT > Content-Type: text/html;charset=utf-8 > Content-Length: 0 > Location: > http://proxy-server1/proxy-error/?a=-&B=&e=0&E=%5BNo%20Error%5D&H=www.gentoo.org&i=10.215.144.48&M=GET&o=&R=/&T=Fri,%2017%20Nov%202017%2012%3A30%3A10%20GMT&U=https%3A%2F%2Fwww.gentoo.org%2F&u=https%3A%2F%2Fwww.gentoo.org%2F&w=IT%40mydomain.org&x=&acl=denied_useragent > X-Squid-Error: 403 Access Denied > X-Cache: MISS from proxy-server1 > X-Cache-Lookup: NONE from proxy-server1:3227 > Connection: close > > Now for plain HTTP with example 2. > > # curl --user-agent MyDeniedUAstring http://www.fltk.org/index.php > -> As expected. It blocks access. > > Full log: > > 2017/11/17 15:56:52.648 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New > connection on FD 85 > 2017/11/17 15:56:52.648 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection > on local=[::]:3228 remote=[::] FD 85 flags=25 > 2017/11/17 15:56:52.648 kid1| 11,2| client_side.cc(2372) parseHttpRequest: > HTTP Client local=66.39.46.122:80 remote=10.215.144.48 FD 8 flags=17 > 2017/11/17 15:56:52.648 kid1| 11,2| client_side.cc(2373) parseHttpRequest: > HTTP Client REQUEST: > --------- > GET /index.php HTTP/1.1 > Host: www.fltk.org > User-Agent: MyDeniedUAstring > Accept: */* > > > ---------- > 2017/11/17 15:56:52.648 kid1| 85,2| client_side_request.cc(745) > clientAccessCheckDone: The request GET http://www.fltk.org/index.php is > DENIED; last ACL checked: allowed_useragent > 2017/11/17 15:56:52.648 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 15:56:52.648 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 15:56:52.648 kid1| 20,2| store.cc(996) checkCachable: > StoreEntry::checkCachable: NO: not cachable > 2017/11/17 15:56:52.648 kid1| 88,2| client_side_reply.cc(2073) > processReplyAccessResult: The reply for GET http://www.fltk.org/index.php is > ALLOWED, because it matched allowed_useragent > 2017/11/17 15:56:52.648 kid1| 11,2| client_side.cc(1409) sendStartOfMessage: > HTTP Client local=66.39.46.122:80 remote=10.215.144.48 FD 8 flags=17 > 2017/11/17 15:56:52.648 kid1| 11,2| client_side.cc(1410) sendStartOfMessage: > HTTP Client REPLY: > --------- > HTTP/1.1 302 Found > Server: squid > Mime-Version: 1.0 > Date: Fri, 17 Nov 2017 14:56:52 GMT > Content-Type: text/html;charset=utf-8 > Content-Length: 0 > Location: > http://proxy-server1/proxy-error/?a=-&B=&e=0&E=%5BNo%20Error%5D&H=www.fltk.org&i=10.215.144.48&M=GET&o=&R=/index.php&T=Fri,%2017%20Nov%202017%2014%3A56%3A52%20GMT&U=http%3A%2F%2Fwww.fltk.org%2Findex.php&u=http%3A%2F%2Fwww.fltk.org%2Findex.php&w=IT%40mydomain.org&x=&acl=denied_useragent > X-Squid-Error: 403 Access Denied > X-Cache: MISS from proxy-server1 > X-Cache-Lookup: NONE from proxy-server1:3227 > Connection: keep-alive > > However, now comes the interesting part. > > # curl --user-agent MyAllowedUAstring http://www.fltk.org/index.php > -> works as expected (I see the web site). > > Full log: > > 2017/11/17 15:55:23.550 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New > connection on FD 85 > 2017/11/17 15:55:23.550 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection > on local=[::]:3228 remote=[::] FD 85 flags=25 > 2017/11/17 15:55:23.551 kid1| 11,2| client_side.cc(2372) parseHttpRequest: > HTTP Client local=66.39.46.122:80 remote=10.215.144.48 FD 8 flags=17 > 2017/11/17 15:55:23.551 kid1| 11,2| client_side.cc(2373) parseHttpRequest: > HTTP Client REQUEST: > --------- > GET /index.php HTTP/1.1 > Host: www.fltk.org > User-Agent: MyAllowedUAstring > Accept: */* > > > ---------- > 2017/11/17 15:55:23.551 kid1| 28,2| RegexData.cc(73) match: > aclRegexData::match: match '(MyAllowedUAstring)' found in 'MyAllowedUAstring' > 2017/11/17 15:55:23.551 kid1| 82,2| external_acl.cc(805) aclMatchExternal: > bllookup("http www.fltk.org 80 /index.php") = lookup needed > 2017/11/17 15:55:23.551 kid1| 82,2| external_acl.cc(808) aclMatchExternal: > "http www.fltk.org 80 /index.php": queueing a call. > 2017/11/17 15:55:23.551 kid1| 82,2| external_acl.cc(1444) Start: fg lookup in > 'bllookup' for 'http www.fltk.org 80 /index.php' > 2017/11/17 15:55:23.551 kid1| 82,2| external_acl.cc(811) aclMatchExternal: > "http www.fltk.org 80 /index.php": return -1. > 2017/11/17 15:55:23.553 kid1| 82,2| external_acl.cc(1372) > externalAclHandleReply: reply={result=OK, notes={message: www.fltk.org site > not found in blacklist; }} > 2017/11/17 15:55:23.553 kid1| 82,2| external_acl.cc(1288) > external_acl_cache_add: external_acl_cache_add: Adding 'http www.fltk.org 80 > /index.php' = ALLOWED > 2017/11/17 15:55:23.553 kid1| 82,2| external_acl.cc(841) aclMatchExternal: > bllookup = ALLOWED > 2017/11/17 15:55:23.553 kid1| 85,2| client_side_request.cc(745) > clientAccessCheckDone: The request GET http://www.fltk.org/index.php is > ALLOWED; last ACL checked: bl_lookup > 2017/11/17 15:55:23.553 kid1| 85,2| client_side_request.cc(721) > clientAccessCheck2: No adapted_http_access configuration. default: ALLOW > 2017/11/17 15:55:23.553 kid1| 85,2| client_side_request.cc(745) > clientAccessCheckDone: The request GET http://www.fltk.org/index.php is > ALLOWED; last ACL checked: bl_lookup > 2017/11/17 15:55:23.554 kid1| 88,2| client_side_reply.cc(593) cacheHit: > clientProcessHit: Vary detected! > 2017/11/17 15:55:23.554 kid1| 17,2| FwdState.cc(133) FwdState: Forwarding > client request local=66.39.46.122:80 remote=10.215.144.48 FD 8 flags=17, > url=http://www.fltk.org/index.php > 2017/11/17 15:55:23.554 kid1| 44,2| peer_select.cc(280) peerSelectDnsPaths: > Found sources for 'http://www.fltk.org/index.php' > 2017/11/17 15:55:23.554 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths: > always_direct = DENIED > 2017/11/17 15:55:23.554 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths: > never_direct = DENIED > 2017/11/17 15:55:23.554 kid1| 44,2| peer_select.cc(288) peerSelectDnsPaths: > ORIGINAL_DST = local=10.215.144.48 remote=66.39.46.122:80 flags=25 > 2017/11/17 15:55:23.554 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths: > timedout = 0 > 2017/11/17 15:55:23.708 kid1| 11,2| http.cc(2229) sendRequest: HTTP Server > local=10.215.144.48:35373 remote=66.39.46.122:80 FD 13 flags=25 > 2017/11/17 15:55:23.708 kid1| 11,2| http.cc(2230) sendRequest: HTTP Server > REQUEST: > --------- > GET /index.php HTTP/1.1 > User-Agent: MyAllowedUAstring > Accept: */* > Host: www.fltk.org > Cache-Control: max-age=259200 > Connection: keep-alive > > > ---------- > 2017/11/17 15:55:23.884 kid1| ctx: enter level 0: > 'http://www.fltk.org/index.php' > 2017/11/17 15:55:23.884 kid1| 11,2| http.cc(719) processReplyHeader: HTTP > Server local=10.215.144.48:35373 remote=66.39.46.122:80 FD 13 flags=25 > 2017/11/17 15:55:23.884 kid1| 11,2| http.cc(720) processReplyHeader: HTTP > Server REPLY: > --------- > HTTP/1.1 200 OK > Date: Fri, 17 Nov 2017 14:55:23 GMT > Server: Apache/2.4.29 > Cache-Control: no-cache > Vary: Accept-Encoding > Keep-Alive: timeout=5, max=100 > Connection: Keep-Alive > Transfer-Encoding: chunked > Content-Type: text/html > > 3a02 > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" > "http://www.w3.org/TR/html4/loose.dtd";> > <html> > <head> > <title>Fast Light Toolkit - Fast Light Toolkit (FLTK)</title> > <meta http-equiv='Pragma' content='no-cache'> > <meta http-equiv='Content-Type' content='text/html; charset=utf-8'> > <link rel='stylesheet' type='text/css' href='fltk.css'> > <link rel='alternate' title='FLTK RSS' type='application/rss+xml' > href='index.rss'> > <link rel='shortcut icon' href='favicon.ico' type='image/x-icon'> > <meta name='keywords' content='gui toolkit,c++,linux,unix,macos > x,x11,windows'> > </head> > <body> > <table width='100%' border='0' cellspacing='0' cellpadding='0' summary='Page'> > <tr class='header'><td valign='top' width='15' rowspan='2'><a > href='index.php'><img src='images/top-left.gif' width='15' height='70' > border='0' alt=''></a></td><td valign='top' width='224' rowspan='2'><a > href='index.php'><img src='images/top-middle.gif' width='224' height='70' > border='0' alt=''></a></td><td width='100%' height='40'><h1>Fast Light > Toolkit</h1> </td><td align='right' nowrap> > <table cellpadding=0 cellspacing=0 border=0><tr><td valign=top nowrap> > <a href=fltk-rss.xml><img src=images/rss-fee > ---------- > 2017/11/17 15:55:23.885 kid1| ctx: exit level 0 > 2017/11/17 15:55:23.885 kid1| 23,2| url.cc(407) urlParse: urlParse: URI has > whitespace: {icap://127.0.0.1:1344/clamav ICAP/1.0 > } > 2017/11/17 15:55:24.038 kid1| 28,2| RegexData.cc(73) match: > aclRegexData::match: match '(MyAllowedUAstring)' found in 'MyAllowedUAstring' > 2017/11/17 15:55:24.038 kid1| 88,2| client_side_reply.cc(2073) > processReplyAccessResult: The reply for GET http://www.fltk.org/index.php is > ALLOWED, because it matched denied_mimetypes_rep > 2017/11/17 15:55:24.038 kid1| 11,2| client_side.cc(1409) sendStartOfMessage: > HTTP Client local=66.39.46.122:80 remote=10.215.144.48 FD 8 flags=17 > 2017/11/17 15:55:24.038 kid1| 11,2| client_side.cc(1410) sendStartOfMessage: > HTTP Client REPLY: > --------- > HTTP/1.1 200 OK > Date: Fri, 17 Nov 2017 14:55:23 GMT > Server: Apache/2.4.29 > Cache-Control: no-cache > Vary: Accept-Encoding > Content-Type: text/html > Via: ICAP/1.0 proxy-server1.hospitalmanacor.org (C-ICAP/0.5.2 > SquidClamav/Antivirus service ) > X-Cache: MISS from proxy-server1 > X-Cache-Lookup: MISS from proxy-server1:3227 > Transfer-Encoding: chunked > Connection: keep-alive > > How can I modify my example 2 settings so this access control works the same > way with both http and https in an ssl-bumped environment. > > Thanks, > > Vieri > _______________________________________________ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users
-- ************************** * C++: Bug to the future * **************************
signature.asc
Description: OpenPGP digital signature
_______________________________________________ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
