hi Eliezer, unfortunately that did not work ie
acl NoSSLInterceptRegEx ssl::server_name_regex (^|.*\.)redshift3d\.com$ acl NoSSLInterceptDstDom ssl::server_name .redshift3d.com acl NoSSLInterceptAnyOf any-of NoSSLInterceptDstDom NoSSLInterceptRegEx ssl_bump peek DiscoverSNIHost ssl_bump splice NoSSLInterceptAnyOf ssl_bump bump all but strange is on the firewall if i make a rule for that network, port 80, 443 to access redshift3d.com it works, so ive narrowed it down to the squid thanks, rob On Thu, 29 Jun 2023 at 11:40, <ngtech1...@gmail.com> wrote: > Hey Rob, > > it’s a great question. > I will assume you are using squid 5.x since it’s the stable one. > There is a configuration reference documentation at: > http://www.squid-cache.org/Versions/v5/cfgman/ > > And the relevant one is ‘acl’: > http://www.squid-cache.org/Versions/v5/cfgman/acl.html > > In the config we the next options: > * dstdom_regex > * dstdomain > * ssl::server_name > * ssl::server_name_regex > > Each one of them is different in two things. > The dstdomain is happening when the http request verification is done, > usually in the http_access part. > So for a CONNECT request in your case it should work but will only work > for http_access rules ie allow or deny the connection to being proxied by > the proxy. > All this happens before the bump stage. > The dstdom_regex is doing the same but with another "lookup" mechanism. > Just to illustrate we will use the domain www.example.com. > > With dstdomain .example.com which is a wildcard domain the www.example > .com will match since what it does it.. > Take apart the .example.com into .com and .example and example > (logically, not the code actually does..) > Then when a request for www.example-2.com arrives the dstdomain acl > checks the strings one by one ie: > com > example-2 > www > > So the "com" would match and since it has a dot "." as a prefix it > continues to the next part of the domain name ie: > example-2 > > and it will try to match it to "example" from the ".example.com'. > then it will fail and there for it will declare the specific definition in > the acl as "do not match. > But for www.example .com it would be different: > > Squid will find that the .com is in the "com" part of www.example.com and > will continue to the next part ie > "example" and it will match. > Then it will see the dot "." which means that all subdomains are a part of > this rule ie both example.com and any other domain > under .example.com will match so... www.example.com and www2.example.com > and w.example.com will match and this is compared to > example.com > > with dstdomain example.com it will only match a full match of example.com > but not www.example.com and it's pointless to define both: > - example.com > - .example.com > > In the same acl file. > Squid will compare it to the basic proxy url request and the Host: header > inside of it. > (Amos might remember which one of these wins or which one is the main one) > > So that's how a dstdomain test works. > The dstdom_regex is applied on the same "part" of the proxy request ie > request url and Host header but... > Tries to match it with a regex that is a much more "CPU" intensive but can > do magic if you know how to use it right. > The pros are that you can use one regex to match about 100 domains however > dstdomain is much faster in many cases. > To test regex in general I love: > https://rubular.com/ > > and it's very simple to put a line with the url or the domain itself and > write the pattern and then get instant result. > There are many sites which does this exact same thing. > You need: activate.redshift3d.com for the test. > > For the server_name dstdom and regex the difference is on what part of the > "known" information on the request done. > Ie it's not on the known plain text proxy request but a "peek" into the > client or server TLS part of the connection bumping. > It's possible on TLS 1.2 with ease but 1.3 with encryption it's a whole > other story. > (Alex and his team is more involved in this part then me) > > It's a bit more complex in the server_name part since from what I remember > the server_name can be the SNI itself or parts of the > certificate of the server. > In the certificate of the server in many cases there is no real domain but > a catch all ie multidomain which is for example: > *.example.com > > The ssl::server_name is like dstdomain so .example.com should match both > SNI with www.example.com and the *.example.com > If it doesn't then it’s a bug.. > > About the ssl::server_name_regex you will need to test and match the regex > against couple possibilities that are in the certificate ie: > *.example.com > activation.example.com > > etc... > > The current certificate I see is: > Certificate: > Data: > Version: 3 (0x2) > Serial Number: > c0:71:02:fc:1f:e5:64:5b:0d:d6:ae:c8:1a:17:e6:80 > Signature Algorithm: sha256WithRSAEncryption > Issuer: C = GB, ST = Greater Manchester, L = Salford, O = Sectigo > Limited, CN = Sectigo RSA Domain Validation Secure Server CA > Validity > Not Before: Dec 14 00:00:00 2022 GMT > Not After : Jan 12 23:59:59 2024 GMT > Subject: CN = activate.redshift3d.com > Subject Public Key Info: > Public Key Algorithm: rsaEncryption > RSA Public-Key: (2048 bit) > Modulus: > 00:cd:f9:bd:58:a4:7a:16:3f:23:12:9d:e0:1e:39: > a0:ef:66:ae:b3:ae:32:5c:69:9e:cc:89:83:fb:a2: > e0:c7:9d:86:aa:e6:9b:b3:d9:0c:bb:35:36:2f:cf: > be:ec:1e:62:ca:d8:ab:16:66:6e:00:8d:f7:42:3d: > 5b:d6:bf:a0:30:a1:c2:6f:5e:69:76:ef:0c:38:73: > d2:e8:42:b8:83:04:a3:2c:da:22:a4:c4:13:71:38: > 36:00:1b:e7:b3:f0:fe:6e:59:17:11:e7:0a:81:3e: > 04:b2:ad:e0:61:c1:15:f3:9a:36:33:24:a6:f1:0d: > b6:e6:32:91:34:54:7c:da:40:14:72:06:25:91:78: > 75:07:84:62:47:7b:3f:05:60:e6:35:d3:27:55:1e: > ff:23:82:18:09:f5:8a:0f:a6:cb:02:bc:d6:09:98: > 54:5b:10:e3:d2:09:4e:31:2a:75:42:29:00:86:9b: > f9:78:2f:fd:99:9e:2e:10:3e:bc:84:f5:9b:3a:3f: > fa:d9:b6:20:c0:2e:86:f3:de:14:be:3a:1a:05:30: > 3b:00:e6:50:8d:4c:13:a3:97:dc:f4:03:9e:84:31: > 0d:23:f2:02:50:6d:53:a5:30:03:b0:fd:f2:46:64: > 49:22:a4:d1:5a:00:fb:78:37:fc:51:b1:f0:6e:41: > 3b:5d > Exponent: 65537 (0x10001) > X509v3 extensions: > X509v3 Authority Key Identifier: > > keyid:8D:8C:5E:C4:54:AD:8A:E1:77:E9:9B:F9:9B:05:E1:B8:01:8D:61:E1 > > X509v3 Subject Key Identifier: > 5D:BC:D9:F2:C4:99:83:5E:C0:3A:F1:BF:FE:F2:E3:92:92:F1:F0:29 > X509v3 Key Usage: critical > Digital Signature, Key Encipherment > X509v3 Basic Constraints: critical > CA:FALSE > X509v3 Extended Key Usage: > TLS Web Server Authentication, TLS Web Client > Authentication > X509v3 Certificate Policies: > Policy: 1.3.6.1.4.1.6449.1.2.2.7 > CPS: https://sectigo.com/CPS > Policy: 2.23.140.1.2.1 > > Authority Information Access: > CA Issuers - URI: > http://crt.sectigo.com/SectigoRSADomainValidationSecureServerCA.crt > OCSP - URI:http://ocsp.sectigo.com > > X509v3 Subject Alternative Name: > DNS:activate.redshift3d.com, DNS: > www.activate.redshift3d.com > CT Precertificate SCTs: > Signed Certificate Timestamp: > Version : v1 (0x0) > Log ID : > 76:FF:88:3F:0A:B6:FB:95:51:C2:61:CC:F5:87:BA:34: > > B4:A4:CD:BB:29:DC:68:42:0A:9F:E6:67:4C:5A:3A:74 > Timestamp : Dec 14 15:34:39.227 2022 GMT > Extensions: none > Signature : ecdsa-with-SHA256 > > 30:44:02:20:6E:D0:96:68:F3:07:A0:5F:DD:C5:B2:61: > > 2F:51:6B:06:4A:3C:7F:9B:DA:08:6D:5A:31:0D:B8:0B: > > 83:7D:D6:37:02:20:64:66:3F:A2:40:87:78:9E:C8:90: > > 7C:EE:7C:77:3C:BB:43:C1:9F:52:54:F1:04:85:B6:AC: > 6B:F2:36:8E:E3:03 > Signed Certificate Timestamp: > Version : v1 (0x0) > Log ID : > 3B:53:77:75:3E:2D:B9:80:4E:8B:30:5B:06:FE:40:3B: > > 67:D8:4F:C3:F4:C7:BD:00:0D:2D:72:6F:E1:FA:D4:17 > Timestamp : Dec 14 15:34:39.153 2022 GMT > Extensions: none > Signature : ecdsa-with-SHA256 > > 30:44:02:20:69:BA:F2:DD:32:B3:48:0C:26:E3:FB:F0: > > DD:4E:5E:10:95:1E:B4:59:5B:67:64:C4:F2:40:7C:A8: > > 62:35:71:FE:02:20:13:15:41:26:E5:9C:DB:34:DA:D7: > > B9:CB:B3:1A:4E:33:C7:46:7F:D9:93:45:25:7F:DE:A3: > 72:EB:07:03:DB:C5 > Signed Certificate Timestamp: > Version : v1 (0x0) > Log ID : > EE:CD:D0:64:D5:DB:1A:CE:C5:5C:B7:9D:B4:CD:13:A2: > > 32:87:46:7C:BC:EC:DE:C3:51:48:59:46:71:1F:B5:9B > Timestamp : Dec 14 15:34:39.124 2022 GMT > Extensions: none > Signature : ecdsa-with-SHA256 > > 30:45:02:20:2C:E2:85:9C:A6:54:1B:1C:31:E5:F8:37: > > E9:CD:09:8B:D8:26:29:E4:C7:65:94:9C:FF:32:D2:41: > > CD:16:A3:51:02:21:00:A0:2F:C3:F7:A6:55:3B:21:EB: > > 9B:CA:6E:4E:07:A2:8C:40:4B:E2:27:D6:82:44:0F:09: > C9:F7:7D:1B:72:6F:13 > Signature Algorithm: sha256WithRSAEncryption > 25:bd:bb:de:57:c0:7f:07:5e:18:62:2e:0b:d3:03:54:a7:45: > ab:c6:1f:e2:f6:58:ff:6e:8e:6b:4f:09:9a:87:66:32:81:7f: > f4:35:4f:7e:65:e5:6a:04:d6:62:62:ff:d9:3a:f2:6f:19:ba: > fa:e6:35:0e:2a:44:5c:3b:ee:9d:97:72:05:86:0c:4c:01:c1: > f0:8c:21:c1:c4:84:54:d8:a8:05:25:18:72:db:f7:53:9b:f1: > 13:d6:0b:bc:92:6e:01:e3:fd:de:a1:45:e9:29:37:e1:2e:64: > 36:b4:4d:38:c1:60:02:6a:17:3d:87:a2:5f:33:3b:86:eb:0d: > cc:dd:fa:d4:43:58:50:43:e7:b7:ec:0a:4f:86:72:15:e5:30: > c9:bb:5f:0b:83:9c:26:6f:60:49:dd:1a:7c:92:45:45:4e:b5: > ce:cd:64:8c:12:83:e9:3d:5c:6b:65:97:75:99:4c:66:eb:d0: > 3a:ca:18:62:8a:08:07:16:ab:09:66:bd:65:43:94:00:d9:79: > 3e:84:b4:60:7d:7e:f9:09:3c:fe:2d:ad:98:94:17:0c:24:8f: > e1:a2:74:b6:3b:68:c0:01:f9:67:e8:b9:d2:6a:65:9e:99:a3: > 4a:5f:39:31:ae:c1:59:02:7b:ef:db:b2:94:06:f8:1a:74:c1: > d7:5b:5b:6a > > So the DNS names it will check are: > Subject: CN = activate.redshift3d.com > DNS:activate.redshift3d.com, DNS: > www.activate.redshift3d.com > > So to summarize the checks of ssl::server_name/ will be done on: > * activate.redshift3d.com > * activate.redshift3d.com > * www.activate.redshift3d.com > > So . redshift3d.com ssl::server_name should match the certificate. > If for any reason it doesn't work you can try ssl::server_name_regex with > something like: > (^|\.)activate\.redshift3d\.com$ > > Or just to verify if there is a bug in squid code try: > (^|\.)activate\.redshift3d\.com > > Now, the splice should be able to take into account also dstdomain and > dstdom_regex but it should match them only if they exist in a plain text > form like in any simple forward proxy CONNECT request. > If for any reason it doesn’t work we should investigate what might cause > this issue. > > I hope the scroll I wrote make sense to you and with hopes it will clear > out the doubts about the wiki article you mentioned. > I believe this is considered a summary of the subject and if Alex and > others might think so it can be converted into an example article in the > wiki. > > Let me know if this makes sense and resolve the issue. > > Yours, > Eliezer > > From: robert k Wild <robertkw...@gmail.com> > Sent: Thursday, June 29, 2023 12:18 > To: ngtech1...@gmail.com > Cc: Squid Users <squid-users@lists.squid-cache.org> > Subject: Re: [squid-users] make URL bypass squid proxy > > very clever, so you bunch all the acls up > > acl NoSSLInterceptAnyOf any-of NoSSLInterceptDstDom > NoSSLInterceptDstDomFile NoSSLInterceptRegEx NoSSLInterceptRegExFile > > the key word is "any-of" ie if the url hits any one do that first > > what about instead of making it > > ssl::server_name_regex > > i make it > > dstdom_regex > > On Thu, 29 Jun 2023 at 01:38, <mailto:ngtech1...@gmail.com> wrote: > Hey Rob, > > The first thing is to allow the domain in the http_acces just to be sure > and use a basic deny all bottom line. > Let me try to simplify your squid.conf > In a link: > https://gist.github.com/elico/b49f4a28d4b5db5ba882b10d40872d5e > > In plain text: > ## START OF FILE > # SSL Interception basic rules > acl DiscoverSNIHost at_step SslBump1 > > acl NoSSLInterceptRegEx ssl::server_name_regex (^|.*\.)redshift3d\.com$ > acl NoSSLInterceptRegExFile ssl::server_name_regex > "/usr/local/squid/etc/no-intercept-ssl-regex.txt" > > acl NoSSLInterceptDstDom ssl::server_name .redshift3d.com > acl NoSSLInterceptDstDomFile ssl::server_name > "/usr/local/squid/etc/no-intercept-ssl-dstdom.txt" > > ## Any of will test what ever rule match first in a first match/hit fasion > acl NoSSLInterceptAnyOf any-of NoSSLInterceptDstDom > NoSSLInterceptDstDomFile NoSSLInterceptRegEx NoSSLInterceptRegExFile > > ssl_bump peek DiscoverSNIHost > ssl_bump splice NoSSLInterceptAnyOf > ssl_bump bump all > > #SSL Bump port > http_port 3128 ssl-bump cert=/usr/local/squid/etc/ssl_cert/myCA.pem > generate-host-certificates=on dynamic_cert_mem_cache_size=4MB > sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s > /var/lib/ssl_db -M 4MB > > ## http_access acls, will apply on incomming requests and not on responses > acl special_url_regex url_regex https?://(^|.*\.)redshift3d\.com\/ > acl special_url_regex_file url_regex > "/usr/local/squid/etc/special_url_regex.txt" > > acl special_url_dst_dom dstdomain .redshift3d.com > acl special_url_dst_dom_file dstdomain > "/usr/local/squid/etc/special_url_dstdom.txt" > > acl special_url_any_of any-of special_url_dst_dom special_url_dst_dom_file > special_url_regex special_url_regex_file > > acl localnet src http://192.168.0.0/16 > acl localnet src http://10.0.0.0/8 > > http_access allow localnet special_url_any_of > http_access deny all > ## END OF FILE > > Once the above will work try to add other http_access rule like reply > access rules > > Let me know what happens, > Eliezer > > From: robert k Wild <mailto:robertkw...@gmail.com> > Sent: Tuesday, June 27, 2023 09:36 > To: mailto:ngtech1...@gmail.com > Cc: Squid Users <mailto:squid-users@lists.squid-cache.org> > Subject: Re: [squid-users] make URL bypass squid proxy > > Hi Eliezer, > > this is a snippet of my whitelist and no intercept SSL config > > #SSL Interception > acl DiscoverSNIHost at_step SslBump1 > acl NoSSLIntercept ssl::server_name_regex > "/usr/local/squid/etc/interceptssl.txt" > ssl_bump peek DiscoverSNIHost > ssl_bump splice NoSSLIntercept > ssl_bump bump all > # > #SSL Bump > http_port 3128 ssl-bump cert=/usr/local/squid/etc/ssl_cert/myCA.pem > generate-host-certificates=on dynamic_cert_mem_cache_size=4MB > sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s > /var/lib/ssl_db -M 4MB > # > #deny up MIME types > acl upmime req_mime_type "/usr/local/squid/etc/mimedeny.txt" > # > #deny URL links > acl url_links url_regex "/usr/local/squid/etc/linksurl.txt" > # > #allow special URL paths > acl special_url url_regex "/usr/local/squid/etc/urlspecial.txt" > # > #deny down MIME types > acl downmime rep_mime_type "/usr/local/squid/etc/mimedeny.txt" > # > http_reply_access allow special_url > http_reply_access deny downmime > #http_access deny upmime > #http_access deny url_links > # > #HTTP_HTTPS whitelist websites > acl whitelist ssl::server_name_regex "/usr/local/squid/etc/urlwhite.txt" > # > http_access allow activation whitelist > http_access deny all > > so basically no SSL interception > > #SSL Interception > acl DiscoverSNIHost at_step SslBump1 > acl NoSSLIntercept ssl::server_name_regex > "/usr/local/squid/etc/interceptssl.txt" > ssl_bump peek DiscoverSNIHost > ssl_bump splice NoSSLIntercept > ssl_bump bump all > > and whitelisting > > #HTTP_HTTPS whitelist websites > acl whitelist ssl::server_name_regex "/usr/local/squid/etc/urlwhite.txt" > > in both txt files ie > > /usr/local/squid/etc/interceptssl.txt > /usr/local/squid/etc/urlwhite.txt > > i have a URL that first i have to whitelist and then if i want squid not > to inspect the url traffic i put it in the SSL interception (i do this as > some websites dont like MITM ) > > but even putting the URL in question in both files im still having issues > with this website ie its still being detected that its passing through a > proxy > > thanks, > rob > > On Mon, 26 Jun 2023 at 23:35, <mailto:mailto:ngtech1...@gmail.com> wrote: > Hey Robert, > > I am not sure what forward proxy setup you have there. > A simple forward proxy? > What tool are you using for whitelisting? > You can use an external acl helper to allow dynamic updates of the > whitelists or > to periodic update your lists and reload. > It will depend on the size of your lists. > What OS are you using for your squid proxy? > > More details will help us help you. > > Eliezer > > From: squid-users <mailto:mailto:squid-users-boun...@lists.squid-cache.org> > On Behalf Of robert k Wild > Sent: Monday, June 26, 2023 22:25 > To: Squid Users <mailto:mailto:squid-users@lists.squid-cache.org> > Subject: [squid-users] make URL bypass squid proxy > > hi all, > > i have set up squid for url whitelisting and no intercept SSL (see below) > > https://wiki.squid-cache.org/ConfigExamples/Caching/AdobeProducts > > but some websites i want the client to bypass the squid proxy and go > straight to the website as i think this is why a url isnt working even when > i add the url to both files ie urlwhite and no intercept SSL > > > > thanks, > rob > > -- > Regards, > > Robert K Wild. > > > -- > Regards, > > Robert K Wild. > > > -- > Regards, > > Robert K Wild. > > -- Regards, Robert K Wild.
_______________________________________________ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
