memory stats from -
the things which stand out are:
- Long Strings: 220 MB
- Short Strings: 2.1 GB
- Comm::Connection: 217 MB
- HttpHeaderEntry: 777 MB
- MemBlob: 773 MB
- Entry: 226 MB
What's the best way of debugging this? It there a way to list all of
the Comm::Connection objec
xt week). I did use netstat on it though, and the number of
established TCP connections was 1090 - that is obviously made up of
client->proxy, proxy->origin and proxy->icap connections - my gut
feeling was that it wasn't enough connections to account for 200-odd MB
of Comm::Conn
eported by top, so it looks like it should be accounted for.
There are similarities though - lots of memory going to HttpHeaderEntry
and Short Strings in both cases.
--
- Steve Hill
Technical Director | Cyfarwyddwr Technegol
OpendiumOnline Safety & Web Filtering http://www.opendi
squid-cache.org/show_bug.cgi?id=4526
...which I had thought to have gone away in Squid 5.1. I will apply the
patch next week and see if the problem goes away again.
--
- Steve Hill
Technical Director | Cyfarwyddwr Technegol
OpendiumOnline Safety & Web Filtering http
had about 300 established
connections, which would never go away.
--
- Steve Hill
Technical Director
OpendiumOnline Safety / Web Filteringhttp://www.opendium.com
Enquiries Support
- ---
sa...@opendium.com
in swap. If this was just things held during "active sessions" I would
expect to see the memory freed up again over night when there isn't much
traffic - I see no such reduction in memory usage.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.
om/a/topic/6206681
2. SSL bump the connection and do some slightly painful real-time
analysis of the data.
For what its worth, we sell filtering systems to schools across the UK
and as far as I know, our product is the only one available that can do
the latter.
See: http://www.open
here is something a little abnormal going on to trigger
the leak. Also bear in mind that a single certificate will show up as 2
separate strings, since it has both a subject and an issuer, so we're
probably actually talking about around 65K certificates.
--
- Steve Hill
Technical
low the videos that are embedded in that page.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:sip:st...@opendium.com
Sales / enquiries cont
f youtube whenever a user visits your page, you're going to
need to ssl bump the requests in order to have an ACL based on the
referrer and path. And as you know, ssl bumping involves sticking a
certificate on each device.
--
- Steve Hill
Technical Director
Opendium Limited
ous intercepted CONNECT requests are always responded to
with an HTTP 409 (Conflict) error page."
As I understand it, turning host_verify_strict on causes problems with
CDNs which use DNS tricks for load balancing, so I'm not sure I
understand the rationale behind preventing it f
an rejecting the SSL handshake in the first place.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:sip:st...@opendium.com
Sa
d ports unfirewalled), so I
wonder if this is something new from Microsoft.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:sip:
tains:
Upgrade: websocket
Connection: Upgrade
Unfortunately, since Squid doesn't support websockets I think there's no
way around this - by the time we see the request and can identify it as
Skype we've already bumped it so we're committed to pass it through
Squid's
and the
vendor has stated that they have no intention of fixing it :(
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:sip:st...@open
as made
to find the IP of the server you're connecting to? You would never make
a DNS request for '*.example.com' so I don't see a reason why you would
send an SNI that has a larger scope than the DNS request you made.
--
- Steve Hill
Technical Director
Opendium Li
tions that will never be fixed
to work, or work around the broken apps within Squid and therefore get
them working without the cooperation of the app developers.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xm
etrying the connection, leaking a
ClientRequestContext each time, and before long we've leaked several
gigabytes of memory (on some networks I'm seeing 16GB or more of leaked
RAM over 24 hours!).
Unfortunately I'm a bit lost in the Squid code and can't quite figure
asing this bug off and on for months - hadn't spotted that there
was a bug report open for it :)
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Sales / enquiries:
Email:sa...@opendium.com
Phone:+44-1792-82456
Is there a way of figuring out if the current request is a bumped
request when the http_access ACL is being checked? i.e. can we tell the
difference between a GET request that is inside a bumped tunnel, and an
unencrypted GET request?
--
- Steve Hill
Technical Director
Opendium
ction sites
too and are seeing good results so far.
--
- Steve Hill
Technical Director
OpendiumOnline Safety / Web Filteringhttp://www.opendium.com
Enquiries Support
- ---
sa...@opendium.comsupp...@opendium.com
+44
o be passed along
with the request, but I think the bug mentioned above would cause those
headers to be discarded mid-request in some cases)
--
- Steve Hill
Technical Director
OpendiumOnline Safety / Web Filteringhttp://www.opendium.com
Enquiries Support
combo of the myportname and proto ACLs should do that.
I think when using a nontransparent proxy you can't tell the difference
between:
1. HTTPS requests inside a bumped CONNECT tunnel, and
2. unencrypted "GET https://example.com/ HTTP/1.1" requests made
directly to the proxy.
-
series without problems. But I don't
think any of our sites have as high req/sec load as you.
--
- Steve Hill
Technical Director
OpendiumOnline Safety / Web Filteringhttp://www.opendium.com
Enquiries Support
- ---
sa...@opendiu
(and
submitted patches). That said, with the schools currently on holiday
those fixes haven't yet been well tested on real-world servers - we'll
find out if there are any issues with them when term starts again :)
--
- Steve Hill
Technical Director
OpendiumOnline Safet
ent knows about. The client and squid may expire the records up to 1
second apart.
So what's the solution? (Notably the validation check can't be disabled
without hacking the code).
--
- Steve Hill
Technical Director
OpendiumOnline Safety / Web Filteringhttp://www.o
0x86_64;%20rv:39.0)%20Gecko/20100101%20Firefox/39.0'
The user name given to the external ACL is "-" even though the request
has been authenticated. Setting a->require_auth in
parse_externalAclHelper() makes it work, but obviously just makes %un
behave like %LOGIN, so isn'
the "note"
directive to explicitly stuff the headers into the notes, but it looks
like the note directive doesn't allow you to use format strings (i.e.
"note icap_headers %adapt::note to "%adapt::
--
- Steve Hill
Technical Director
Opendium Limited http:/
Hey!
New message, please read <http://thecontentsplash.com/perhaps.php?nb0k3>
Steve Hill
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
rt.
Has anyone seen this before?
Cheers.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:sip:st...@opendium.com
Sales / e
he SSL handshake
(use wireshark to confirm). In this case, Squid has no way to know what
name to stick in the cert, so will just use the IP instead.
2. The bumping is happening in step 1 instead of step 2 for some reason.
See: http://bugs.squid-cache.org/show_bug.cgi?id=4327
--
- Steve Hil
break everything
that isn't http/https since there will be nothing on the squid server to
handle that traffic.
It doesn't sound like a great idea to me - why not just redirect
http/https traffic at the gateway (TPROXY) instead of mangling DNS?
--
- Steve Hill
Technical Director
O
but it isn't servicing any requests. I realise
that it is a bug for Squid to crash in the first place, but it's
compounded by the occasional complete loss of service when it happens.
Any help would be appreciated. Thanks. :)
--
- Steve Hill
Technical Director
Opendium Limit
"+Sign=signTrusted+SignHash=SHA256" part would indicate that
this is a Squid database key, which is very confusing since with the
certificate cache disabled I wouldn't expect to see these at all.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.
84306 KB 3%
memPool accounted: 84306 KB 3%
memPool unaccounted: 2917158 KB 97%
I am using SMP workers, but turning that off doesn't fix the issue.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
n I've got time. :)
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:sip:st...@opendium.com
Sales / enquiries contacts:
On 30.09.14 15:13, Amos Jeffries wrote:
IIRC the valgrind report is strangely at the end of mgr:info rather
than mgr:mem.
In that case the wiki is wrong. :)
Anyway, it doesn't show up on either of them for me so I guess you might
be right that it isn't "real" leaked memory. (I still thought
On 29.09.14 10:04, Steve Hill wrote:
I _think_ I have narrowed it down to something ICAP related
Looks like I was wrong - it actually seems to be external ACL related.
I have an external ACL defined as:
external_acl_type preauth cache=0 children-max=1 concurrency=100 ttl=0
negative_ttl=0
s aren't
reproducing the same issue - back to the drawing board. :(
--
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:
On 01.10.14 13:54, Amos Jeffries wrote:
I recently opened a bug about this, that I will update now:
http://bugs.squid-cache.org/show_bug.cgi?id=4088
Thank you for the reminder. I will start work on this next.
I'm afraid the patch you added to that bug report doesn't work for me
(in fact, i
On 30.09.14 16:13, Amos Jeffries wrote:
I'm trying to figure out if there's a way of convincing valgrind to
dump info about all the currently allocated memory while the
program is still running - there would be a lot of legitimate stuff
in the report, but hopefully a few hundred MB of memory tha
On 08.10.14 15:05, Amos Jeffries wrote:
New patch added to bug 4088. Please see if it resolves the
external_acl_type leak.
Seems to fix the problem - thank you!
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager
_localhost
adaptation_access iceni_respmod_postcache deny dstdomain_localhost
adaptation_access iceni_respmod_postcache allow all
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:
dly,
mgr:cached_ssl_cert doesn't appear to work with SMP workers.
My Squid process sizes are still larger than I would expect, so I'll
need to do more investigation, but reducing dynamic_cert_mem_cache_size
has stopped the rapid unbounded growth I have been seeing.
--
- Steve Hi
ighest TLS version supported by both server and client?
It works correctly when FireFox connects directly to the web server
rather than going through the proxy.
So my question is: is the web server broken, or am I misunderstanding
something?
Many thanks.
--
- Steve Hill
Technical Director
he same URI, so the client never gets the object it
requested.
For now I have worked around it with:
request_header_access Via deny https
request_header_access X-Forwarded-For deny https
But it does make me wonder if inserting the headers into bumped traffic
is a sensible thing to do.
--
anks for this - I have emailed them, which I fully expect them to
ignore :)
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:sip
rafficServer [c
s f ]), http/1.1 r15.ycpi.dee.yahoo.net (ApacheTrafficServer [cMsSfW])
Server: ATS
Strict-Transport-Security: max-age=172800
Location:
https://uk.finance.yahoo.com/news/degrees-lead-best-paid-careers-141513989.html
Content-Length: 0
Age: 0
Connection: keep-alive
--
- St
I'm seeing a lot of this in both 3.4.6 and 3.4.9:
2014/11/18 15:08:48 kid1| assertion failed: DestinationIp.cc:60:
"checklist->conn() && checklist->conn()->clientConnection != NULL"
I've looked through Bugzilla and couldn't see anything regarding this
h openssl's s_client directly.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:sip:st...@opendium.com
Sales / enquiries
deny dstdomain_localhost
adaptation_access iceni_respmod_postcache allow all
--
- Steve
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:
tunnel is clients making websockets connections over https,
which of course won't work through a bumped connection since Squid
doesn't support HTTP upgrade requests)
Many thanks.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct con
h (Negotiate and Basic)
- SSL bump
- Both TPROXY and non-transparent (majority of the traffic is
non-transparent)
- Uses an upstream proxy for most HTTP (not HTTPS)
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:
ething wrong with the sharing/persistence
if we're accumulating so many "token" notes. As well as the performance
problems, there could be some race conditions lurking here?
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts
-with-dl'
'--with-openssl' '--with-pthreads' 'build_alias=x86_64-redhat-linux-gnu'
'host_alias=x86_64-redhat-linux-gnu'
'target_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-fPIE -Os -g -pipe
-fsigned-char -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURC
t have the same keys as B's notes, before using appendNewOnly() to
merge them?
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:s
t the "correct" way is to fix it - we could
specifically avoid appending "token" notes in the Negotiate/NTLM code,
or we could do something more generic in the absorb() method. (My
preference is the latter unless anyone can think why it would be a bad
idea).
--
- Steve H
On 06.01.15 12:15, Steve Hill wrote:
Alternatively, A->absorb(B) could be altered to remove any notes from A
that have the same keys as B's notes, before using appendNewOnly() to
merge them?
I've implemented this for now in the attached patch and am currently
testing it. In
ead of the internal cert generator?
Thanks.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:sip:st...@opendium.com
Sales / enquirie
not using ssl_crtd). I also can't see
anything wrong with the certificate chain.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:
#x27;d welcome any input!).
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:sip:st...@opendium.com
Sales / enquiries contacts:
. as
you would expect.
Am I missing something?
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:sip:st...@opendium.com
Sales
the helper existed). Although
I've got to admit that I was a bit surprised to be told that the way
I've been successfully using Squid is impossible. :)
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@o
k this data together.
However, the proxy does not always have control of the DHCP/DHCPv6 servers.
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:st...@opendium.com
Email:st...@opendium.com
Phone:
ure HTTP proxy authentication. Luckily iOS can use WISPr
to automatically log into a portal, sadly vanilla Android still doesn't
include a WISPr client (I'd put money on this being down to patents!).
--
- Steve Hill
Technical Director
Opendium Limited http://www
hey always demand that I spend a lot of
my time collecting debug logs, but then they sit on the report and never
actually fix it (again, I've never had a resolution to a bug I've
reported to Apple, despite supplying them with extensive debugging).
/rant :)
--
- Steve Hill
Technical Dir
certificate.
3. Clients require access to some external servers to validate HTTPS
certs before they have authenticated.
4. If you want to support WISPr then (2) and (3) are mandatory.
5. External ACL caching
You might be able to do it with internal ACLs, but... pain :)
--
- Steve Hill
code 0xf3
) at main.cc:1511
#42 0x7ffe148af2e9 in SquidMainSafe (argc=Unhandled dwarf expression
opcode 0xf3
) at main.cc:1243
#43 main (argc=Unhandled dwarf expression opcode 0xf3
) at main.cc:1236
(sorry about the DWARF errors - it looks like I've got a version
mismatch between gcc an
68 matches
Mail list logo
