2010/4/1 Konstantin Kolinko :
>> [snip]
>>
>>> # Problematic frame:
>>> # C [libapr-1.so.0+0x1c0a0]
>>
>> [snip]
>>>
>>> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
>>> j org.apache.tomcat.jni.Socket.accept(J)J+0
>>
>> So, this happens in APR's Socket.accept method? I suspect t
> [snip]
>
>> # Problematic frame:
>> # C [libapr-1.so.0+0x1c0a0]
>
> [snip]
>>
>> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
>> j org.apache.tomcat.jni.Socket.accept(J)J+0
>
> So, this happens in APR's Socket.accept method? I suspect that if
> something were wrong in that met
> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
> Subject: Re: [OT] Re: jvm exits without trace
>
> So, this happens in APR's Socket.accept method? I suspect that if
> something were wrong in that method, it would be apparent to roughly
> 50% of the world&
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Taylan,
On 3/30/2010 5:51 AM, Taylan Develioglu wrote:
> We're still having some crashes (segfaults), they may or not be related
> to the previous ones.
:(
> It seems libapr had something to do with it.
[snip]
> # Problematic frame:
> # C [libapr
Peter Crowther wrote:
On 27 March 2010 00:22, David Kerber wrote:
BTW, after looking back at my development notes, the jdbc-odbc bridges,
while a pain to set up, have better performance than the type 4 drivers that
Sybase also offers.
That's frightening, given the extra layers of code for
On 27 March 2010 00:22, David Kerber wrote:
> BTW, after looking back at my development notes, the jdbc-odbc bridges,
> while a pain to set up, have better performance than the type 4 drivers that
> Sybase also offers.
>
> That's frightening, given the extra layers of code for the ODBC bridge. I
David kerber wrote:
It looks like we may have had one of these this AM, but under somewhat
different conditions: Windows server 2008, JVM 1.6.0_17 64-bit Server
VM, TC 5.5.28. No tracks in any of the TC logs, and the Windows event
viewer had the singularly un-useful "The Tomcat5 service termi
On 3/26/2010 3:39 PM, Caldarale, Charles R wrote:
From: David kerber [mailto:dcker...@verizon.net]
Subject: Re: jvm exits without trace
What alternative is there, without going to a java database?
*Every* real database that I'm aware of supplies type 4 JDBC drivers; none use
the
On 3/26/2010 3:27 PM, Caldarale, Charles R wrote:
From: Christopher Schultz [mailto:ch...@christopherschultz.net]
Subject: Re: jvm exits without trace
Yikes: whenever I see jdbcodbc I cringe!
As everyone should.
Looks like whatever ntdll.dll is. That doesn't sound like a
JDBC driver
> From: David kerber [mailto:dcker...@verizon.net]
> Subject: Re: jvm exits without trace
>
> What alternative is there, without going to a java database?
*Every* real database that I'm aware of supplies type 4 JDBC drivers; none use
the mind-bogglingly bad JDBC-ODBC bridge. C
On 3/26/2010 3:19 PM, Christopher Schultz wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
David,
On 3/26/2010 1:40 PM, David kerber wrote:
It looks like we may have had one of these this AM, but under somewhat
different conditions: Windows server 2008, JVM 1.6.0_17 64-bit Server
VM, TC 5
> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
> Subject: Re: jvm exits without trace
>
> Yikes: whenever I see jdbcodbc I cringe!
As everyone should.
> Looks like whatever ntdll.dll is. That doesn't sound like a
> JDBC driver cause to me.
It'
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
David,
On 3/26/2010 1:40 PM, David kerber wrote:
> It looks like we may have had one of these this AM, but under somewhat
> different conditions: Windows server 2008, JVM 1.6.0_17 64-bit Server
> VM, TC 5.5.28. No tracks in any of the TC logs, and t
)
vm_info: Java HotSpot(TM) 64-Bit Server VM (14.3-b01) for windows-amd64
JRE (1.6.0_17-b04), built on Oct 11 2009 00:46:08 by "java_re" with MS
VC++ 8.0
time: Fri Mar 26 10:21:01 2010
elapsed time: 80912 seconds
On 3/22/2010 3:33 PM, Carl wrote:
Dan,
6u18 did not work for us,
Message -
From: "George Sexton"
To: "'Tomcat Users List'"
Sent: Monday, March 22, 2010 2:48 PM
Subject: RE: [OT] Re: jvm exits without trace
-Original Message-
From: Dan Armbrust [mailto:daniel.armbrust.l...@gmail.com]
Sent: Monday, March 22, 2010 12:17
On 22/03/2010 22:12, Martin Gainty wrote:
>
> Carl-
>
> enable logging FileHandler and set level to finest e.g.
> org.apache.catalina.level=FINEST
Not recommended unless you actually want Tomcat to take over 20 minutes
to start up.
Added to which, that isn't going to help track down this JVM b
er aucune responsabilité
pour le contenu fourni.
> From: c...@etrak-plus.com
> To: users@tomcat.apache.org
> Subject: Re: [OT] Re: jvm exits without trace
> Date: Mon, 22 Mar 2010 14:33:54 -0500
>
> Dan,
>
> 6u18 did not work for us, crashed with the same regularity a
> -Original Message-
> From: Dan Armbrust [mailto:daniel.armbrust.l...@gmail.com]
> Sent: Monday, March 22, 2010 12:17 PM
> To: Tomcat Users List
> Subject: Re: [OT] Re: jvm exits without trace
>
> On Tue, Mar 16, 2010 at 4:46 PM, Carl wrote:
> > My approach
mcat Users List"
Sent: Monday, March 22, 2010 1:17 PM
Subject: Re: [OT] Re: jvm exits without trace
On Tue, Mar 16, 2010 at 4:46 PM, Carl wrote:
My approach is to get something (a JVM) that works and then gradually
change
until it breaks. Then, I know what is causing the problem. To date, I
On Tue, Mar 16, 2010 at 4:46 PM, Carl wrote:
> My approach is to get something (a JVM) that works and then gradually change
> until it breaks. Then, I know what is causing the problem. To date, I
> haven't been able to get a JVM that works.
>
I have had a lot of issues finding stable JVMs since
On 17 March 2010 10:22, Taylan Develioglu wrote:
> Ofcourse this works better if I really attach the file.
>
> [...]
> Java Threads: ( => current thread )
> 0x7f3d3c174000 JavaThread "MSN-6488" daemon [_thread_in_native,
> id=28966, stack(0x42a4f000,0x42a7)]
>
[...]
Are
On 17 March 2010 10:16, Taylan Develioglu wrote:
> Here's a hs_err file after a crash I had yesterday.
The list usually strips attachments; could you paste it?
> It's a sigsegv in GCTaskThread. From the occupation in eden it looks
> like it happened during a scavenge (ParNew).
>
> Ick. To ro
Here's a hs_err file after a crash I had yesterday. We turned off some
things in our code without restarting and the crashes have virtually
stopped but we do still get the off one here and there where the
application has not been restarted, could be that the problem lingers
and builds up in time, w
Carl wrote:
My approach is to get something (a JVM) that works and then gradually
change until it breaks. Then, I know what is causing the problem. To
date, I haven't been able to get a JVM that works.
I think we understand that, and agree.
Our remarks were "tongue in cheek", if that is the
tell.
Thanks,
Carl
- Original Message -
From: "André Warnier"
To: "Tomcat Users List"
Sent: Tuesday, March 16, 2010 3:20 PM
Subject: Re: [OT] Re: jvm exits without trace
Ognjen Blagojevic wrote:
Hi,
"Tomcat dies suddenly" thread was exciting almost as &q
Ognjen Blagojevic wrote:
Hi,
"Tomcat dies suddenly" thread was exciting almost as "Prison Break" TV
series. I couldn't wait to find out what would be solution to the
problem, and I must admit that just downgrading to lower sub-sub-sub
version JVM left me a bit disappointed. :)
+1
It sounds
Ognjen Blagojevic wrote:
Hi,
"Tomcat dies suddenly" thread was exciting almost as "Prison Break" TV
series. I couldn't wait to find out what would be solution to the
problem, and I must admit that just downgrading to lower sub-sub-sub
version JVM left me a bit disappointed. :)
Well, I would
ge - From: "Taylan Develioglu"
To: "Tomcat Users List"
Sent: Tuesday, March 16, 2010 7:41 AM
Subject: Re: jvm exits without trace
With parent I meant the main JVM process as opposed to forked processes
or threads, sorry to confuse you there. Stracing the threads generat
ot;Taylan Develioglu"
To: "Tomcat Users List"
Sent: Tuesday, March 16, 2010 7:41 AM
Subject: Re: jvm exits without trace
With parent I meant the main JVM process as opposed to forked processes
or threads, sorry to confuse you there. Stracing the threads generates
too much data t
With parent I meant the main JVM process as opposed to forked processes
or threads, sorry to confuse you there. Stracing the threads generates
too much data to store so I had to settle with the parent process.
To answer your other questions.
The code is 100% pure java, why it causes this messy cr
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Taylan,
On 3/15/2010 10:19 AM, Taylan Develioglu wrote:
> The cause for the crashes was in our own application code, we're
> currently investigating the exact reason.
Yeah, I'd like to second Chuck's question: was it native code?
> A strace of the p
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Chuck,
On 3/15/2010 11:24 AM, Caldarale, Charles R wrote:
>> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
>> Subject: Re: jvm exits without trace
>>
>> On 3/15/2010 10:22 AM, Caldarale, Charles R wrote:
>
> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
> Subject: Re: jvm exits without trace
>
> On 3/15/2010 10:22 AM, Caldarale, Charles R wrote:
> >> From: Taylan Develioglu [mailto:tdevelio...@ebuddy.com]
> >> Subject: Re: jvm exits without tra
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Chuck,
On 3/15/2010 10:22 AM, Caldarale, Charles R wrote:
>> From: Taylan Develioglu [mailto:tdevelio...@ebuddy.com]
>> Subject: Re: jvm exits without trace
>>
>> The cause for the crashes was in our own application code
> From: Taylan Develioglu [mailto:tdevelio...@ebuddy.com]
> Subject: Re: jvm exits without trace
>
> The cause for the crashes was in our own application code
Java or native code?
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is
> difficulties with the SSL implementation (somne browsers would work and some
> > wouldn't with seemingly the same setup.)
> >
> > Thanks,
> >
> > Carl
> > - Original Message -
> > From: "Taylan Develioglu"
> > To: "Tomc
27;t with seemingly the same setup.)
>
> Thanks,
>
> Carl
> - Original Message -
> From: "Taylan Develioglu"
> To: "Tomcat Users List"
> Sent: Thursday, March 11, 2010 6:13 AM
> Subject: Re: jvm exits without trace
>
>
> >a differe
same setup.)
Thanks,
Carl
- Original Message -
From: "Taylan Develioglu"
To: "Tomcat Users List"
Sent: Thursday, March 11, 2010 6:13 AM
Subject: Re: jvm exits without trace
a different kernel did not help either...
On Thu, 2010-03-11 at 11:37 +0100, Taylan Deve
gt;>>> I have used jdk 1.6.0_17 and 1.6.0_18 with the same results... have
> > > >>>> not
> > > >>>> tried 1.6.0_16. Please post your results of this trial.
> > > >>>>
> > > >>>>> Runni
>>>>
> > >>>>> Running tomcat on the
> > >>>>> foreground might show something, but then again I could be waiting
> > >>>>> for a
> > >>>>> month for it to happen.
> > >>>>
> > >
ou need to defined "serving 2172 clients" a bit more precisely
> before you can say this, no ?
>
>
> >
> > On Tue, 2010-03-09 at 16:40 +0100, Caldarale, Charles R wrote:
> >>> From: Taylan Develioglu [mailto:tdevelio...@ebuddy.com]
> >>> Subj
es R wrote:
From: Taylan Develioglu [mailto:tdevelio...@ebuddy.com]
Subject: RE: jvm exits without trace
where peak busy-threads used to be ~50 with APR, now it has become ~200
with JIO.
To be expected when you have unlimited keep-alives configured. Each HTTP
connection requires a separate threa
> From: Taylan Develioglu [mailto:tdevelio...@ebuddy.com]
> Subject: RE: jvm exits without trace
>
> If every connection requires its own thread there should be 2172
> threads.
Only if the client *chooses* to maintain the keep-alive. Browsers and other
clients are free t
evelio...@ebuddy.com]
> > Subject: RE: jvm exits without trace
> >
> > where peak busy-threads used to be ~50 with APR, now it has become ~200
> > with JIO.
>
> To be expected when you have unlimited keep-alives configured. Each HTTP
> connection requires a separa
> From: Taylan Develioglu [mailto:tdevelio...@ebuddy.com]
> Subject: RE: jvm exits without trace
>
> where peak busy-threads used to be ~50 with APR, now it has become ~200
> with JIO.
To be expected when you have unlimited keep-alives configured. Each HTTP
connection requ
rence (no executor is used):
- APR:
- JIO:
On Fri, 2010-03-05 at 19:13 +0100, Caldarale, Charles R wrote:
> > From: Christopher Schultz [mailto:ch...@christopherschultz.net]
> > Subject: Re: jvm exits without trace
> >
> > I thought he said he was using APR, n
> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
> Subject: Re: jvm exits without trace
>
> I thought he said he was using APR, not NIO.
He was, but IIRC, switched away from it to see if that would affect the
outages. What we don't know is what was switche
On 05/03/2010 17:21, Christopher Schultz wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Chuck,
On 3/5/2010 10:07 AM, Caldarale, Charles R wrote:
From: Christopher Schultz [mailto:ch...@christopherschultz.net]
Subject: Re: jvm exits without trace
Do you have a lot of static resources
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Chuck,
On 3/5/2010 10:07 AM, Caldarale, Charles R wrote:
>> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
>> Subject: Re: jvm exits without trace
>>
>> Do you have a lot of static resources being served?
> From: Christopher Schultz [mailto:ch...@christopherschultz.net]
> Subject: Re: jvm exits without trace
>
> Do you have a lot of static resources being served?
Or still allow keep-alives with the non-NIO connector?
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Taylan,
On 3/5/2010 4:45 AM, Taylan Develioglu wrote:
> It's performing rather poorly performance wise, compared to the apr
> connector. The number of threads required to handle the requests has
> gone up significantly over the board.
That's interest
it to happen.
> >>>>
> >>>> Yes, this has been part of my problem as anytime we change something, we
> >>>> have to wait a week for the server to fail.
> >>>>
> >>>> In one sense, I am fortunate that I have a little more flexibility than
>
lan Develioglu"
To: "Tomcat Users List"
Sent: Wednesday, February 24, 2010 8:31 AM
Subject: Re: jvm exits without trace
Hello Carl,
The failures we've seen are in anywhere between 8 hours to a week of
runtime. Most of them have (still) been running for almost a month
without fa
ibility than
> >> you.
> >> I have two servers (different hardware) but only need one in service at a
> >> time. Therefore, I always have one server I can test ideas on although I
> >> have never been able to develop a meaningful stress test, i.e., the only
&
quot;Taylan Develioglu"
To: "Tomcat Users List";
Sent: Wednesday, February 24, 2010 5:09 AM
Subject: Re: jvm exits without trace
The GC log shows plenty of heap space left in all the spaces.
I purposely didn't bother replacing the variables because I figured
they
would not
ugh I
> have never been able to develop a meaningful stress test, i.e., the only way
> I can test a change is to put it in production.
>
> Thanks,
>
> Carl
>
> - Original Message -
> From: "Taylan Develioglu"
> To: "Tomcat Users List"
Taylan Develioglu wrote:
for some reason I keep calling you chuck... I hope I'm not offending
anyone :O
We all do that from time to time.
It comes from the fact that we know he (Chuck) is always watching us
from the sidelines, to catch anyone who doesn't know his Servlet Specs
from his Socket
for some reason I keep calling you chuck... I hope I'm not offending
anyone :O
On Fri, 2010-02-26 at 13:55 +0100, Taylan Develioglu wrote:
> Chuck I am aware.
>
> A SIGSEGV is a signal sent by the kernel. Not a violation itsself.
>
> A sigsegv is sent when an invalid memory access is attempted b
Hi Chris,
There's no doubt about it. The amount free is what's left after
everything is taken into account, heap, jvm, jni, permgen.
And trust me I'd like it to be the oom killer, but it's not.
They could survive, but then I could throw away half of my ram. Not
seeing any point in doing that (d
Chuck I am aware.
A SIGSEGV is a signal sent by the kernel. Not a violation itsself.
A sigsegv is sent when an invalid memory access is attempted by a
process in userspace, in other words a page fault occurs, when the page
is actually present in physical memory but cannot be accessed by the
progr
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Taylann,
On 2/24/2010 8:31 AM, Taylan Develioglu wrote:
> Most memory is commited to tomcat, where a 24G machine would have 18G
> allocated to heap, 128M to permgen and some unspecified amount would get
> used by jni for apr. About 4G remains free aft
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Taylan,
On 2/24/2010 8:31 AM, Taylan Develioglu wrote:
> We have also had failures with hotspot error files (hs_err) present, and
> the cause specified was indeed SIGSEGV indicating a page fault. But I
> don't know if the two are related.
Just to be
I'll be sure to post an update if u16 resolves it. Or any other progress
for that matter.
In the meantime don't be shy either :)
On Wed, 2010-02-24 at 14:52 +0100, Carl wrote:
> Taylan,
>
> > The failures we've seen are in anywhere between 8 hours to a week of
> > runtime.
>
> The timing of the
It's possible,
I'm going to try an earlier jvm first. u16 was the previous one running
production, will try moving back to u16.
If that fails removing APR is the next thing to try out.
After that I'm going to try beating the dev team with a stick (I know
you're reading this!).
This is incredibl
develop a meaningful stress test, i.e., the only way
I can test a change is to put it in production.
Thanks,
Carl
- Original Message -
From: "Taylan Develioglu"
To: "Tomcat Users List"
Sent: Wednesday, February 24, 2010 8:31 AM
Subject: Re: jvm exits without trac
t; Thanks,
>
> Carl
>
> - Original Message -----
> From: "Taylan Develioglu"
> To: "Tomcat Users List" ;
> Sent: Wednesday, February 24, 2010 5:09 AM
> Subject: Re: jvm exits without trace
>
>
> > The GC log shows plenty of heap space left
On 24/02/2010 11:16, Taylan Develioglu wrote:
Thank you Konstantin, I've read the thread you mentioned.
I should have mentioned the mysterious exit happens on several different
servers with different hardware and configuration. So it's very unlikely
it's being caused by a hardware issue.
It's a
Carl wrote:
Taylan,
...
How does this compare to what you are experiencing?
Well, I note at least the crashes without traces in the logs, and the
common usage of SSL.
Is Taylan getting any Segfaults ?
-
To unsubscribe, e
y app is always used, the remaining
apps are used sporatically.)
How does this compare to what you are experiencing?
Thanks,
Carl
- Original Message -
From: "Taylan Develioglu"
To: "Tomcat Users List" ;
Sent: Wednesday, February 24, 2010 5:09 AM
Subject: Re: jvm e
Taylan Develioglu wrote:
Hi,
I have jvm's, running tomcat and our application, exiting mysteriously,
and was wondering if anyone could give me some advice on how to debug
this thing.
There is nothing in catalina.out, nor our application logs, and no
hotspot error file. GC log looks normal. No t
Thank you Konstantin, I've read the thread you mentioned.
I should have mentioned the mysterious exit happens on several different
servers with different hardware and configuration. So it's very unlikely
it's being caused by a hardware issue.
It's also not the oom killer as I mentioned before, I
2010/2/24 Taylan Develioglu :
> Hi,
>
> I have jvm's, running tomcat and our application, exiting mysteriously,
> and was wondering if anyone could give me some advice on how to debug
> this thing.
>
> There is nothing in catalina.out, nor our application logs, and no
> hotspot error file. GC log l
The GC log shows plenty of heap space left in all the spaces.
I purposely didn't bother replacing the variables because I figured they
would not be relevant.
But if you think they might provide clues they're as follows:
JAVA_HEAP_SIZE=18432M
JAVA_EDEN_SIZE=$(($(echo $JAVA_HEAP_SIZE|sed 's/M$\|G$
On 24/02/2010 09:36, Taylan Develioglu wrote:
I thought I'd add the connector definitions too, :
On Wed, 2010-02-24 at 10:23 +0100, Taylan Develioglu wrote:
Hi,
I have jvm's, running tomcat and our application, exiting mysteriously,
and was wondering if anyone could give me some
I thought I'd add the connector definitions too, :
On Wed, 2010-02-24 at 10:23 +0100, Taylan Develioglu wrote:
> Hi,
>
> I have jvm's, running tomcat and our application, exiting mysteriously,
> and was wondering if anyone could give me some advice on how to debug
> this thing.
>
> T
Hi,
I have jvm's, running tomcat and our application, exiting mysteriously,
and was wondering if anyone could give me some advice on how to debug
this thing.
There is nothing in catalina.out, nor our application logs, and no
hotspot error file. GC log looks normal. No trace in system logs.
I am
76 matches
Mail list logo
