I have seen OOM issue 6.0.24-90. After downgrading the tomcat to 6.0.24-78, the same environment does not show OOM anymore.
Thanks Sudhansu On Fri, Oct 16, 2015 at 10:58 AM, ilya <ilya.mailing.li...@gmail.com> wrote: > Out of curiosity, i decide to run ab on one of the smalller lab > environments that had older tomcat 6.0.24 and ACS 4.5.2+ , i was able to > compete all 100,000 requests in 15 or so minutes... > > > > # ab -n 100000 -c 1 http://localhost:8080/client/ > This is ApacheBench, Version 2.3 <$Revision: 655654 $> > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > Licensed to The Apache Software Foundation, http://www.apache.org/ > > Benchmarking localhost (be patient) > > > Completed 10000 requests > Completed 20000 requests > Completed 30000 requests > Completed 40000 requests > Completed 50000 requests > Completed 60000 requests > Completed 70000 requests > Completed 80000 requests > Completed 90000 requests > Completed 100000 requests > Finished 100000 requests > > > Server Software: Apache-Coyote/1.1 > Server Hostname: localhost > Server Port: 8080 > > Document Path: /client/ > Document Length: 221754 bytes > > Concurrency Level: 1 > Time taken for tests: 1837.645 seconds > Complete requests: 100000 > Failed requests: 0 > Write errors: 0 > Total transferred: 22196600000 bytes > HTML transferred: 22175400000 bytes > Requests per second: 54.42 [#/sec] (mean) > Time per request: 18.376 [ms] (mean) > Time per request: 18.376 [ms] (mean, across all concurrent requests) > Transfer rate: 11795.73 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 0 0.0 0 6 > Processing: 3 18 65.5 5 13705 > Waiting: 0 2 23.4 2 7414 > Total: 3 18 65.5 5 13706 > > Percentage of the requests served within a certain time (ms) > 50% 5 > 66% 6 > 75% 7 > 80% 7 > 90% 10 > 95% 206 > 98% 206 > 99% 207 > 100% 13706 (longest request) > > > No crash of CloudStack has been observed. > > > My ulimit output in the lab env.. > > # ulimit -a > core file size (blocks, -c) 0 > data seg size (kbytes, -d) unlimited > scheduling priority (-e) 0 > file size (blocks, -f) unlimited > pending signals (-i) 30488 > max locked memory (kbytes, -l) 64 > max memory size (kbytes, -m) unlimited > open files (-n) 1024 > pipe size (512 bytes, -p) 8 > POSIX message queues (bytes, -q) 819200 > real-time priority (-r) 0 > stack size (kbytes, -s) 8192 > cpu time (seconds, -t) unlimited > max user processes (-u) 30488 > virtual memory (kbytes, -v) unlimited > file locks (-x) unlimited > > > On 10/15/15 9:31 PM, Carlos Reátegui wrote: > > What all happened with the work to embed tomcat or jetty? Seems like > that could help with these issues. > > > >> On Oct 15, 2015, at 9:23 PM, ilya <ilya.mailing.li...@gmail.com> wrote: > >> > >> The stock tomcat 6.0.24 is poorly maintained by CentOS/RedHat. > >> > >> We've seen several issues with latest 6.0.24.x tomcat builds. > >> > >> We also found tomcat6-6.0.43+ to be alot more stable. > >> > >> Though its not in public CentOS repos, Marcus has a build that works > >> well and has been tested in very large environments with uptime of 30+ > >> days. (30+ days cycle is broken due to scheduled maintenance - not > crash).. > >> > >> > >> Try tomcat from repo below, > >> http://marcus.mlsorensen.com/cloudstack-extras/tomcat6/ > >> > >> let me know how it works.. > >> > >> > >>> On 10/15/15 1:32 AM, windyii wrote: > >>> Hi all, > >>> > >>> > >>> > >>> Our CloudStack 4.5.2 with tomcat6 was constantly running out of memory > in a > >>> few days. > >>> > >>> We changed JAVA_OPS to "-Xmx4g" in tomcat6.conf. But it didn't help. > >>> > >>> We used apache benchmark to send 100000 http requests to a fresh > installed > >>> CloudStack 4.5.2 with no zone setup. > >>> > >>> ab -n 100000 -c 1 http://localhost:8080/client/ > >>> > >>> The CS always run out of memory after 35,000 requests. > >>> > >>> The same to a fresh CloudStack 4.3.0. > >>> > >>> A clean tomcat6 on another CentOS host passed the ab test. > >>> > >>> Finally, we installed tomcat7 and change CS to use tomcat7. Both CS > 4.5.2 > >>> and CS 4.3.0 passed ab test. > >>> > >>> We suppose it is a serious issue. > >>> > >>> Is there any idea? > >>> > >>> > >>> > >>> Our setup: > >>> > >>> CS 4.5.2/Centos 6.5/Tomcat 6.0.24 > >>> > >>> Tomcat 7.0.33 > >>> > >>> > >>> > >>> -- > >>> > >>> Qian > >>> > >>> > >>> > >>> >
