Robinson, Eric wrote:
Intuitively, I would have never thought that it was
/possible/ to run 160 or so instances of Tomcat on
anything but some type of very expensive hardware.
I assure you that it works admirably! And each of the instances runs a
sophisticated medical application with hundreds of JSPs and class files.
The data above, and your comments, seem to make unlikely
the OOM hypothesis, unless you have one application suddenly
acting up.
Also, we run a script that checks the logs for OOM errors and those are
not showing up.
On which of those servers is the one Tomcat instance dying ?
on both ?
Both occasionally. But now that we're digging into it, it has not
happened for several days on either one. :-/
Is that Tomcat instance running an application
which the other ones don't ?
They all run copies of the same application, albeit slightly different
versions. But the one that is shutting down is running the same version
as other instances that are not shutting down.
What kind of memory usage do the Tomcat instances have
(in total, including the Heap) ?
Depends on which server and when you check. Right now on app03:
[r...@app03 ~]# ps axo rss,vsz,comm,pid|grep java|sort -g
44984 251104 java 9151
47972 251876 java 10753
50816 253200 java 19077
59540 253160 java 17079
61780 254840 java 12740
73196 244908 java 26675
76580 252984 java 29065
78224 262196 java 981
89080 255308 java 17223
93968 255196 java 13046
98272 256228 java 17481
99440 256012 java 25020
101744 264704 java 9529
106160 258664 java 19425
108476 257452 java 26549
109712 266980 java 8853
113688 261296 java 14086
117596 270040 java 26159
118356 269264 java 8912
121316 270460 java 10800
123248 263196 java 5202
125764 257728 java 28540
126544 258128 java 7774
128084 267096 java 25420
128448 267420 java 13290
128508 259092 java 29286
129588 258800 java 21942
130068 265700 java 12465
130924 260472 java 29974
131484 268980 java 8812
132208 261088 java 7585
132560 268264 java 17343
133632 264320 java 16501
134232 261116 java 29183
135548 269132 java 10609
136932 273096 java 28742
140224 273052 java 12246
140624 274484 java 15351
141780 263020 java 12765
144948 264512 java 9292
148512 276672 java 9300
148900 267820 java 9547
150448 724588 java 12883
152224 270336 java 13083
153040 271260 java 9236
153300 293472 java 9634
153648 276708 java 31556
154324 272836 java 9041
154520 265512 java 9894
155176 272788 java 11453
155940 271532 java 10843
155940 274996 java 15897
156920 273980 java 8772
157240 271088 java 28906
157596 273128 java 16910
158852 275100 java 16096
160516 267404 java 8813
161060 279840 java 11403
161840 277244 java 11150
162176 274532 java 11876
162508 271484 java 9341
162672 274292 java 13402
162960 274220 java 10226
163460 279256 java 25700
163840 282796 java 9988
164248 277304 java 18237
164800 277576 java 26965
164928 278300 java 32556
164992 274972 java 11334
165284 282936 java 9587
165608 275548 java 6217
165992 277956 java 24426
166180 271832 java 18542
166548 274660 java 10662
167004 277668 java 10179
167064 284676 java 9386
167656 278904 java 12672
167712 278332 java 6112
167716 278484 java 715
168052 278188 java 3376
168224 274312 java 2910
168720 277072 java 32519
170104 274900 java 7194
170864 278304 java 16399
171036 275596 java 10375
171392 279968 java 12195
171884 281372 java 11207
172716 276524 java 23927
173192 279232 java 2755
173636 279772 java 11569
173688 278308 java 18800
174560 277796 java 10721
174660 279300 java 31576
175448 278176 java 12907
175784 287360 java 9198
176632 281812 java 15212
177588 293016 java 11750
178272 282860 java 19846
178704 280868 java 12023
179476 283092 java 28283
182580 283768 java 18964
183496 284244 java 15746
184928 282928 java 9942
185100 287952 java 30676
186124 288888 java 16051
186312 294096 java 9100
187360 305840 java 29874
190032 303316 java 11266
190616 292284 java 15054
191056 291612 java 10280
192256 291188 java 10098
192900 292848 java 10935
193132 287036 java 1986
196076 300276 java 10042
196460 302416 java 9967
197840 308108 java 14775
197912 291812 java 9854
198336 306508 java 1688
199552 311776 java 19234
200184 306876 java 14413
201132 309984 java 17799
204824 313368 java 13146
205136 310404 java 11031
206208 312032 java 18016
207116 310244 java 32263
207632 297772 java 15668
208404 305860 java 8734
208668 316744 java 9813
209960 311528 java 19346
210196 312944 java 14085
210248 310976 java 30073
212204 317592 java 30448
212256 316912 java 13032
212492 317780 java 10984
213128 309752 java 11801
214236 314068 java 12344
215596 320272 java 18188
216080 317312 java 17917
217160 311804 java 7073
217360 313656 java 20060
217856 320292 java 15051
218236 317512 java 14922
221620 317980 java 5873
221848 320096 java 12812
224772 323984 java 16675
227464 334344 java 16750
230412 322308 java 16218
235400 352016 java 1127
235516 339468 java 31681
238000 341372 java 2794
243416 361304 java 13202
252400 352100 java 19814
252836 350860 java 13547
261732 356168 java 12613
304756 393312 java 7747
364600 491072 java 5101 <--Here's the occasional bad boy
396240 503424 java 15556
399472 739912 java 30979
412684 522140 java 15828
601184 740096 java 6090
614564 733536 java 19163
622328 773108 java 21707
629036 750472 java 5503
And on app04...
[r...@app04 ~]# ps axo rss,vsz,comm,pid|grep java|sort -g
57724 254164 java 20550
58552 254648 java 20477
68976 704456 java 8949
69044 704520 java 8877
69368 704880 java 9357
70592 705560 java 8761
71428 704560 java 12263
71832 704604 java 16067
72832 705500 java 15760
73100 705500 java 15846
73116 706156 java 12142
73280 707036 java 11594
74484 704904 java 19899
75144 705796 java 8573
75148 705900 java 16335
76588 707580 java 18092
76612 707564 java 19510
76624 707560 java 19422
76828 707784 java 17730
76972 712048 java 10581
80896 710204 java 19822
88420 248424 java 20294
90164 715000 java 13688
90904 249056 java 31472
92492 247816 java 20225
95012 713452 java 9241
99796 254544 java 19955
100392 254008 java 27814
104668 722816 java 11063
110360 718232 java 11996
114592 724312 java 9296
116844 715536 java 8721
118700 712580 java 12803
119436 715904 java 10440
121788 254272 java 19272
125160 717432 java 11394
126572 719056 java 10959
128220 716864 java 19345
130120 736520 java 9457
130632 721272 java 9109
132704 716616 java 14898
132724 721124 java 12498
132928 721304 java 11155
134844 718220 java 8687
134976 718860 java 11834
136908 718156 java 13089
137764 718056 java 16815
137984 719912 java 18675
141164 725420 java 10019
141776 723256 java 11703
143368 722312 java 10628
143468 723848 java 12936
143708 723344 java 10912
144340 719792 java 9151
144892 714784 java 19712
146540 729248 java 11651
147972 717152 java 9733
148340 725520 java 11445
148564 723112 java 10201
150920 721160 java 9196
151932 729708 java 30865
153476 725488 java 10485
153872 727864 java 15905
153984 723652 java 17409
155172 717676 java 15167
155316 725368 java 11108
156276 715884 java 14742
159352 723208 java 10675
161060 727472 java 9968
162728 725000 java 10355
163572 726664 java 11010
163584 725524 java 10812
164540 728712 java 11349
166196 723012 java 8649
166336 724284 java 9780
167164 725612 java 10393
170228 740788 java 10156
172012 720144 java 10064
174304 716784 java 17273
174532 732936 java 11253
175540 725636 java 11501
180384 731224 java 11198
182580 739276 java 9641
182664 725020 java 15477
188560 722140 java 13359
191072 725832 java 16182
191644 723472 java 9419
192040 725712 java 18245
192296 729044 java 9867
192312 721156 java 13568
193108 722264 java 10859
195128 728516 java 15115
196988 728348 java 17919
197176 728348 java 9021
198208 732976 java 10760
200680 730128 java 8915
202100 727268 java 17611
203736 729252 java 13433
204516 727428 java 17157
210064 726256 java 8835
210200 720956 java 16539
212644 728452 java 15251
215072 729040 java 18326
215308 731744 java 18384
215824 729712 java 13303
215956 737092 java 9688
216944 734224 java 9825
217508 728696 java 9554
217640 723284 java 8611
223764 726140 java 12580
227700 734256 java 10111
228372 333316 java 13733
228704 728684 java 15309
230384 739248 java 11543
234760 730616 java 17809
234944 737760 java 31312
235964 734200 java 17488
236220 730264 java 13211
236920 729784 java 12496
237044 727368 java 15593
237088 725664 java 19162
238108 738512 java 9923
238968 729736 java 16923
239792 733860 java 16761
240952 749396 java 14819
242188 719332 java 19605
245164 726040 java 18429
246656 724124 java 12351
255040 726388 java 16451
255472 730544 java 18606
260656 736716 java 18187
261236 724660 java 14605
262044 725720 java 19556
263172 729056 java 486
270464 728988 java 18523
273688 732928 java 19121
277432 729352 java 18470
283376 763044 java 17050
285500 727892 java 18867
291040 732404 java 12670
294008 731052 java 12882
301480 735372 java 13628
304224 725820 java 12760
309376 726396 java 18933
311780 735440 java 9502
313860 742148 java 10297
319984 731772 java 18731
344632 734584 java 14528
353688 730104 java 13483
374108 733288 java 17265
390840 729440 java 12978
404984 727700 java 11883
432752 766520 java 10537
497380 743672 java 14219
531704 744664 java 14656
534164 729548 java 18014
539992 736864 java 9597
542228 733612 java 19211
575672 734008 java 15038
604780 746120 java 31131 <--Here's the occasional bad boy.
604892 751856 java 10243
605168 751156 java 15380
605736 737988 java 16616
609016 731540 java 19261
Do you start them with any kind of other memory-related
parameter for the JVM apart from -Xms and -Xmx ?
Nope!
> 364600 491072 java 5101 <--Here's the occasional bad boy
> 604780 746120 java 31131 <--Here's the occasional bad boy.
Those sizes appear to be in Kbyte, so we are talking in one case of a process with a
resident size of 360 MB, in the other of 600 MB.
I guess thus, that these are not instances of JVM started with -Xms64M and
-Xmx64M ?
Mmm. So these instances are running the same application and version of ditto, on
different hosts, and they both crash sometimes.
But the same application/version is running in other instances, which do not
crash.
Are the ones that crash used by the same users on both instances ? and are these users
different ones than the ones using the other instances of the app that never crash ?
Just trying to figure out what unites them, and differenciates them from the
others..
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
