I sent this message yesterday just before leaving for the day and I see it was quarantined for "too big". So I'm trying again after removing the quoted original message. Thanks for the input. Sorry about taking so long to reply. Here's some vmstat output. I think the prstat was taken during the vmstat run (prstat and vmstat do not have timestamps so I'm only guessing). (I collected the output of prstat -amL, vmstat 15 and iostat -cxnzTd 15 by starting three PuTTY sessions with PuTTY logging each session. When I went about my work and after the user training session that stressed the server I went back and looked through the log files.) I did not run mpstat.
I shifted the data right to get the headings and data to line up. I find it hard to follow when everything is shifted left. kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr m1 m2 m4 m5 in sy cs us sy id 0 0 0 7819744 375848 122 272 39 3 3 0 0 0 0 0 0 759 6535 2894 32 2 66 0 0 0 7823704 378896 223 896 3 3 3 0 0 0 0 0 0 727 7130 2873 34 4 63 0 0 0 7811312 381656 423 1673 27 15 15 0 0 0 0 1 1 934 14244 3556 39 5 56 0 0 0 7801536 378520 386 895 13 2 2 0 0 0 0 0 0 865 7227 3102 48 3 49 0 0 0 7828120 378152 168 499 19 4 4 0 0 0 0 0 0 822 7681 2948 44 2 54 0 0 0 7828664 379760 143 337 1 3 3 0 0 0 0 0 0 713 6562 2675 29 5 65 0 0 0 7827704 377000 208 637 1 3 3 0 0 0 0 0 0 939 9791 3000 50 3 47 0 0 0 7818504 373800 272 1249 4 2 2 0 0 0 0 0 0 1240 9695 3039 62 6 33 1 0 0 7802360 368136 178 775 31 6 6 0 0 0 0 16 15 1649 12336 3467 91 4 6 16 0 0 7795552 363872 265 1047 44 3 3 0 0 0 0 10 10 2545 11871 4739 95 5 0 12 0 0 7767856 331728 182 1466 36 4 4 0 0 0 0 0 0 2603 18001 6178 94 6 0 15 0 0 7691616 276584 453 2377 631 6 6 0 0 0 0 1 1 3156 16782 7217 91 9 0 16 0 0 7678016 252064 315 1567 292 2 2 0 0 0 0 1 1 2959 14525 6753 94 6 0 16 0 0 7670472 247104 299 1193 125 2 2 0 0 0 0 0 0 2440 15443 6079 94 6 0 15 0 0 7605832 195504 213 1384 154 14 14 23272 0 0 0 0 0 2769 18587 7330 93 7 0 18 0 0 7538040 154624 369 1807 220 20 20 44904 0 0 0 1 1 3250 21050 8026 91 9 0 15 0 0 7564584 142944 471 2066 1049 72 208 101672 145 1 1 1 1 2979 24819 7354 92 8 0 12 0 0 7574456 155400 340 1459 383 10 10 20960 0 0 0 12 12 2486 22536 5526 94 6 0 10 0 0 7571840 151048 622 2759 1268 17 17 112968 0 0 0 18 18 2828 27129 5398 91 8 0 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr m1 m2 m4 m5 in sy cs us sy id 1 0 0 7561528 142480 178 606 218 1 1 113888 0 0 0 0 0 1335 12978 3616 74 3 23 7 0 0 7548384 136376 268 1111 209 6 6 74136 0 0 0 0 0 2378 12732 5157 94 5 1 9 0 0 7540216 133608 186 1132 104 365 886 81640 560 3 3 0 0 1572 15904 3583 92 5 3 2 0 0 7534872 133144 271 1673 205 1046 2688 20536 1744 8 8 2 2 1334 15872 3346 81 6 13 0 0 0 7529464 158584 268 926 284 4 4 65160 0 0 0 0 0 1217 14766 2927 71 4 25 4 0 0 7552976 169576 291 1146 153 11 11 39416 0 0 0 0 0 1461 11725 3522 75 4 20 6 0 0 7521696 161456 151 556 62 3 3 8136 0 0 0 0 0 1647 9467 3477 97 3 0 17 0 0 7521880 134808 203 1368 168 225 504 2720 294 2 2 0 0 1595 12081 3925 94 6 0 16 0 0 7495080 130056 241 978 183 170 429 24336 287 1 1 0 0 1945 10023 4445 90 10 0 15 0 0 7484192 128064 194 953 579 762 1909 5624 1224 6 6 0 0 2305 12226 6164 93 7 0 13 0 0 7460704 126728 198 817 103 633 1463 1136 905 4 4 0 0 2093 9639 4650 95 5 0 17 0 0 7423448 126672 150 1059 269 1320 3053 1672 1889 10 10 0 0 2238 12118 5042 93 7 0 17 0 0 7336400 129920 351 1878 475 1226 2672 98288 1581 8 8 25 25 3193 17806 6979 91 9 0 22 0 0 7346864 132376 293 1509 449 726 1525 74136 942 6 6 1 1 2792 13112 6719 94 6 0 16 0 0 7343464 131280 79 248 27 12 12 15288 0 0 0 3 3 2934 14676 7219 96 4 0 12 0 0 7320840 127888 227 1070 220 708 1284 3744 686 5 5 0 0 4229 19397 10198 93 7 0 12 0 0 7320496 131848 209 1014 257 647 906 41472 358 5 5 1 1 4674 20462 10408 95 5 0 15 0 0 7276704 131672 473 1718 917 949 1313 87616 419 7 7 1 1 2446 20843 5275 95 5 0 12 0 0 7266352 138888 321 1758 99 104 127 18072 30 1 1 1 1 2273 14143 5193 95 5 0 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr m1 m2 m4 m5 in sy cs us sy id 14 0 0 7246352 139288 427 1401 1408 396 600 3744 248 3 3 24 24 3432 20015 5511 95 5 0 14 0 0 7265656 139600 332 1337 400 14 14 512 0 0 0 4 4 3423 22849 7321 95 5 0 12 0 0 7265880 130840 189 696 148 430 581 0 170 3 3 0 0 2973 21186 6491 95 5 0 13 0 0 7255296 132264 260 1509 281 895 1253 101672 445 7 7 0 0 3262 14583 7395 93 7 0 9 0 0 7258072 141904 234 962 143 10 10 20960 0 0 0 2 2 3108 18407 7125 94 6 0 10 0 0 7248672 133536 274 1478 177 138 271 66728 188 1 1 0 0 2122 10229 4711 95 5 0 13 0 0 7190728 134784 432 1700 258 991 1450 43792 631 8 8 0 0 1629 14228 3097 94 6 0 9 0 0 7235272 141784 171 704 107 0 0 64240 0 0 0 0 0 1542 7156 3194 97 3 0 12 0 0 7231544 137520 222 889 243 12 12 13248 0 0 0 0 0 1859 13256 3930 96 4 0 13 0 0 7210568 128664 151 553 351 374 504 2744 143 3 3 0 0 1764 10405 3382 95 5 0 17 0 0 7197888 129352 172 843 351 1945 2795 47944 1083 17 17 0 0 1949 12543 3819 95 5 0 12 0 0 7169952 132056 164 652 145 282 381 9896 116 3 3 0 0 1444 6882 3149 93 7 0 15 0 0 7174416 130832 149 826 229 398 546 48656 172 4 4 0 0 1341 7721 2613 86 14 0 18 0 0 7151840 135128 436 2256 784 859 1240 112968 467 7 7 0 0 2234 11054 3334 94 6 0 15 0 0 7122744 138800 318 1582 366 7 7 99144 0 0 0 26 26 2099 10231 3297 96 4 0 9 0 0 7111776 128696 152 540 124 62 78 20440 19 1 1 0 0 1313 5285 2502 98 2 0 12 0 0 7141040 133952 184 818 123 7 7 42424 0 1 1 0 0 1493 10212 2704 96 4 0 9 0 0 7142384 131144 261 974 161 144 250 11152 141 3 3 1 1 1298 10488 2580 97 3 0 12 0 0 7131424 130744 429 1497 660 666 965 2328 386 5 5 2 2 1257 18226 2513 97 3 0 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr m1 m2 m4 m5 in sy cs us sy id 11 0 0 7132360 138152 309 1681 272 353 507 79904 177 3 3 0 0 1419 12346 2563 97 3 0 13 0 0 7108304 137736 469 1875 1602 1540 2039 91512 829 14 14 24 24 2164 16437 3079 95 5 0 14 0 0 7043488 129400 327 1356 628 2064 2765 18864 1480 16 16 2 2 1930 19646 3531 96 4 0 11 0 0 7110984 168768 143 342 365 10 10 3912 0 1 1 0 0 1543 7776 2719 98 2 0 15 0 0 7132096 185200 382 1568 223 5 5 23736 0 3 3 0 0 1501 10946 2849 95 5 0 12 0 0 7104928 168848 269 1371 140 4 4 76848 0 3 3 2 2 1264 10605 2638 96 4 0 9 0 0 7102328 153704 255 1319 244 1062 1382 66728 853 9 9 0 0 1339 8783 2614 97 3 0 9 0 0 7051224 135160 206 658 276 628 867 39416 629 5 5 0 0 1263 6279 2479 97 3 0 12 0 0 7072752 153256 284 1093 166 1 1 8136 0 0 0 0 0 1551 6539 2634 97 3 0 11 0 0 6981600 134416 282 1351 270 739 975 54056 649 6 6 0 0 1557 9083 2822 96 4 0 10 0 0 7023184 144672 195 772 169 3 3 47304 0 1 1 0 0 1340 5813 2581 97 3 0 10 0 0 7044784 144992 108 413 95 3 3 9768 0 2 2 0 0 1192 5469 2444 97 3 0 9 0 0 7053648 135896 155 871 187 1518 2086 98328 1059 12 12 0 0 1295 7452 2517 97 3 0 10 0 0 7007312 130696 150 654 207 635 798 20272 287 5 5 0 0 1267 7893 2573 97 3 0 15 0 0 6995696 133912 306 1828 490 2460 3611 23112 1509 19 19 0 0 1465 7475 2421 80 20 0 19 0 0 6937360 132536 296 1306 822 223 268 4792 67 2 2 26 26 2192 10807 3318 79 21 0 10 0 0 6948448 146544 268 934 278 1091 1448 888 497 10 10 0 0 1502 11974 2642 97 3 0 10 0 0 6954808 169992 284 996 193 3 3 17536 0 3 4 0 0 1391 8105 2719 97 3 0 12 0 0 6955008 152272 358 1538 277 14 14 54912 0 9 9 1 1 1327 12667 2561 96 4 0 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr m1 m2 m4 m5 in sy cs us sy id 16 0 0 6974384 171000 358 1653 863 3 3 43792 0 6 6 1 1 1604 15035 2858 96 4 0 9 0 0 6953464 164224 636 2482 426 70 36 35480 0 6 6 2 2 1481 15594 2776 95 5 0 7 0 0 6943680 141896 417 1545 1011 19 19 74136 0 1 1 25 25 1799 15960 3060 95 5 0 10 0 0 6942704 152752 482 1747 365 3 3 15288 0 3 3 5 5 1593 16708 2898 96 4 0 7 0 0 6909024 135816 113 794 151 801 1070 35480 381 8 8 0 0 1410 5645 2549 98 2 0 10 0 0 6872464 129728 337 1408 376 538 678 7680 199 13 13 0 0 1972 13532 3573 94 6 0 7 0 0 6882896 147328 318 1335 182 17 17 1608 0 4 5 1 2 1627 18924 3524 94 6 0 4 0 0 6897304 150328 478 1338 406 6 6 66728 0 6 6 0 0 1775 12833 3151 95 5 0 5 0 0 6903528 150560 372 1301 134 4 4 66728 0 5 5 0 0 1597 10876 2901 97 3 0 4 0 0 6903144 147936 253 625 21 53 49 13760 0 0 0 3 3 1235 5613 2619 98 2 0 2 0 0 6895040 139128 303 838 186 60 18 2864 0 1 1 0 0 1465 9857 2954 97 3 0 1 0 0 6894600 136544 379 880 168 2 2 96 0 4 4 0 0 1411 6755 2837 97 3 0 2 0 0 6894272 134864 373 772 213 40 4 0 0 4 4 0 0 1511 7745 2828 93 6 1 0 0 0 6903768 139416 142 366 220 4 3 0 0 5 5 0 0 1139 5535 2852 65 2 32 0 0 0 6951080 162816 283 944 259 9 9 50024 0 6 6 0 0 1201 9828 3122 57 4 39 0 0 0 6959656 159304 244 1358 618 3 3 51568 0 2 2 8 8 1280 7319 3035 55 4 41 0 0 0 7130248 286872 272 708 57 5 5 10640 0 1 1 19 18 1214 13315 2912 56 3 41 0 0 0 7202856 350624 69 322 55 8 8 2224 0 2 2 0 0 1060 5331 2669 54 3 43 0 0 0 7798960 952112 293 1068 201 4 4 0 0 11 11 0 0 1098 13879 2654 64 13 24 r maxes out at 22. The server is a V490 with one CPU board (2 UltraSPARC IV's @ 1.35 GHz) with 8 GB of memory and two 2Gb FC connections to a 6130. Solaris is 9 4/04. The users are not impressed with the application response and (more importantly) IT (my) management would like to do something about that. Also, this is a new server and a new application. (Solaris/Unix is new to us. The server is one of two racks of Sun servers supporting our move to an ERP. This is one of the big servers. ) We are just starting to roll out some production reports on this server. The reports have been developed in-house and the staff feel that they can tweak their design to improve performance (and I agree). Management is a little impatience and looking for other options. We have a not yet used V490 and we could move it's CPU board to this server, as a test. It is easy to get the developers to run some tests but not easy to get enough to really stress it. Thanks again for the help and sorry about taking so long to reply, Glen Gunselman Systems Software Specialist TCS Emporia State University
_______________________________________________ perf-discuss mailing list perf-discuss@opensolaris.org
