Alex Hi, Looks like in 2.4 your apps spent most of the time waiting for data (select) - and in 2.6 they spend more time doing I/O (writev). I assume this is network I/O - but it might be disk. If it's network, check for things that might worsen network I/O: MTU / Csum offloads / Net card driver + settings / epoll etc etc - you know the drill :)
--Aviv On Tue, Apr 27, 2010 at 06:17, Alexander Indenbaum <alexander.indenb...@gmail.com> wrote: > Hey, > > I'm playing with a Linux embedded box. Some time ago, we (finally) > switched from kernel 2.4.36 to 2.6.18.8. Naturally libc was also > upgraded. What have not changed was the sources of the applications: > Firefox 2.0 running on top of custom X server. GTK/pango/cairo/etc are > also identical in both systems. So far so good, yet we have found out > that the kernel/libc upgrade basically broke the customer's code: > CSS/Javascript timers driven application, running inside the browser, > which started to run much slower, from user experience point of view. > Unfortunately, the customer's user experience is extremely important > ;) > > Black box analysis showed differences in system behavior, see bellow. > Any idea about what else should be checked are greatly appreciated. > And yeah, the final goal is a happy customer. Please share you > thoughts - bounce some ideas. > > Thanks in advance, > ~baum > > > --------------------------------------------------------------------------------------------------------------------------------------------------------------------- > TOP > 2.4 The system CPU is 5% idle, utilization is between browser and X > server in 60%-40% ratio. > 2.6 The system CPU is 80% idle, utilization is between browser and X > server in 60%-40% ratio. > > --------------------------------------------------------------------------------------------------------------------------------------------------------------------- > STRACE (strace -c -p PID / during 100 secs). NOTE: In case of Firefox, > it is not a really apple to apple comparison, since only "active > thread" pid was straced. > 2.4 kernel - "FAST user experience" > X server > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > 90.70 15.259293 1045 14600 204 select > 4.55 0.765248 19 39678 11956 read > 3.59 0.604554 33 18350 2 writev > 0.52 0.087052 62 1415 munmap > 0.46 0.077625 3 29071 gettimeofday > 0.10 0.017304 3 4951 1821 sigreturn > 0.07 0.011463 8 1415 old_mmap > 0.01 0.002002 3 764 setitimer > ------ ----------- ----------- --------- --------- ---------------- > 100.00 16.824541 110244 13983 total > > Firefox browser > ------ ----------- ----------- --------- --------- ---------------- > 47.54 3.022460 1947 1552 select > 19.00 1.207781 272 4446 poll > 12.43 0.789926 41 19280 write > 8.32 0.529035 29 18218 37 readv > 7.37 0.468285 179 2618 2 writev > 1.51 0.095897 5 19428 1253 read > 1.20 0.076559 50 1544 old_mmap > 0.75 0.047951 7 6426 brk > 0.64 0.040452 9 4471 ioctl > 0.58 0.036982 24 1545 munmap > 0.57 0.035924 2 15927 gettimeofday > 0.09 0.005809 5 1156 kill > ------ ----------- ----------- --------- --------- ---------------- > 100.00 6.357061 96611 1292 total > > 2.6 kernel - "SLOW user experience" > X server > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > 55.00 0.003374 1 5287 writev > 35.18 0.002158 0 8580 2217 select > 9.81 0.000602 0 14486 6008 read > 0.00 0.000000 0 17114 gettimeofday > 0.00 0.000000 0 41 munmap > 0.00 0.000000 0 4454 setitimer > 0.00 0.000000 0 2543 2479 sigreturn > 0.00 0.000000 0 41 mmap2 > ------ ----------- ----------- --------- --------- ---------------- > 100.00 0.006134 52546 10704 total > > Firefox browser > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > 81.95 0.021973 3 7421 writev > 13.55 0.003634 0 13674 5906 read > 3.68 0.000986 0 13012 select > 0.51 0.000138 1 170 stat64 > 0.31 0.000082 1 118 futex > 0.00 0.000000 0 90 write > 0.00 0.000000 0 1789 gettimeofday > 0.00 0.000000 0 249 poll > ------ ----------- ----------- --------- --------- ---------------- > 100.00 0.026813 36523 5906 total > > _______________________________________________ > Linux-il mailing list > Linux-il@cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > _______________________________________________ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
