Re: Pin-pointing the root of unusual application latencies

2007-07-26 Thread John Sigler
[ Adding linux-net to the mix ] John Sigler wrote: ( check_dektec_in-1095 |#0): new 271 us user-latency. ( check_dektec_in-1095 |#0): new 275 us user-latency. ( check_dektec_in-1095 |#0): new 290 us user-latency. ( check_dektec_in-1095 |#0): new 297 us user-latency. ( check_dektec_in-1095 |#0):

Re: Pin-pointing the root of unusual application latencies

2007-07-26 Thread John Sigler
John Sigler wrote: Len Brown wrote: John Sigler wrote: # cat /proc/interrupts CPU0 0: 37XT-PIC-XTtimer 1: 2XT-PIC-XTi8042 2: 0XT-PIC-XTcascade 7: 0XT-PIC-XTacpi 10:175XT-P

Re: Pin-pointing the root of unusual application latencies

2007-07-26 Thread John Sigler
John Sigler wrote: Len Brown wrote: John Sigler wrote: # cat /proc/interrupts CPU0 0: 37XT-PIC-XTtimer 1: 2XT-PIC-XTi8042 2: 0XT-PIC-XTcascade 7: 0XT-PIC-XTacpi 10:175XT-P

Re: Pin-pointing the root of unusual application latencies

2007-07-26 Thread John Sigler
Len Brown wrote: John Sigler wrote: # cat /proc/interrupts CPU0 0: 37XT-PIC-XTtimer 1: 2XT-PIC-XTi8042 2: 0XT-PIC-XTcascade 7: 0XT-PIC-XTacpi 10:175XT-PIC-XTeth2, Dt

Re: Pin-pointing the root of unusual application latencies

2007-07-25 Thread Len Brown
On Wednesday 25 July 2007 10:05, John Sigler wrote: > # cat /proc/interrupts > CPU0 >0: 37XT-PIC-XTtimer >1: 2XT-PIC-XTi8042 >2: 0XT-PIC-XTcascade >7: 0XT-PIC-XTacpi > 10:175

Re: Pin-pointing the root of unusual application latencies

2007-07-25 Thread Karsten Wiese
Am Mittwoch, 25. Juli 2007 schrieb John Sigler: > Karsten Wiese wrote: > > > John Sigler wrote: > > > >> Is there some form of priority inheritance? Does the IRQ handler get a > >> priority boost if a high priority task is waiting for it? > > > > No. But that would be "nice to have". > > No to

Re: Pin-pointing the root of unusual application latencies

2007-07-25 Thread John Sigler
Karsten Wiese wrote: John Sigler wrote: Is there some form of priority inheritance? Does the IRQ handler get a priority boost if a high priority task is waiting for it? No. But that would be "nice to have". No to the first question? to the second question? or to both? :-) In kernel/futex.

Re: Pin-pointing the root of unusual application latencies

2007-07-25 Thread Karsten Wiese
Am Mittwoch, 25. Juli 2007 schrieb John Sigler: > > Is there some form of priority inheritance? Does the IRQ handler get a > priority boost if a high priority task is waiting for it? No. But that would be "nice to have". Karsten - To unsubscribe from this list: send the line "unsubscribe

Re: Pin-pointing the root of unusual application latencies

2007-07-25 Thread John Sigler
Ingo Molnar wrote: John Sigler wrote: Ingo Molnar wrote: does your test-app have higher priority than softirq--4 ? PID 4 is [softirq-timer/0] and has priority 50 in SCHED_FIFO. My process has priority 80 in SCHED_RR. It is waiting for IRQ10. My user-space app has higher priority than ev

Re: Pin-pointing the root of unusual application latencies

2007-07-25 Thread Ingo Molnar
* John Sigler <[EMAIL PROTECTED]> wrote: > > does your test-app have higher priority than softirq--4 ? > > PID 4 is [softirq-timer/0] and has priority 50 in SCHED_FIFO. My > process has priority 80 in SCHED_RR. It is waiting for IRQ10. > > My user-space app has higher priority than everything

Re: Pin-pointing the root of unusual application latencies

2007-07-25 Thread John Sigler
(Dropping [EMAIL PROTECTED]) Ingo Molnar wrote: John Sigler wrote: With a pair of the following in the middle: softirq--4 0 670us : call_rcu (rt_run_flush) softirq--4 0D..1 670us : __rcu_advance_callbacks (call_rcu) Any idea what went wrong in the above function trace? Why is

Re: Pin-pointing the root of unusual application latencies

2007-07-25 Thread Ingo Molnar
* John Sigler <[EMAIL PROTECTED]> wrote: > >With a pair of the following in the middle: > > > >softirq--4 0 670us : call_rcu (rt_run_flush) > >softirq--4 0D..1 670us : __rcu_advance_callbacks (call_rcu) > Any idea what went wrong in the above function trace? Why is the > kernel sp

Re: Pin-pointing the root of unusual application latencies

2007-07-25 Thread John Sigler
John Sigler wrote: Ingo Molnar wrote: add 'notrace' to the definition of read_tsc in arch/i386/kernel/tsc.c ( check_dektec_in-1095 |#0): new 271 us user-latency. ( check_dektec_in-1095 |#0): new 275 us user-latency. ( check_dektec_in-1095 |#0): new 290 us user-latency. ( check_dektec_in-1095

Re: Pin-pointing the root of unusual application latencies

2007-07-24 Thread John Sigler
John Sigler wrote: ( check_dektec_in-1095 |#0): new 271 us user-latency. ( check_dektec_in-1095 |#0): new 275 us user-latency. ( check_dektec_in-1095 |#0): new 290 us user-latency. ( check_dektec_in-1095 |#0): new 297 us user-latency. ( check_dektec_in-1095 |#0): new 345 us user-latency. ( check

Re: Pin-pointing the root of unusual application latencies

2007-07-24 Thread John Sigler
Ingo Molnar wrote: add 'notrace' to the definition of read_tsc in arch/i386/kernel/tsc.c ( check_dektec_in-1095 |#0): new 271 us user-latency. ( check_dektec_in-1095 |#0): new 275 us user-latency. ( check_dektec_in-1095 |#0): new 290 us user-latency. ( check_dektec_in-1095 |#0): new 297 us use

Re: Pin-pointing the root of unusual application latencies

2007-07-23 Thread John Sigler
Ingo Molnar wrote: add 'notrace' to the definition of read_tsc in arch/i386/kernel/tsc.c OK. (or do echo 1 > /proc/sys/kernel/trace_use_raw_cycles if you are using recent enough -rt) Is patch-2.6.20-rt8 recent enough? # ./trace-it 1 >trace # cat trace preemption latency trace v1.1.5 on 2.

Re: Pin-pointing the root of unusual application latencies

2007-07-23 Thread Ingo Molnar
* John Sigler <[EMAIL PROTECTED]> wrote: > > ./trace-it 1 > trace.txt > > > > does it produce lots of trace entries? If not then > > CONFIG_FUNCTION_TRACING is not enabled. Once you see lots of output > > in the file, the tracer is up and running and you can start tracing > > the latency in yo

Re: Pin-pointing the root of unusual application latencies

2007-07-23 Thread John Sigler
Ingo Molnar wrote: John Sigler wrote: Here's a /proc/latency_trace dump. What is there to understand? # cat /proc/latency_trace preemption latency trace v1.1.5 on 2.6.20.7-rt8 latency: 26 us, #2/2, CPU#0 | (M:rt VP:0, KP:0,

Re: Pin-pointing the root of unusual application latencies

2007-07-23 Thread Ingo Molnar
* John Sigler <[EMAIL PROTECTED]> wrote: > Here's a /proc/latency_trace dump. What is there to understand? > > # cat /proc/latency_trace > preemption latency trace v1.1.5 on 2.6.20.7-rt8 > > latency: 26 us, #2/2, CPU#0 | (M:rt