Hello again.
First of all, thank you all for your replies. I've been working hard
for long days and nights before sending this question, but I guess I'm
still too newbie in the Linux kernel and specially with hardware
matters such as pci, network card specifications, bus latencies, and
so on.
First I will try to explain what I have to do. My job is to capture
packets in a SMP machine (call it "ksensor") as fast as I can, and
later process them. In theory, that machine will have to be analyzing
the whole traffic in a Gigabit ethernet link. In order to simulate
that heavy traffic I wanted to use another machine, the injector,
which has to send packets as fast as it can (that job is already done
by pktgen). But the problem has been that I couldnt send packets as
fast as it was needed to have losses in the "ksensor" machine.
So, after these all tests I'm thinking of using two injector machines
which will send to ksensor, in order to achieve an aggregated
throughput. By the way, if a switch is connected to all these
machines, there wouldn't be any collision, right? (Because the
communication is one way only, i think)
I'll try to describe the specifications of these machines:
KSENSOR (receiver)
-------------------------------------------------------------------------
Machine: Dual AMD Opteron 1800MHz
NIC: tg3 Broadcom BCM5704
PCI-X: 100MHz : 64-bit
#lspci -vvv -xxx -s 3
0000:02:03.0 Ethernet controller: Broadcom Corporation NetXtreme
BCM5704 Gigabit Ethernet (rev 02)
Subsystem: Advanced Micro Devices [AMD]: Unknown device 36c0
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR+ FastB2B-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium
>TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 64 (16000ns min), Cache Line Size: 0x10 (64 bytes)
Interrupt: pin A routed to IRQ 16
Region 0: Memory at fe8d0000 (64-bit, non-prefetchable) [size=64K]
Region 2: Memory at fe8c0000 (64-bit, non-prefetchable) [size=64K]
Expansion ROM at ff400000 [disabled] [size=64K]
Capabilities: [40] Capabilities: [48] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable+ DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
Capabilities: [58] Message Signalled Interrupts: 64bit+
Queue=0/3 Enable-
Address: 00000d5e09000204 Data: 3820
00: e4 14 48 16 06 01 b0 02 02 00 00 02 10 40 80 00
10: 04 00 8d fe 00 00 00 00 04 00 8c fe 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 22 10 c0 36
30: 00 00 8b fe 40 00 00 00 00 00 00 00 0a 01 40 00
40: 07 48 08 00 18 02 43 04 01 50 02 c0 00 21 00 64
50: 03 58 00 00 40 02 c5 10 05 00 86 00 04 02 00 09
60: 5e 0d 00 00 20 38 00 00 98 02 02 20 00 40 9f 76
70: e2 30 00 00 c6 00 00 80 4c 5b 03 00 00 00 00 00
80: 00 00 00 00 7e 95 ae bb 34 00 13 04 82 90 28 02
90: 09 97 00 01 00 00 00 e9 00 00 00 00 c3 01 00 00
a0: 00 00 00 00 fb 00 00 00 00 00 00 00 0e 00 00 00
b0: 00 00 00 00 00 00 00 0b 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
INJECTOR A
------------------------------------------------------
Pentium 4 2.80 GHz
NIC: e1000 Intel 82547GI
PCI: 33 MHz : 32-bit
INJECTOR B
------------------------------------------------------
Dual Pentium III 866Mhz
NIC: e1000 Intel 82540EM
PCI: 33 MHz : 32-bit
Now, some questions about what you told me:
>From Jesse:
> Also, you'll get a big improvement from running a UP kernel with
> packetgen.
Why? Pktgen threads can be bound to a specific processor. However, my
problem was the contrary. The Pentium 4 (single processor) couldnt
send faster than the old Pentium 3 (two processors - but just one used
with pktgen).
>From Jesse:
> reply with output of lspci -vvv -xxx -s <your ethernet adapter's slot>
> and ethtool -d ethX output and we can tell your bus speed from those.
I have copied the output of the tg3 NIC in the "ksensor" machine, but
can't find in it any clue it says: "PCI-X: 100MHz : 64-bit" (message
output in the NIC probe). I have also issued the "command ethtool -d
eth2" but it dumps a very large array of registers values which
personally cant understand, and the output is far too large to output
here.
>From Robert:
> Also make sure HW_FLOW control from the receiver is not throttling the
> sender
Well I'm not sure what hw_flow is. Is it similar to TCP congestion
control? I mean, if the sender is too fast for the receiver, it says
some way to stop it? What is the benefit of that? I mean, if the
packets are arriving so fast that the network card has to say "stop
it" probably the kernel wont be able to capture them, right?
By the way, as I said, the receiving machine has to analyze all the
traffic in a Gigabit link (it works much like an Intrusion Detection
System). So the normal thing would be just to listen the link. So, for
that matter the nic is put in promiscuous mode. I don't know if that
has to do anything with flow control (may be it's turned off when the
card is in promiscous mode).
Anyway, forgetting for a moment about this specific test I'm trying to
do, should it be for my application (something like an IDS) the
HW_FLOW turned on or off? (I guess it's "off", since the capture would
be pasive).
I hope the questions were clear enough. In a following email I will
send the ethtool statistics for each machine during a test. May be
that can help you to understand the "limiting factors" in my scenario.
Thank you
Regards
Aritz
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
