> On 19 Dec 2024, at 02:17, Maurice Janssen <maur...@z74.net> wrote:
>
> On Wed, Dec 18, 2024 at 11:51:26AM +1000, David Gwynne wrote:
>> Hey Maurice,
>>
>> bluhm@ just did a talk at eurobsdcon that might help you understand the
>> different points to look at. my tldr version is if packets are being
>> dropped on the firewall it will be somewhere in the receive path. my
>> experience is that once a packet has made it into the network stack
>> it tends to get transmitted fine.
>
> Thanks for your reply. This has given me some new things to look at.
>
>> the places that packets can get dropped are in:
>>
>> 1. hardware
>>
>> hardware counters from igc should be visible using kstat(1) now. the
>> igc-stats that you're looking at would be any of the errs/colls, or the
>> rx no bufs counter.
>
> kstat(1) shows me that there are (nearly) no errors on the rx side, but
> showed about 470k dropped packets (qdrops) on the tx side for igc2.
wait, what?
do you have PCIe ASPM enabled in the bios?
>
>> 2. rxq drops and errors
>>
>> the rxq is where packets received by the driver are put before being
>> handed over to the network stack. again, these are visible as kstats,
>> but are also aggregated as interface level counters you can see with
>> netstat -i. you can look at drops and errors individiually with netstat
>> -id and netstat -ie respectively.
>
> netstat -id shows all zeros in de Idrop column, but the same 470k packets
> in the Odrop column.
> Not sure if this can also be caused by unplugging the cable. I did that
> sereral times during testing/swapping switches on the igc2 port.
> I will reboot to reset the counters and keep an eye on it.
>
>> the fact that you're using pppoe on top of a vlan adds some potential
>> for loss too. there's a queue in between ethernet input processing and
>> pppoe processing that could be dropping packets. i don't think we have a
>> way to see drops on that queue though.
>>
>> if you're feeling brave you can try the diff i sent to the 'PPPoE
>> passthrough with "GigaHub" is very slow' thread on misc@ a week or so
>> ago which skips the queue for pppoe data packets.
>
> I think I can give it a try in a week or two.
> For what it's worth: I see only 138 qdrops for pppoe0:0:txq:0 in kstat
> output. And no drops on the rxq, but all counters are at zero so I guess
> that doesn't say much.
>
>> 3. pf drops
>>
>> i think systat pf and pfctl -si are the best way to see counters here.
>> however, i feel like there's some things that pf does and doesn't do
>> sometimes that aren't captured well.
>>
>> 4. tx drops
>>
>> again, i think this is unlikely, but you should be able to see tx
>> interface drops using the txq kstats or drops/errors in netstat -i
>> output. there's tx counters in the igc-stats too.
>
> Furthermore, netstat -ss -p udp shows me:
> udp:
> 295407643 datagrams received
> 1 with bad data length field
> 2100 with bad checksum
> 592550 with no checksum
> 289009064 input packets software-checksummed
> 318512247 output packets software-checksummed
> 25112 dropped due to no socket
> 239806 broadcast/multicast datagrams dropped due to no socket
> 431246 dropped due to full socket buffers
> 294709378 delivered
> 348403092 datagrams output
> 295132792 missed PCB cache
>
> But I don't know if there dropped packets are also on igc2.
>
> Thanks again and best regards,
> Maurice
