date:20160429

Re: netmap overrun counters

2016-04-29 Thread Luigi Rizzo

On Thu, Apr 28, 2016 at 02:53:25PM -0700, bazzoola wrote:
> Thanks Adrian, and thanks Luigi for the explanation:
> 
> On 04/28/2016 01:15 PM, Luigi Rizzo wrote:
> > 
> > please re-read the relevant part of the manual page:
> > 
> >RECEIVE RINGS
> >  On receive rings, after a netmap system call, the slots in the range
> >  head... tail-1 contain received packets.  User code should process them
> >  and advance head and cur past slots it wants to return to the kernel.
> >  cur may be moved further ahead if the user code wants to wait for more
> >  packets without returning all the previous slots to the kernel.
> > 
> >  At the next NIOCRXSYNC/select()/poll(), slots up to head-1 are returned
> >  to the kernel for further receives, and tail may advance to report new
> >  incoming packets.
> >  Below is an example of the evolution of an RX ring:
> > 
> >  after the syscall, there are some (h)eld and some (R)eceived slots
> > head  cur tail
> >  | |   |
> >  v v   v
> >   RX  [..hh..]
> > 
> >  user advances head and cur, releasing some slots and holding others
> > head cur  tail
> >   |  | |
> >   v  v v
> >   RX  [..*hhhRR...]
> > 
> >  NICRXSYNC/poll()/select() recovers slots and reports new packets
> > head curtail
> >   |  |   |
> >   v  v   v
> >   RX  [...hhh]
> > 
> > 
> > tail advances if there are new packets _and_ can at most go one
> > slot before head. At that point the buffer is full and the NIC
> > starts dropping packets until your application consumes packets,
> > advance???s head+cur and makes room so that the NIC can copy new
> > 
> > packets to the buffers and the driver advances tail
> > 
> > Basically, all I am trying to do is detect if frames are dropped in my
> > application using netmap API.
> > 
> 
> I am looking at
> https://www.freebsd.org/cgi/man.cgi?query=netmap&manpath=FreeBSD+11-current
> 
> "Passing the NETMAP_DO_RX_POLL flag to NIOCREGIF updates receive rings
> even without read events"
> 
> This means that even if I don't update cur/head pointers in my
> application then netmap will keep updating its rings. Is statement
> correct? If yes, how is this useful if tail doesn't increment.

your interpretation is not correct. Update the ring means that tai
advances only up to the available space. If you don't update head/cur,
then when the ring is full tail will stop there and the NIC will start
dropping packets.

> 
> > 
> > ???wrong model :)
> > netmap per se never drops packets because it does not run code.
> > 
> > If your application does not read fast enough it is the NIC that
> > drops packets, and counts them as overrun; netmap cannot know
> > how many of them.
> 
> I have a simple test (without NETMAP_DO_RX_POLL set), I send UDP packets
> with a known counter and monitor that counter in my netmap application.
> If there is a mismatch I know a packet was dropped.
> 
> I also monitor sysctl em.0.overrun before and after I run the program
> and it stays 0.
> 
> After around 20 seconds of of capturing frames (and storing them in
> memory), Swap kicks in (my program starts paging) and I detect the 1st
> dropped packet using my application. However, sysctl *overrun for em
> never reports drops. Should I look at a different stat?

I have no idea. This is NIC specific, there may be other stats that
report the queue overflows.

cheers
luigi
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] D6120: tcp/syncache: Set flowid and hash type properly for SYN|ACK

2016-04-29 Thread hselasky (Hans Petter Selasky)

hselasky added a comment.


  Basically what you're doing is to loop back the flowid of the received packet 
- right?
  
  Maybe you could put that in a code comment.

REVISION DETAIL
  https://reviews.freebsd.org/D6120

EMAIL PREFERENCES
  https://reviews.freebsd.org/settings/panel/emailpreferences/

To: sepherosa_gmail.com, adrian, rwatson, gnn, lstewart, glebius, delphij, 
mike-karels.net, jtl, network, transport, hiren, sbruno
Cc: hselasky, freebsd-net-list
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] D6120: tcp/syncache: Set flowid and hash type properly for SYN|ACK

2016-04-29 Thread sepherosa_gmail.com (Sepherosa Ziehau)

sepherosa_gmail.com added a comment.


  In https://reviews.freebsd.org/D6120#130844, @hselasky wrote:
  
  > Basically what you're doing is to loop back the flowid of the received 
packet - right?
  >
  > Maybe you could put that in a code comment.
  
  
  Yep, since we don't have inp to loopback the SYN's flowid here.  So we use 
the SYN's flowid directly.  OK, will add.

REVISION DETAIL
  https://reviews.freebsd.org/D6120

EMAIL PREFERENCES
  https://reviews.freebsd.org/settings/panel/emailpreferences/

To: sepherosa_gmail.com, adrian, rwatson, gnn, lstewart, glebius, delphij, 
mike-karels.net, jtl, network, transport, hiren, sbruno
Cc: hselasky, freebsd-net-list
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] D6120: tcp/syncache: Set flowid and hash type properly for SYN|ACK

2016-04-29 Thread Phabricator

This revision was automatically updated to reflect the committed changes.
Closed by commit rS298769: tcp/syncache: Set flowid and hash type properly for 
SYN|ACK (authored by sephe).

CHANGED PRIOR TO COMMIT
  https://reviews.freebsd.org/D6120?vs=15647&id=15727#toc

REPOSITORY
  rS FreeBSD src repository

CHANGES SINCE LAST UPDATE
  https://reviews.freebsd.org/D6120?vs=15647&id=15727

REVISION DETAIL
  https://reviews.freebsd.org/D6120

AFFECTED FILES
  head/sys/netinet/tcp_syncache.c

CHANGE DETAILS
  diff --git a/head/sys/netinet/tcp_syncache.c b/head/sys/netinet/tcp_syncache.c
  --- a/head/sys/netinet/tcp_syncache.c
  +++ b/head/sys/netinet/tcp_syncache.c
  @@ -127,7 +127,8 @@
   static void   syncache_drop(struct syncache *, struct syncache_head *);
   static void   syncache_free(struct syncache *);
   static void   syncache_insert(struct syncache *, struct syncache_head *);
  -static intsyncache_respond(struct syncache *, struct syncache_head *, 
int);
  +static intsyncache_respond(struct syncache *, struct syncache_head *, 
int,
  + const struct mbuf *);
   static struct socket *syncache_socket(struct syncache *, struct 
socket *,
struct mbuf *m);
   static void   syncache_timeout(struct syncache *sc, struct syncache_head 
*sch,
  @@ -457,7 +458,7 @@
free(s, M_TCPLOG);
}
   
  - syncache_respond(sc, sch, 1);
  + syncache_respond(sc, sch, 1, NULL);
TCPSTAT_INC(tcps_sc_retransmitted);
syncache_timeout(sc, sch, 0);
}
  @@ -1307,7 +1308,7 @@
s, __func__);
free(s, M_TCPLOG);
}
  - if (syncache_respond(sc, sch, 1) == 0) {
  + if (syncache_respond(sc, sch, 1, m) == 0) {
sc->sc_rxmits = 0;
syncache_timeout(sc, sch, 1);
TCPSTAT_INC(tcps_sndacks);
  @@ -1474,7 +1475,7 @@
/*
 * Do a standard 3-way handshake.
 */
  - if (syncache_respond(sc, sch, 0) == 0) {
  + if (syncache_respond(sc, sch, 0, m) == 0) {
if (V_tcp_syncookies && V_tcp_syncookiesonly && sc != &scs)
syncache_free(sc);
else if (sc != &scs)
  @@ -1505,7 +1506,8 @@
   }
   
   static int
  -syncache_respond(struct syncache *sc, struct syncache_head *sch, int locked)
  +syncache_respond(struct syncache *sc, struct syncache_head *sch, int locked,
  +const struct mbuf *m0)
   {
struct ip *ip = NULL;
struct mbuf *m;
  @@ -1686,6 +1688,10 @@
   
M_SETFIB(m, sc->sc_inc.inc_fibnum);
m->m_pkthdr.csum_data = offsetof(struct tcphdr, th_sum);
  + if (m0 != NULL && M_HASHTYPE_GET(m0) != M_HASHTYPE_NONE) {
  + m->m_pkthdr.flowid = m0->m_pkthdr.flowid;
  + M_HASHTYPE_SET(m, M_HASHTYPE_GET(m0));
  + }
   #ifdef INET6
if (sc->sc_inc.inc_flags & INC_ISIPV6) {
m->m_pkthdr.csum_flags = CSUM_TCP_IPV6;

EMAIL PREFERENCES
  https://reviews.freebsd.org/settings/panel/emailpreferences/

To: sepherosa_gmail.com, adrian, rwatson, gnn, lstewart, glebius, delphij, 
mike-karels.net, jtl, network, transport, hiren, sbruno
Cc: hselasky, freebsd-net-list
diff --git a/head/sys/netinet/tcp_syncache.c b/head/sys/netinet/tcp_syncache.c
--- a/head/sys/netinet/tcp_syncache.c
+++ b/head/sys/netinet/tcp_syncache.c
@@ -127,7 +127,8 @@
 static void	 syncache_drop(struct syncache *, struct syncache_head *);
 static void	 syncache_free(struct syncache *);
 static void	 syncache_insert(struct syncache *, struct syncache_head *);
-static int	 syncache_respond(struct syncache *, struct syncache_head *, int);
+static int	 syncache_respond(struct syncache *, struct syncache_head *, int,
+		const struct mbuf *);
 static struct	 socket *syncache_socket(struct syncache *, struct socket *,
 		struct mbuf *m);
 static void	 syncache_timeout(struct syncache *sc, struct syncache_head *sch,
@@ -457,7 +458,7 @@
 			free(s, M_TCPLOG);
 		}
 
-		syncache_respond(sc, sch, 1);
+		syncache_respond(sc, sch, 1, NULL);
 		TCPSTAT_INC(tcps_sc_retransmitted);
 		syncache_timeout(sc, sch, 0);
 	}
@@ -1307,7 +1308,7 @@
 			s, __func__);
 			free(s, M_TCPLOG);
 		}
-		if (syncache_respond(sc, sch, 1) == 0) {
+		if (syncache_respond(sc, sch, 1, m) == 0) {
 			sc->sc_rxmits = 0;
 			syncache_timeout(sc, sch, 1);
 			TCPSTAT_INC(tcps_sndacks);
@@ -1474,7 +1475,7 @@
 	/*
 	 * Do a standard 3-way handshake.
 	 */
-	if (syncache_respond(sc, sch, 0) == 0) {
+	if (syncache_respond(sc, sch, 0, m) == 0) {
 		if (V_tcp_syncookies && V_tcp_syncookiesonly && sc != &scs)
 			syncache_free(sc);
 		else if (sc != &scs)
@@ -1505,7 +1506,8 @@
 }
 
 static int
-syncache_respond(struct syncache *sc, struct syncache_head *sch, int locked)
+syncache_respond(struct syncache *sc, struct syncache_head *sch, int locked,
+const struct mbuf *

[Differential] D6148: tcp/syncache: Add comment for syncache_respond

2016-04-29 Thread sepherosa_gmail.com (Sepherosa Ziehau)

sepherosa_gmail.com created this revision.
sepherosa_gmail.com added reviewers: network, adrian, rwatson, gnn, lstewart, 
glebius, delphij, mike-karels.net, jtl, hiren, sbruno, hselasky.
sepherosa_gmail.com added a subscriber: freebsd-net-list.
Herald added a reviewer: transport.

REVISION SUMMARY
  Suggested by: hiren, hps

REVISION DETAIL
  https://reviews.freebsd.org/D6148

AFFECTED FILES
  sys/netinet/tcp_syncache.c

CHANGE DETAILS
  diff --git a/sys/netinet/tcp_syncache.c b/sys/netinet/tcp_syncache.c
  --- a/sys/netinet/tcp_syncache.c
  +++ b/sys/netinet/tcp_syncache.c
  @@ -1505,6 +1505,10 @@
return (rv);
   }
   
  +/*
  + * Send SYN|ACK to the peer.  Either in response to the peer's SYN,
  + * i.e. m0 != NULL, or upon 3WHS ACK timeout, i.e. m0 == NULL.
  + */
   static int
   syncache_respond(struct syncache *sc, struct syncache_head *sch, int locked,
   const struct mbuf *m0)
  @@ -1688,6 +1692,11 @@
   
M_SETFIB(m, sc->sc_inc.inc_fibnum);
m->m_pkthdr.csum_data = offsetof(struct tcphdr, th_sum);
  + /*
  +  * If we have peer's SYN and it has a flowid, then let's assign it to
  +  * our SYN|ACK.  ip6_output() and ip_output() will not assign flowid
  +  * to SYN|ACK due to lack of inp here.
  +  */
if (m0 != NULL && M_HASHTYPE_GET(m0) != M_HASHTYPE_NONE) {
m->m_pkthdr.flowid = m0->m_pkthdr.flowid;
M_HASHTYPE_SET(m, M_HASHTYPE_GET(m0));

EMAIL PREFERENCES
  https://reviews.freebsd.org/settings/panel/emailpreferences/

To: sepherosa_gmail.com, network, transport, adrian, rwatson, gnn, lstewart, 
glebius, delphij, mike-karels.net, jtl, hiren, sbruno, hselasky
Cc: freebsd-net-list
diff --git a/sys/netinet/tcp_syncache.c b/sys/netinet/tcp_syncache.c
--- a/sys/netinet/tcp_syncache.c
+++ b/sys/netinet/tcp_syncache.c
@@ -1505,6 +1505,10 @@
 	return (rv);
 }
 
+/*
+ * Send SYN|ACK to the peer.  Either in response to the peer's SYN,
+ * i.e. m0 != NULL, or upon 3WHS ACK timeout, i.e. m0 == NULL.
+ */
 static int
 syncache_respond(struct syncache *sc, struct syncache_head *sch, int locked,
 const struct mbuf *m0)
@@ -1688,6 +1692,11 @@
 
 	M_SETFIB(m, sc->sc_inc.inc_fibnum);
 	m->m_pkthdr.csum_data = offsetof(struct tcphdr, th_sum);
+	/*
+	 * If we have peer's SYN and it has a flowid, then let's assign it to
+	 * our SYN|ACK.  ip6_output() and ip_output() will not assign flowid
+	 * to SYN|ACK due to lack of inp here.
+	 */
 	if (m0 != NULL && M_HASHTYPE_GET(m0) != M_HASHTYPE_NONE) {
 		m->m_pkthdr.flowid = m0->m_pkthdr.flowid;
 		M_HASHTYPE_SET(m, M_HASHTYPE_GET(m0));

___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: Working divert socket example prog?

2016-04-29 Thread Ian Smith

On Fri, 29 Apr 2016 00:32:05 -0300, lpa lpa wrote:
 > On Thu, Apr 28, 2016 at 4:06 PM, Nikolay Denev  wrote:

 >> Hi,
 >>
 >> Have you looked at the natd(8) source code?

 > yes but it's a complete application, it does a lot of stuff and I am 
 > not able to "clean" it up to become a simple divert application which 
 > reinjects packet so I can measure something. I was not able, so far, 
 > to make thementioned divert-loop work, not breaking natd is a much 
 > longer step to a non programmer

You could have a look at Mike Makonnen's ipfw-classifyd, which diverts 
packets with the intent of applying Linux L7 filters to traffic, then 
feeding matching packets via dummynet pipes, originally for throttling
P2P applications and such.

You don't need to be interested in what it does with packets to benefit 
from knowing how it works; it's maybe an order of magnitude simpler than 
natd; and it can return from divert to different ipfw rule numbers.

This didn't become a port, not sure why, but maybe it might help you 
(though you'd still need some coding to make it useful for your needs):

http://people.freebsd.org/~mtm/ipfw-classifyd.tar.bz2

cheers, Ian

> On Thu, Apr 28, 2016 at 7:21 AM, lpa lpa 
> wrote:
> > Do anyone have a working example code of a divert loop program?
> >
> > I tried building this one[1] but it seems to be for FreeBSD 5 and won't
> > build on latest system. I want to make simple measurement of pps rate
> for a
> > packet traveling inside a divert socket and getting reinjected back while
> > printing some logs.
> >
> > However I am not a coder :(
> >
> > Thank you.
> >
> > [1]https://github.com/DianeRay/divert-loop/tree/master/divert-loop
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Differential] D6137: tcp/lro: Refactor the free/active list operation.

2016-04-29 Thread gallatin (Andrew Gallatin)

gallatin accepted this revision.
gallatin added a comment.


  Thanks, I'm good w/this version

REVISION DETAIL
  https://reviews.freebsd.org/D6137

EMAIL PREFERENCES
  https://reviews.freebsd.org/settings/panel/emailpreferences/

To: sepherosa_gmail.com, network, adrian, delphij, glebius, hselasky, np, 
decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, transport, 
rrs, gallatin
Cc: freebsd-net-list
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 208409] [PATCH] igb and ALTQ

2016-04-29 Thread bugzilla-noreply

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=208409

--- Comment #4 from Sean Bruno  ---
Eric:

I'm going to commit this today.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 208409] [PATCH] igb and ALTQ

2016-04-29 Thread bugzilla-noreply

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=208409

--- Comment #5 from Eric Joyner  ---
(In reply to Sean Bruno from comment #4)

Okay; this looks fine.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Re: netmap overrun counters

[Differential] D6120: tcp/syncache: Set flowid and hash type properly for SYN|ACK

[Differential] D6120: tcp/syncache: Set flowid and hash type properly for SYN|ACK

[Differential] D6120: tcp/syncache: Set flowid and hash type properly for SYN|ACK

[Differential] D6148: tcp/syncache: Add comment for syncache_respond

Re: Working divert socket example prog?

[Differential] D6137: tcp/lro: Refactor the free/active list operation.

[Bug 208409] [PATCH] igb and ALTQ

[Bug 208409] [PATCH] igb and ALTQ

9 matches

Site Navigation

Mail list logo

Footer information