On Thu, Jul 11, 2013 at 4:52 PM, Gleb Smirnoff <gleb...@freebsd.org> wrote:
> On Thu, Jul 11, 2013 at 04:49:25PM +0200, Luigi Rizzo wrote:
> L> >> IMO, this should be a single counter accessible via sysctl, with no
> L> >> printf(). Those, who need details on whether this is micro-burst or
> L> >> persistent condition, can run monitoring software that draws plots.
> L> >
> L> >
> L> > The single counter wouldn't tell you anything because it misses which
> L> > socket/accept queue is affected by the overflow. The inpcb pointer
> L> > can be cross-refrenced with netstat -a.
> L> >
> L> > Andriy for example would never have found out about this problem other
> L> > than receiving vague user complaints about aborted connection attempts.
> L> > Maybe after spending many hours searching for the cause he may have
> L> > interfered from endless scrolling in Wireshark that something wasn't
> L> > right and blame syncache first. Only later it would emerge that he's
> L> > either receiving too many connections or his application is too slow
> L> > dealing with incoming connections.
> L> >
> L> > If you can recommend a suitable and general sysadmin friendly monitoring
> L> > software that will point out this problem I'm all ears.
> L>
> L> the problem with these non-throttled messages is that they often
> L> cause thrashing -- you become slighly slow, messages start being
> L> generated and your system becomes a lot slower, making it hard
> L> to recover.
> L>
> L> What i usually do is throttle (in the kernel) and count the number of
> L> message suppressed. Something like this (in a macro):
> L>
> L> static int ctr, last_tick;
> L> if (ticks - last_tick > suppression_delay) {
> L> printf("got this error ... (%d times)\n", ... , ctr);
> L> ctr = 0;
> L> last_tick = tick;
> L> } else {
> L> ctr++;
> L> }
> L>
> L> the errors may not be exactly the same, the counter is race_prone
> L> (you can make it atomic if you really feel like) but the whole point is
> L> to get the idea that something is very wrong, not the exact count
> L> or pointer
>
> btw, there is ready function for that: ppsratecheck(), already utilized
> for suppressing some error messages.
yes, i think i saw it before. To me, the convenience of the macro is that
it can also wrap the declaration of the static variables and the printf.
I basically have macros like this (see sys/dev/netmap/netmap_kern.h)
RD(max_pps, "printf format ", arguments....) // rate-limited printf
ND(same arguments as above) // compiles to no-op
so i can quickly add the messages or disable them by simply changing
the macro name
FWIW the macro in netmap_kern.h does not have the counter of suppressed
messages (I just thought about it , but i should probably add it as a feature)
cheers
luigi
> --
> Totus tuus, Glebius.
--
-----------------------------------------+-------------------------------
Prof. Luigi RIZZO, ri...@iet.unipi.it . Dip. di Ing. dell'Informazione
http://www.iet.unipi.it/~luigi/ . Universita` di Pisa
TEL +39-050-2211611 . via Diotisalvi 2
Mobile +39-338-6809875 . 56122 PISA (Italy)
-----------------------------------------+-------------------------------
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
