Re: Does anyone use nscd?
Lawrence Stewart writes: > If the machine running nscd loses connectivity with the DNS server for > a while and does a DNS lookup during that time, nscd will cache the > -ve reply indefinitely for all users, which breaks all sorts of > crap. Have to forcibly run "nscd -I all" to fix. I will find and fix > this bug one day if noone beats me to it... Definitely a bug, nscd is only supposed to cache negative responses for 60 seconds. I hope you find the time to track it down :) Is it 100% reproducable? How long does the DNS server have to be unreachable before it happens? > I'd like to see it stay in base. Moving it (slowly) towards a point > where we can turn it on by default would be cool. Agreed, in principle. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
"Daniel O'Connor" writes: > I'd be interested in testing your workaround(s) :) It wasn't a workaround, actually, just a one-line change that enables additional logging (when running with from the console -nst) which might help me figure out why it crashes. See my reply to Artem Belevich earlier in this thread. While we're at it, I'd be very grateful if someone could email me a quick and dirty guide to setting up an LDAP server for testing. I have too much on my plate right now to start reading documentation... DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
On 05/10/2011, at 19:13, Dag-Erling Smørgrav wrote: > "Daniel O'Connor" writes: >> I'd be interested in testing your workaround(s) :) > > It wasn't a workaround, actually, just a one-line change that enables > additional logging (when running with from the console -nst) which might > help me figure out why it crashes. See my reply to Artem Belevich > earlier in this thread. OK I'll dig it up.. > While we're at it, I'd be very grateful if someone could email me a > quick and dirty guide to setting up an LDAP server for testing. I have > too much on my plate right now to start reading documentation… Hmm, most of the heavy lifting for me was done by the net/smbldap-tools port but it was still fiddly and it was a while ago. Hopefully an LDAP guru can offer more :( -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
On 05/10/2011 09:43, Dag-Erling Smørgrav wrote: > While we're at it, I'd be very grateful if someone could email me a > quick and dirty guide to setting up an LDAP server for testing. I have > too much on my plate right now to start reading documentation... The Quick Start guide on the OpenLDAP site is pretty good: http://www.openldap.org/doc/admin24/quickstart.html although steps 1 -- 8 just boil down to 'install from ports' on FreeBSD. Notes: 1) Don't enable SASL -- it adds a lot of complexity but doesn't change anything fundamental in the way LDAP works for testing purposes. 2) The default schema include inetOrgPerson and Posix which is enough to deal with basic Unix users and groups. If you want to do anything more advanced (eg. sudo related or OpenSSH LPK patches) then you'll need to import some external schema. I recommend always copying the schema files into $PREFIX/etc/openldap/schema or else casually removing a port could prevent your slapd from restarting days or weeks later... 3) The structure of an LDAP tree is site-specific and can be quite different between different organizations, but in essence it consists of sorting and grouping various classes of objects into various subdirectories of your directory tree. For testing purposes, impose at least a minimal amount of structure. As the quick start guide suggests, use the dc=example,dc=com form based on your domain name to root your LDAP tree. Within that, create some sub-directories 'ou=Users', 'ou=Groups', 'ou=Hosts' for storing objects of the appropriate types. This should provide a reasonable parallel to what most people would use in production. 4) ACLs and permissions are pretty complex in LDAP. This is something where you are going to have to spend some quality time with the manuals I'm afraid. 5) phpldapadmin is a pretty good tool for populating a directory with test data. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matt...@infracaninophile.co.uk Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: Does anyone use nscd?
04.10.2011 22:47, Sean M. Collins пишет: > I've never heard of the utility until you mentioned it. > > I'd nuke it, since really there are more popular alternatives like Redis > and Memcached in the ports tree that most people will reach for first. > Please look at http://www.freebsd.org/cgi/query-pr.cgi?pr=75855 before making final decision. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: how are callouts handled in cpu_idle() ?
On 10/01/11 04:25, Alexander Motin wrote: Hi. Adrian Chadd wrote: What happens if this occurs: * cpu_idle() is entered * you enter critical_enter() and call cpu_idleclock() * an interrupt occurs, scheduling an immediate callout * .. but you've already set the clock register, so it won't be serviced until the wait returns. Perhaps interrupts have to be disabled before critical_enter() and cpu_idletick() to ensure an interrupt-driven callout doesn't get delayed? Use of critical section in cpu_idle() from the beginning was based on number of assumptions about filter interrupt handler's limitations. These handlers are not guarantied to get updated system time/ticks and they are discouraged to use callouts. If callout scheduled from interrupt filter during system wake up, system has no ticks counter updated yet and you may get callout scheduled into the past (it will be run immediately) or at least much earlier (up to 250ms) then requested. In your case callout indeed may get delayed (up to the same 250ms). All that is not a problem for regular interrupt threaded interrupts -- interrupt thread execution will be delayed until all stuff get updated. Can you explain why the critical section is there in more detail? It seems like all of our problems arise because of it. -Nathan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
On Wednesday 05 October 2011 10:46:47 Eugene Grosbein wrote: > 04.10.2011 22:47, Sean M. Collins пишет: > > I've never heard of the utility until you mentioned it. > > > > I'd nuke it, since really there are more popular alternatives like Redis > > and Memcached in the ports tree that most people will reach for first. > I had not used it before I saw this thread.. But now I am 8-) I kind of like it, actually.. Especially for DNS caching.. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
4.4BSD timeslice management
Hello, I'm looking into the code of 4.4BSD scheduler (and related files) of FreeBSD version 7.2-RELEASE on an amd64 machine in order to understand how to manage time slices. I'm trying to dynamically change the length of the time slice on a thread by thread basis to "give more priority" or "less priority" to threads. In function sched_clock (sched_4bsd.c) I found a piece of code in which thread's flags are added with TDF_NEEDRESCHED when the time slice (quantum) expires but I failed to notice any point in the code that actually preempt the thread. Adding a per-thread parameter (e.g., in struct thread) I could set TDF_NEEDRESCHED when the dynamic time slice expires but how can I be sure the thread gets preempted? Where can I set a per-thread parameter (maybe after cpu_switch?!)? Is there a timer periodically calling a sort of context switch function or the context switches happen only when triggered by the kernel code? Thanks Riccardo Cattaneo___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: how are callouts handled in cpu_idle() ?
On 5 October 2011 23:12, Nathan Whitehorn wrote: > Can you explain why the critical section is there in more detail? It seems > like all of our problems arise because of it. It seems to make some of my MIPS cases more problematic. I don't know about the i386. DES reports that his weird/crashing fileserver issues are fixed by setting idletick=1. I'd love to hear from other users who have fixed their HEAD/9.0 issues by setting kernel.eventtimer.idletick=1. I have a feeling that some corner cases in interrupt handling and task scheduling are now (more) exposed due to the timer code in 9. I'm hoping to replicate some issues on my i386 eeepc (which uses ACPI for halting, rather than HLT) and see if that also contributes. Adrian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
2011/10/4 Dag-Erling Smørgrav : > Any chance of getting a backtrace from an unpatched nscd? Ideally with > the change described here: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/136073#reply1 > > To test, stop nscd, then run it from the command line like so: > > $ su - > # cd /tmp > # ulimit -c 0 > # /usr/sbin/nscd -nst > (do something in another terminal that causes it to crash) > # echo backtrace | gdb -batch -x /dev/stdin /usr/sbin/nscd nscd.core > > and send me the output from both nscd and gdb once it crashes. In my case it's top that dies with SIGPIPE. nscd keeps running just fine. So, there's no backtrace from nscd. top receives SIGPIPE after it tries to write to the socket with nscd on the other end. Apparently nscd closes connection on its end. Running ktrace on top I see that before the write to nscd socket, there's a read that returned 0 bytes. Here's top's backtrace. Alas I don't have libc with debug symbols handy: Program received signal SIGPIPE, Broken pipe. 0x000800abe8cc in write () from /lib/libc.so.7 (gdb) where #0 0x000800abe8cc in write () from /lib/libc.so.7 #1 0x000800aa3f44 in ftell () from /lib/libc.so.7 #2 0x000800aa415f in ftell () from /lib/libc.so.7 #3 0x000800aa2031 in __h_errno () from /lib/libc.so.7 #4 0x000800a98311 in nsdispatch () from /lib/libc.so.7 #5 0x000800a84d95 in getpwent_r () from /lib/libc.so.7 #6 0x000800a84911 in acl_get_brand_np () from /lib/libc.so.7 #7 0x00404f7b in machine_init (statics=0x7fffe770, do_unames=1 '\001') at /usr/srcdir/src.git/usr.bin/top/machine.c:258 #8 0x0040a9ab in main (argc=1, argv=0x7fffe8c8) at /usr/srcdir/src.git/usr.bin/top/../../contrib/top/top.c:464 --Artem ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
2011/10/5 Michael Bushkov : > There are probably 2 things here: > 1. There's some error in nsswitch<->nscd communication protocol that > causes nsswitch to write into the closed socket. This is not trivial > to investigate and will require analyzing nscd and client process logs > side by side (and possibly adding some more logging). > 2. Consequences of the aforementioned problem can probably be > corrected by using _setsockopt(..., SO_NOSIGPIPE) in > __open_cached_connection() in nscachedcli.c > (http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/net/nscachedcli.c?rev=1.3). > > I have no access to FreeBSD desktop at the moment - Artem, it would be > cool if you can try the second solution. That's exactly what I did and it fixed the problem on the client side. I've posted the patch setting SO_NOSIGPIPE earlier in this thread: http://lists.freebsd.org/pipermail/freebsd-hackers/2011-October/036539.html --Artem ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
There are probably 2 things here: 1. There's some error in nsswitch<->nscd communication protocol that causes nsswitch to write into the closed socket. This is not trivial to investigate and will require analyzing nscd and client process logs side by side (and possibly adding some more logging). 2. Consequences of the aforementioned problem can probably be corrected by using _setsockopt(..., SO_NOSIGPIPE) in __open_cached_connection() in nscachedcli.c (http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/net/nscachedcli.c?rev=1.3). I have no access to FreeBSD desktop at the moment - Artem, it would be cool if you can try the second solution. Cheers, Michael 2011/10/5 Artem Belevich : > 2011/10/4 Dag-Erling Smørgrav : >> Any chance of getting a backtrace from an unpatched nscd? Ideally with >> the change described here: >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/136073#reply1 >> >> To test, stop nscd, then run it from the command line like so: >> >> $ su - >> # cd /tmp >> # ulimit -c 0 >> # /usr/sbin/nscd -nst >> (do something in another terminal that causes it to crash) >> # echo backtrace | gdb -batch -x /dev/stdin /usr/sbin/nscd nscd.core >> >> and send me the output from both nscd and gdb once it crashes. > > In my case it's top that dies with SIGPIPE. nscd keeps running just > fine. So, there's no backtrace from nscd. > > top receives SIGPIPE after it tries to write to the socket with nscd > on the other end. Apparently nscd closes connection on its end. > Running ktrace on top I see that before the write to nscd socket, > there's a read that returned 0 bytes. > > Here's top's backtrace. Alas I don't have libc with debug symbols handy: > > Program received signal SIGPIPE, Broken pipe. > 0x000800abe8cc in write () from /lib/libc.so.7 > (gdb) where > #0 0x000800abe8cc in write () from /lib/libc.so.7 > #1 0x000800aa3f44 in ftell () from /lib/libc.so.7 > #2 0x000800aa415f in ftell () from /lib/libc.so.7 > #3 0x000800aa2031 in __h_errno () from /lib/libc.so.7 > #4 0x000800a98311 in nsdispatch () from /lib/libc.so.7 > #5 0x000800a84d95 in getpwent_r () from /lib/libc.so.7 > #6 0x000800a84911 in acl_get_brand_np () from /lib/libc.so.7 > #7 0x00404f7b in machine_init (statics=0x7fffe770, > do_unames=1 '\001') at /usr/srcdir/src.git/usr.bin/top/machine.c:258 > #8 0x0040a9ab in main (argc=1, argv=0x7fffe8c8) at > /usr/srcdir/src.git/usr.bin/top/../../contrib/top/top.c:464 > > --Artem > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > > > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
Michael Bushkov writes: > 2. Consequences of the aforementioned problem can probably be > corrected by using _setsockopt(..., SO_NOSIGPIPE) in > __open_cached_connection() in nscachedcli.c That sounds like a workaround rather than a fix... DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
2011/10/5 Dag-Erling Smørgrav : > Michael Bushkov writes: >> 2. Consequences of the aforementioned problem can probably be >> corrected by using _setsockopt(..., SO_NOSIGPIPE) in >> __open_cached_connection() in nscachedcli.c > > That sounds like a workaround rather than a fix... Not necessarily. Using SO_NOSIGPIPE is a valid option when someone wants to see read/write on a closed socket fail and return -1 with errno=EPIPE. Quick grep in libc shows that resolver code in lib/libc/resolv/res_send.c also sets SO_NOSIGPIPE for exactly that reason. --Artem ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
On Tue, 4 Oct 2011 18:51+0200, Dag-Erling Smørgrav wrote: > Trond Endrestøl writes: > > It's in daily use at Gjøvik Technical College (Fagskolen i Gjøvik), > > here in Norway. Both the mail and web servers authenticates our users > > by LDAP, and nscd certainly speeds up the lookups. > > OK. No trouble with clients dying of SIGPIPE? I could never reproduce > the bug, but both users who reported problems used ldap, and I don't > have an LDAP server to test against, so I thought it might be specific > to LDAP. Not in my (somewhat limited) experience. Trond. -- -- Trond Endrestøl | trond.endres...@fagskolen.gjovik.no ACM, NAS, NUUG, SAGE, USENIX |FreeBSD 8.2-STABLE & Alpine 2.00___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
On Wed, 5 Oct 2011 12:54+1030, Daniel O'Connor wrote: > On 05/10/2011, at 2:30, Michel Talon wrote: > > > Des wrote: > >> Does anyone actually use nscd? > > > > I am using it since a lot of time. I have not experienced annoying bugs > > in all that time. The last time i have been hit is when installing some > > new softs which require adding some user and some group with pw. Of > > course this doesn't work well with caching these data, and i had > > completely forgotten i was using a cache. This is very perplexing. > > In my experience ncsd seems to cache negative hits forever, > regardless of the setting for negative-time-to-live. I'm glad to see I'm not the only one who has noticed this odd behaviour of nscd. Shame on me for not speaking up sooner, but I feared I might be proved wrong (again), and yes, that's a lame excuse. :-/ > If I am installing ports which create a new user or group I have to > restart nscd. I also find if openldap dies (not infrequent) I have > to restart nscd after restarting openldap.. After bulk loading ~250 students into our LDAP (Novell eDirectory) each fall, and deleting the graduated students, I restart nscd on our servers just to make sure the caches doesn't contain any negative results. Maybe I should set up a cron job to restart nscd once a day until the source code is cleaned up. Trond. -- -- Trond Endrestøl | trond.endres...@fagskolen.gjovik.no ACM, NAS, NUUG, SAGE, USENIX |FreeBSD 8.2-STABLE & Alpine 2.00___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
On Tue, 4 Oct 2011 17:20+0200, Dag-Erling Smørgrav wrote: > Does anyone actually use nscd? It's in daily use at Gjøvik Technical College (Fagskolen i Gjøvik), here in Norway. Both the mail and web servers authenticates our users by LDAP, and nscd certainly speeds up the lookups. Trond. -- -- Trond Endrestøl | trond.endres...@fagskolen.gjovik.no ACM, NAS, NUUG, SAGE, USENIX |FreeBSD 8.2-STABLE & Alpine 2.00___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Does anyone use nscd?
In the last episode (Oct 04), Trond Endrestol said: > On Tue, 4 Oct 2011 18:51+0200, Dag-Erling Smorgrav wrote: > > Trond Endrestol writes: > > > It's in daily use at Gjovik Technical College (Fagskolen i Gjovik), > > > here in Norway. Both the mail and web servers authenticates our users > > > by LDAP, and nscd certainly speeds up the lookups. > > > > OK. No trouble with clients dying of SIGPIPE? I could never reproduce > > the bug, but both users who reported problems used ldap, and I don't > > have an LDAP server to test against, so I thought it might be specific > > to LDAP. > > Not in my (somewhat limited) experience. On a tangent, I also heavily recommend using the nss-pam-ldapd port instead of nss_ldap. It includes a daemon called nslcd which is the only process that links to the ldap libary. The nss module is a tiny plug that talks to nslcd using a simple protocol. It really reduces the socket count to your ldap server, and removes the potential namespace problems caused by dlopening libldap.so in every process. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
