On Thu, Oct 18, 2007 at 10:16:52AM -0400, Bill Sommerfeld wrote: > On Thu, 2007-10-18 at 08:04 -0500, Gary Mills wrote: > > Here's a suggestion on the cause: > > > > The root problem seems to be an interaction between Solaris' concept > > of global memory consistency and the fact that Cyrus spawns many > > processes that all memory map (mmap) the same file. Whenever any > > process updates any part of a memory mapped file, Solaris freezes all > > of the processes that have that file mmaped, updates their memory > > tables, and then re-schedules the processes to run. When we have > > problems we see the load average go extremely high and no useful work > > gets done by Cyrus. > > that sounds like a somewhat mangled description of the cross-calls done > to invalidate the TLB on other processors when a page is unmapped. > (it certainly doesn't happen on *every* update to a mapped file). > > from grepping the source code it looks like cyrus is both multithreaded > and a heavy user of munmap.
Here's a process summary of our Cyrus back-end which uses ZFS for its mail store: 10:08am up 38 day(s), 12:11, 1 user, load average: 2.36, 2.02, 1.89 %CPU NUM COMM 5.2 1788 imapd 0.7 11 pop3d 0.6 43 lmtpd 0.2 2 /usr/local/cyrus/bin/master 0.1 1 ps 0.1 1 idled 0.1 1 fsflush 0.1 1 /usr/sbin/syslogd 0.1 1 /opt/local/mysql/libexec/mysqld The imapd, pop3d, and lmtpd processes are single-threaded. There's one for each client connection. `master' on the other hand is supposed to be multi-threaded, but `prstat' shows only one thread now. There are two because one is the mupdate master and the other is the back-end master. What's the command to show cross calls? -- -Gary Mills- -Unix Support- -U of M Academic Computing and Networking- _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
