Ok, I shall introduce a new configuration option --with-abstract-sockets then.
Copying this message to the mailing list so that other devs also know of it. Sree On 01/31/2014 03:40 PM, Christian Grothoff wrote: > Hi! > > Why #ifdef? We could have an extra option "USE_ABSTRACT_SOCKETS", > and if that is set and access controls are permissive (as in, any > user/group has full access anyway), then we if (cond) to abstract > sockets. That should be fine, as you can obviously disable ACL > on supermuc and set the abstract sockets option. > > My 2 cents > > -Christian > > On 01/31/2014 02:32 PM, Sree Harsha Totakura wrote: >> Hi Christian, >> >> We discussed about the slow startup of peers on the supermuc a while >> ago. Here is something I found related to it. The 'bind' calls are >> still relatively slow than they used to be earlier. >> >> I suspect this is due to the changes you introduced to fix #2887. With >> these changes, gnunet *always* uses named sockets and since the compute >> nodes on the supermuc have a shared global network file system creating >> a socket file on these is slowing us down. >> >> I foresee that we will have the same problem on other testbeds where it >> is very likely to have a network file system. So, should I ifdef the >> code with a new configure flag to use abstract domain sockets and forgo >> the user access permission checks in the case where abstract sockets are >> used? >> >> - >> Sree >> >> On 12/04/2013 01:57 PM, Christian Grothoff wrote: >>> Hi! >>> >>> 'stat' calls should now be reduced significantly in SVN HEAD. >>> 'bind' might also succeed more often, which might also help. >>> >>> Please let me know if this helped, and how much. >>> >>> Best, >>> >>> Christian >>> >>> On 12/04/2013 01:16 PM, Sree Harsha Totakura wrote: >>>> Hi Christian, >>>> >>>> Just a status update on the SuperMUC: >>>> >>>> I found the reason why our processes do not receive some signals -- >>>> SIGTERM, SIGINT, SIGPIPE, SIGHUP were masked by the MPI run time. Since >>>> the signal mask is preserved through exec, our processes where are >>>> exec'ed from MPI processes also have these signals masked out. MSH now >>>> clears this mask before execing child processes and this seems to work. >>>> >>>> The other problem we are having now is the embarrassingly slow startup >>>> of peers on the compute nodes; with the ARM processes often found in >>>> "uninterruptable sleep" (D) state. I traced the system calls from all >>>> the started ARM services and found that the processes are spending about >>>> a considerably time in `stat()'. The following are the top 20 system >>>> calls ordered according to the time spent executing them. They were >>>> collected and aggregated over 200 ARM services: >>>> >>>> stat 12972.281245 >>>> bind 5504.138075 >>>> select 187.148530 >>>> open 82.702509 >>>> access 78.527948 >>>> read 11.191872 >>>> fcntl 10.993160 >>>> close 5.692947 >>>> setsockopt 3.294542 >>>> socket 2.986985 >>>> brk 2.909969 >>>> execve 2.174772 >>>> mmap 1.771800 >>>> mprotect 1.767051 >>>> umask 1.240802 >>>> chmod 0.925778 >>>> rt_sigaction 0.753291 >>>> listen 0.722637 >>>> fstat 0.596976 >>>> fadvise64 0.357256 >>>> >>>> On a related note, I observed that MPI tasks we start on the SuperMUC >>>> are always pinned to CPUs. All processes which are started from an MPI >>>> task get pinned to the same CPU. I can confirm this as we only start >>>> one MSH task per compute node and all the processes which are started >>>> from this MSH process contend and run on the same core. I tried to >>>> disable this process affinity, but this is being restricted by the >>>> administrators. For now, I am able to change the affinity setting to >>>> use a round-robin fashion, but still this means that processes are >>>> *always* pinned to a CPU; previously they all get pinned to a single CPU >>>> because we only start 1 MSH task, now they are pinned across 32 CPUs. >>>> In any case, I believe that this behavior is harming us. >>>> >> _______________________________________________ GNUnet-developers mailing list GNUnet-developers@gnu.org https://lists.gnu.org/mailman/listinfo/gnunet-developers
