2015-12-05 11:51 GMT+01:00 Mark Geisert <m...@maxrnd.com>: > Mark Geisert wrote: >> >> Corinna Vinschen wrote: >>> >>> On Nov 23 16:54, Mark Geisert wrote: >>>> >>>> John Hein wrote: >>>>> >>>>> Mark Geisert wrote at 23:45 -0800 on Nov 22, 2015: >>>>> > Corinna Vinschen wrote: >>>>> > > On Nov 21 01:21, Mark Geisert wrote: >>>>> > [...] so I wonder if there's >>>>> > >> some unintentional serialization going on somewhere, but I >>>>> don't know yet >>>>> > >> how I could verify that theory. >>>>> > > >>>>> > > If I'm allowed to make an educated guess, the big serializer >>>>> in Cygwin >>>>> > > are probably the calls to malloc, calloc, realloc, free. We >>>>> desperately >>>>> > > need a new malloc implementation better suited to >>>>> multi-threading. >> >> [...] >>>>> >>>>> >>>>> Someone recently mentioned on this list they were working on porting >>>>> jemalloc. That would be a good choice. >>>> >>>> >>>> Indeed; thanks for the reminder. Somehow I hadn't followed that thread. >>> >>> >>> Indeed^2. Did you look into the locking any further to see if there's >>> more than one culprit? I guess we've a rather long way to a "lock-less >>> kernel"... > > [...] >> >> But that is just groundwork to identifying which locks are suffering the >> most contention. To identify them at source level I think I'll also >> need to record the caller's RIP when they are being locked. > > > In the OP's very good testcase the most heavily contended locks, by far, are > those internal to git's builtin/pack-objects.c. I plan to show actual stats > after some more cleanup, but I did notice something in that git source file > that might explain the difference between Cygwin and MinGW when running this > testcase... > > #ifndef NO_PTHREADS > > static pthread_mutex_t read_mutex; > #define read_lock() pthread_mutex_lock(&read_mutex) > #define read_unlock() pthread_mutex_unlock(&read_mutex) > > static pthread_mutex_t cache_mutex; > #define cache_lock() pthread_mutex_lock(&cache_mutex) > #define cache_unlock() pthread_mutex_unlock(&cache_mutex) > > static pthread_mutex_t progress_mutex; > #define progress_lock() pthread_mutex_lock(&progress_mutex) > #define progress_unlock() pthread_mutex_unlock(&progress_mutex) > > #else > > #define read_lock() (void)0 > #define read_unlock() (void)0 > #define cache_lock() (void)0 > #define cache_unlock() (void)0 > #define progress_lock() (void)0 > #define progress_unlock() (void)0 > > #endif > > Is it possible the MinGW version of git is compiled with NO_PTHREADS > #defined? If so, it would mean there's no locking being done at all and > would explain the faster execution and near 100% CPU utilization when > running under MinGW.
Nah, there is no threading enabled when there is no pthreads. How would that work? :D See thread-utils.h #ifndef NO_PTHREADS #include <pthread.h> extern int online_cpus(void); extern int init_recursive_mutex(pthread_mutex_t*); #else #define online_cpus() 1 #endif Looks like there is indeed a bug in git code when passing "--threads" explicitly to "git pack-objects", because they show warning about threads being unsupported, but doesn't overwrite delta_search_threads value. I will go to git's ML about it. This is completely not related to our issue. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
