Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-25 Thread Duy Nguyen
On Thu, Feb 25, 2016 at 3:37 AM, David Turner wrote: > On Sat, 2016-02-20 at 15:59 +0700, Duy Nguyen wrote: >> On Thu, Feb 18, 2016 at 12:17 PM, David Turner < >> dtur...@twopensource.com> wrote: >> > LMDB has a few features that make it suitable for usage in git: >> > ... >> >> I'm reading lmdb d

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-25 Thread Duy Nguyen
On Thu, Feb 25, 2016 at 3:43 AM, David Turner wrote: >> > > > > I'm not sure I get this comment. D/F conflicts are no longer >> > > > > a >> > > > > thing >> > > > > for lmdb backend, right? >> > > > >> > > > I'm trying to avoid the lmdb backend creating a set of refs >> > > > that >> > > > the >>

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-24 Thread David Turner
On Sat, 2016-02-20 at 09:58 +0700, Duy Nguyen wrote: > > On Fri, 2016-02-19 at 09:54 +0700, Duy Nguyen wrote: > > > On Fri, Feb 19, 2016 at 3:23 AM, David Turner < > > > dtur...@twopensource.com> wrote: > > > > > > +static int read_per_worktree_ref(const char *submodule, > > > > > > const > > > > >

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-24 Thread David Turner
On Sat, 2016-02-20 at 20:14 +0700, Duy Nguyen wrote: > On Fri, Feb 19, 2016 at 3:23 AM, David Turner < > dtur...@twopensource.com> wrote: > > On Thu, 2016-02-18 at 15:50 +0700, Duy Nguyen wrote: > > > > [snip] > > > > Thanks; applied the above > > > > > This permission makes me wonder if we need

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-24 Thread David Turner
On Sat, 2016-02-20 at 15:59 +0700, Duy Nguyen wrote: > On Thu, Feb 18, 2016 at 12:17 PM, David Turner < > dtur...@twopensource.com> wrote: > > LMDB has a few features that make it suitable for usage in git: > > ... > > I'm reading lmdb documents and hitting the caveat section [1]. > Random though

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-20 Thread Junio C Hamano
David Turner writes: > So just add this after every mkdir? > > if (shared_repository) > adjust_shared_perm(db_path); The function itself checks shared_repository configuration, so the caller does not have to bother. You should rather treat it as a declaration and a documenta

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-20 Thread Duy Nguyen
On Fri, Feb 19, 2016 at 3:23 AM, David Turner wrote: > On Thu, 2016-02-18 at 15:50 +0700, Duy Nguyen wrote: > > [snip] > > Thanks; applied the above > >> This permission makes me wonder if we need adjust_shared_perm() here >> and some other places. > > So just add this after every mkdir? > >

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-20 Thread Duy Nguyen
On Thu, Feb 18, 2016 at 12:17 PM, David Turner wrote: > LMDB has a few features that make it suitable for usage in git: > ... I'm reading lmdb documents and hitting the caveat section [1]. Random thoughts * "There is normally no pure read-only mode, since readers need write access to locks and

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-19 Thread Duy Nguyen
> On Fri, 2016-02-19 at 09:54 +0700, Duy Nguyen wrote: >> On Fri, Feb 19, 2016 at 3:23 AM, David Turner < >> dtur...@twopensource.com> wrote: >> > > > +static int read_per_worktree_ref(const char *submodule, const >> > > > char >> > > > *refname, >> > > > +struct MDB_val

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-19 Thread Junio C Hamano
David Turner writes: > Something like the following? > > commit aad6b84fd1869f6e1cf6ed15bcece0c2f6429e9d > Author: David Turner > Date: Thu Feb 18 17:09:29 2016 -0500 > > refs: break out some functions from resolve_ref_1 > > A bunch of resolve_ref_1 is not backend-specific, so we

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-19 Thread David Turner
On Thu, 2016-02-18 at 15:50 +0700, Duy Nguyen wrote: > Caveat: I did not study how to use lmdb. I just guessed what it does > based on function names. I don't know much about refs handling either > (especially since the transaction thing is introduced) > > > diff --git a/Documentation/technical/re

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-19 Thread David Turner
On Fri, 2016-02-19 at 09:54 +0700, Duy Nguyen wrote: > On Fri, Feb 19, 2016 at 3:23 AM, David Turner < > dtur...@twopensource.com> wrote: > > > > +static int read_per_worktree_ref(const char *submodule, const > > > > char > > > > *refname, > > > > +struct MDB_val *val, i

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-18 Thread Duy Nguyen
On Fri, Feb 19, 2016 at 3:23 AM, David Turner wrote: >> > +static int read_per_worktree_ref(const char *submodule, const char >> > *refname, >> > +struct MDB_val *val, int >> > *needs_free) >> >> From what I read, I suspect these _per_worktree functions will be >> ident

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-18 Thread Junio C Hamano
David Turner writes: > On Thu, 2016-02-18 at 15:50 +0700, Duy Nguyen wrote: > > [snip] > > Thanks; applied the above Please. Your other messages did excessively quote parts of the message you are not responding to, but this will not tell anybody but you what "the above" refers to, not even to D

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-18 Thread David Turner
On Thu, 2016-02-18 at 15:50 +0700, Duy Nguyen wrote: [snip] Thanks; applied the above > This permission makes me wonder if we need adjust_shared_perm() here > and some other places. So just add this after every mkdir? if (shared_repository) adjust_shared_perm(db_path);

Re: [PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-18 Thread Duy Nguyen
Caveat: I did not study how to use lmdb. I just guessed what it does based on function names. I don't know much about refs handling either (especially since the transaction thing is introduced) > diff --git a/Documentation/technical/refs-lmdb-backend.txt > b/Documentation/technical/refs-lmdb-back

[PATCH v5 25/27] refs: add LMDB refs storage backend

2016-02-17 Thread David Turner
Add a database backend for refs using LMDB. This backend runs git for-each-ref about 30% faster than the files backend with fully-packed refs on a repo with ~120k refs. It's also about 4x faster than using fully-unpacked refs. In addition, and perhaps more importantly, it avoids case-conflict is