Re: propagating repo corruption across clone

2013-03-28 Thread Jeff Mitchell
On Wed, Mar 27, 2013 at 2:51 PM, Rich Fromm wrote: > Nevertheless, I will try to contact Jeff and point him at this. My initial > reading of his blog posts definitely gave me the impression that this was a > --mirror vs. not issue, but it really sounds like his main problem was using > --local.

Re: propagating repo corruption across clone

2013-03-28 Thread Jeff Mitchell
On Tue, Mar 26, 2013 at 11:47 PM, Junio C Hamano wrote: > The difference between --mirror and no --mirror is a red herring. > You may want to ask Jeff Mitchell to remove the mention of it; it > only adds to the confusion without helping users. If you made > byte-for-byte copy of corrupt repositor

Re: propagating repo corruption across clone

2013-03-27 Thread Jeff King
On Wed, Mar 27, 2013 at 03:49:38PM -0400, Jeff King wrote: > On Wed, Mar 27, 2013 at 11:23:15AM -0700, Rich Fromm wrote: > > > But I'm still somewhat confused about what is and is not checked under what > > conditions. Consider the three statements: > > > > # 1 > > git clone --mirror myuser@myh

Re: propagating repo corruption across clone

2013-03-27 Thread Jeff King
On Wed, Mar 27, 2013 at 11:23:15AM -0700, Rich Fromm wrote: > But I'm still somewhat confused about what is and is not checked under what > conditions. Consider the three statements: > > # 1 > git clone --mirror myuser@myhost:my_repo > > # 2 > git clone --mirror --config transfer.fsckObjects=tr

Re: propagating repo corruption across clone

2013-03-27 Thread Junio C Hamano
Rich Fromm writes: > Apologies if my questions are considered slightly off topic -- I'm not > positive if this is supposed to be a list for developers, and not users. The list is both for users and developers. > However, I think there may be room for some additional clarity in the docs. > The

Re: propagating repo corruption across clone

2013-03-27 Thread Rich Fromm
it-scm.com/book/en/Git-on-the-Server-The-Protocols http://git-scm.com/book/en/Git-Internals-Transfer-Protocols -- View this message in context: http://git.661346.n2.nabble.com/propagating-repo-corruption-across-clone-tp7580504p7580845.html Sent from the git mailing list archive at Nabble.com. -- To uns

Re: propagating repo corruption across clone

2013-03-27 Thread Rich Fromm
and everything else." The latter sounds like a stronger statement to me. But if that's true, perhaps should the relevant section(s) of git-config(1) explicitly note that this is not equivalent to a full git-fsck ? -- View this message in context: http://git.661346.n2.nabble.com/prop

Re: propagating repo corruption across clone

2013-03-27 Thread Sitaram Chamarty
On Wed, Mar 27, 2013 at 8:33 PM, Junio C Hamano wrote: > Sitaram Chamarty writes: > >> On Wed, Mar 27, 2013 at 9:17 AM, Junio C Hamano wrote: >> >>> To be paranoid, you may want to set transfer.fsckObjects to true, >>> perhaps in your ~/.gitconfig. >> >> do we have any numbers on the overhead of

Re: propagating repo corruption across clone

2013-03-27 Thread Junio C Hamano
Sitaram Chamarty writes: > On Wed, Mar 27, 2013 at 9:17 AM, Junio C Hamano wrote: > >> To be paranoid, you may want to set transfer.fsckObjects to true, >> perhaps in your ~/.gitconfig. > > do we have any numbers on the overhead of this? > > Even a "guesstimate" will do... On a reasonably slow

Re: propagating repo corruption across clone

2013-03-26 Thread Sitaram Chamarty
On Wed, Mar 27, 2013 at 9:17 AM, Junio C Hamano wrote: > To be paranoid, you may want to set transfer.fsckObjects to true, > perhaps in your ~/.gitconfig. do we have any numbers on the overhead of this? Even a "guesstimate" will do... -- To unsubscribe from this list: send the line "unsubscribe

Re: propagating repo corruption across clone

2013-03-26 Thread Junio C Hamano
Rich Fromm writes: > Jeff King wrote >> Fundamentally the problem is >> that the --local transport is not safe from propagating corruption, and >> should not be used if that's a requirement. > > I've read Jeff Mitchell's blog post, his update, relevant parts of the > git-clone(1) man page, and a

Re: propagating repo corruption across clone

2013-03-26 Thread Jonathan Nieder
Hi, Rich Fromm wrote: >The host executing the clone > command is different than the the host on which the remote repository lives, > and I am using ssh as a transport protocol. If there is corruption, can I > or can I not expect the clone operation

Re: propagating repo corruption across clone

2013-03-26 Thread Rich Fromm
the workaround to run `git fsck` on the resulting clone? -- View this message in context: http://git.661346.n2.nabble.com/propagating-repo-corruption-across-clone-tp7580504p7580771.html Sent from the git mailing list archive at Nabble.com. -- To unsubscribe from this list: send the line "unsubsc

Re: propagating repo corruption across clone

2013-03-26 Thread Jeff King
On Tue, Mar 26, 2013 at 09:59:42PM -, Philip Oakley wrote: > Which way does `git bundle file.bundl --all` perform after the changes > for both the 'transport' checking and being reliable during updates. Bundles are treated at a fairly low level the same as a remote who provides us a particula

Re: propagating repo corruption across clone

2013-03-26 Thread Philip Oakley
From: "Jeff King" Sent: Tuesday, March 26, 2013 4:55 PM On Tue, Mar 26, 2013 at 09:43:01AM -0400, Jeff Mitchell wrote: On Mon, Mar 25, 2013 at 4:07 PM, Jeff King wrote: > On Mon, Mar 25, 2013 at 12:32:50PM -0400, Jeff Mitchell wrote: >> For commit corruptions, the --no-hardlinks, non --mirror

Re: propagating repo corruption across clone

2013-03-26 Thread Jeff King
On Tue, Mar 26, 2013 at 09:43:01AM -0400, Jeff Mitchell wrote: > On Mon, Mar 25, 2013 at 4:07 PM, Jeff King wrote: > > On Mon, Mar 25, 2013 at 12:32:50PM -0400, Jeff Mitchell wrote: > >> For commit corruptions, the --no-hardlinks, non --mirror case refused > >> to create the new repository and ex

Re: propagating repo corruption across clone

2013-03-26 Thread Jeff Mitchell
On Mon, Mar 25, 2013 at 4:07 PM, Jeff King wrote: > On Mon, Mar 25, 2013 at 12:32:50PM -0400, Jeff Mitchell wrote: >> For commit corruptions, the --no-hardlinks, non --mirror case refused >> to create the new repository and exited with an error code of 128. The >> --no-hardlinks, --mirror case spe

Re: propagating repo corruption across clone

2013-03-25 Thread Duy Nguyen
On Mon, Mar 25, 2013 at 10:56 PM, Jeff King wrote: > On Mon, Mar 25, 2013 at 10:31:04PM +0700, Nguyen Thai Ngoc Duy wrote: > >> On Mon, Mar 25, 2013 at 9:56 PM, Jeff King wrote: >> > There are basically three levels of transport that can be used on a >> > local machine: >> > >> > 1. Hard-linkin

Re: propagating repo corruption across clone

2013-03-25 Thread Jeff King
On Mon, Mar 25, 2013 at 12:32:50PM -0400, Jeff Mitchell wrote: > I think what was conflating the issue in my testing is that with > --mirror it implies --bare, so there would be checking of the objects > when the working tree was being created, hence --mirror won't show the > error a normal clone

Re: propagating repo corruption across clone

2013-03-25 Thread Jeff King
On Mon, Mar 25, 2013 at 01:01:59PM -0700, Junio C Hamano wrote: > Jeff King writes: > > > We _do_ see a problem during the checkout phase, but we don't propagate > > a checkout failure to the exit code from clone. That is bad in general, > > and should probably be fixed. Though it would never f

Re: propagating repo corruption across clone

2013-03-25 Thread Junio C Hamano
Jeff King writes: > We _do_ see a problem during the checkout phase, but we don't propagate > a checkout failure to the exit code from clone. That is bad in general, > and should probably be fixed. Though it would never find corruption of > older objects in the history, anyway, so checkout shoul

Re: propagating repo corruption across clone

2013-03-25 Thread Jeff Mitchell
On Mon, Mar 25, 2013 at 11:56 AM, Jeff King wrote: > On Mon, Mar 25, 2013 at 10:31:04PM +0700, Nguyen Thai Ngoc Duy wrote: > >> On Mon, Mar 25, 2013 at 9:56 PM, Jeff King wrote: >> > There are basically three levels of transport that can be used on a >> > local machine: >> > >> > 1. Hard-linkin

Re: propagating repo corruption across clone

2013-03-25 Thread Jeff King
On Mon, Mar 25, 2013 at 10:31:04PM +0700, Nguyen Thai Ngoc Duy wrote: > On Mon, Mar 25, 2013 at 9:56 PM, Jeff King wrote: > > There are basically three levels of transport that can be used on a > > local machine: > > > > 1. Hard-linking (very fast, no redundancy). > > > > 2. Byte-for-byte cop

Re: propagating repo corruption across clone

2013-03-25 Thread Duy Nguyen
On Mon, Mar 25, 2013 at 9:56 PM, Jeff King wrote: > There are basically three levels of transport that can be used on a > local machine: > > 1. Hard-linking (very fast, no redundancy). > > 2. Byte-for-byte copy (medium speed, makes a separate copy of the > data, but does not check the int

Re: propagating repo corruption across clone

2013-03-25 Thread Jeff King
On Mon, Mar 25, 2013 at 09:43:23AM -0400, Jeff Mitchell wrote: > > But I haven't seen exactly what the corruption is, nor exactly what > > commands they used to clone. I've invited the blog author to give more > > details in this thread. > > The syncing was performed via a clone with git clone --

Re: propagating repo corruption across clone

2013-03-25 Thread Jeff Mitchell
On Sun, Mar 24, 2013 at 3:23 PM, Jeff King wrote: > On Sun, Mar 24, 2013 at 08:01:33PM +0100, Ævar Arnfjörð Bjarmason wrote: > >> On Sun, Mar 24, 2013 at 7:31 PM, Jeff King wrote: >> > >> > I don't have details on the KDE corruption, or why it wasn't detected >> > (if it was one of the cases I me

Re: propagating repo corruption across clone

2013-03-24 Thread Ilari Liusvaara
On Sun, Mar 24, 2013 at 02:31:33PM -0400, Jeff King wrote: > > Fscking the incoming objects does work, but of course it comes at a cost > in the normal case (for linux-2.6, I measured an increase in CPU time > with "index-pack --strict" from ~2.5 minutes to ~4 minutes). And I think > it is probabl

Re: propagating repo corruption across clone

2013-03-24 Thread Jeff King
On Sun, Mar 24, 2013 at 08:01:33PM +0100, Ævar Arnfjörð Bjarmason wrote: > On Sun, Mar 24, 2013 at 7:31 PM, Jeff King wrote: > > > > I don't have details on the KDE corruption, or why it wasn't detected > > (if it was one of the cases I mentioned above, or a more subtle issue). > > One thing wor

Re: propagating repo corruption across clone

2013-03-24 Thread Ævar Arnfjörð Bjarmason
On Sun, Mar 24, 2013 at 7:31 PM, Jeff King wrote: > > I don't have details on the KDE corruption, or why it wasn't detected > (if it was one of the cases I mentioned above, or a more subtle issue). One thing worth mentioning is this part of the article: "Originally, mirrored clones were in fact

propagating repo corruption across clone

2013-03-24 Thread Jeff King
I saw this post-mortem on recent disk corruption seen on git.kde.org: http://jefferai.org/2013/03/24/too-perfect-a-mirror/ The interesting bit to me is that object corruption propagated across a clone (and oddly, that --mirror made complaints about corruption go away). I did a little testing an