Re: git fsck failure on OS X with files >= 4 GiB

2015-10-29 Thread Atousa Duprat
Thank you for the feedback. I have revised the proposed patch as suggested, allowing the use of SHA1_MAX_BLOCK_SIZE to enable the chunked implementation. When building for OSX with the CommonCrypto library we error out if SHA1_MAX_BLOCK_SIZE is not defined, which will avoid compiling a version of

Re: git fsck failure on OS X with files >= 4 GiB

2015-10-29 Thread Junio C Hamano
Atousa Duprat writes: > [PATCH] Limit the size of the data block passed to SHA1_Update() > > This avoids issues where OS-specific implementations use > a 32-bit integer to specify block size. Limit currently > set to 1GiB. > --- > cache.h | 20 +++- > 1 file changed, 19 insertio

Re: git fsck failure on OS X with files >= 4 GiB

2015-10-29 Thread Atousa Duprat
Here is a solution that avoids problems with OS-specific implementations of SHA_Update() by limiting the size of each update request to 1GiB and calling the function repeatedly in a loop. Atousa -- [PATCH] Limit the size of the data block passed to SHA1_Update() This avoids issues where OS-spec

Re: git fsck failure on OS X with files >= 4 GiB

2015-10-29 Thread Filipe Cabecinhas
Defining BLK_SHA1 = YesPlease (when calling make) should just change the SHA functions, instead of completely removing OpenSSL or CommonCrypto. Regards, Filipe On Thu, Oct 29, 2015 at 3:46 AM, Rafael Espíndola wrote: > Awesome, building with > > NO_OPENSSL = 1 > NO_GETTEXT = 1 > > produces a

Re: git fsck failure on OS X with files >= 4 GiB

2015-10-29 Thread Rafael Espíndola
Awesome, building with NO_OPENSSL = 1 NO_GETTEXT = 1 produces a working git :-) Cheers, Rafael On 28 October 2015 at 23:37, Filipe Cabecinhas wrote: > I did some debugging, and it seems CC_SHA1_Update (used by > write_sha1_file_prepare if APPLE_COMMON_CRYPTO is defined in the Makefile) > take

Re: git fsck failure on OS X with files >= 4 GiB

2015-10-28 Thread Filipe Cabecinhas
I did some debugging, and it seems CC_SHA1_Update (used by write_sha1_file_prepare if APPLE_COMMON_CRYPTO is defined in the Makefile) takes a uint32_t as a "length" parameter, which explains why it stops working at 4GiB (UINT_MAX+1). In the OS X 10.11 SDK header CommonCrypto/CommonDigest.h, we hav

git fsck failure on OS X with files >= 4 GiB

2015-10-28 Thread Rafael Espíndola
I first noticed this with "2.4.9 (Apple Git-60)", but it reproduces with git built from 37023ba381b6d251d7140a997b39b566dbc63c42. Create two files with just 0s: -rw-r--r-- 1 espindola staff 4294967296 28 Oct 11:09 exactly-4gib -rw-r--r-- 1 espindola staff 4294967295 28 Oct 11:09 one-less-th