On Sun, May 27, 2012 at 11:42:55AM -0400, Joey Hess wrote: > maximilian attems wrote: > > Trying out git annex on a rather large collection (~200k taking 120Gb) > > of bzip2 data snapshot files. In contrast to other git commands git annex > > operations seem not to properly scale to that number of files. > > What kind of operations?
any, ranging from git commit to git annex add, even git annex status is taking ages. > The limiting factor is typically the speed of things like git ls-files. well, I think that the git annex format needs a deep rethink to scale. I do prefer wasting disk space (for useless bz2 git tracking) in order to have everyday's git speed for adding new data and pushing it out. git subtree seems a good new solution for splitting out the newest and most interesting data chunks for laptop processing. happy hacking -- maks -- To UNSUBSCRIBE, email to [email protected] with a subject of "unsubscribe". Trouble? Contact [email protected]
