[9fans] du vs. ls: duplication or not?

2012-01-14 Thread tlaronde
This is a slightly different point of view from the thread my blunders have started. du(1) stands for Disk Usage. But does this, with the "historical" use, has any real sense now, with a namespace that can be, not only built by presenting a same file via different paths, but a combination of local

Re: [9fans] du vs. ls: duplication or not?

2012-01-14 Thread Charles Forsyth
Du answers the question: (roughly) how big are these files (the files in these directories)? It still does that reasonably well in Plan 9 (I'm not ruling out possible improvements, but "it works for me!"). It doesn't answer questions about physical storage. On my Linux machines, I do use it as a gu

Re: [9fans] du vs. ls: duplication or not?

2012-01-14 Thread David du Colombier
Russ Cox did a recursive ls few years ago. http://swtch.com/lsr.c I use a slightly modified version which can display most Dir attributes. -- David du Colombier

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread erik quanstrom
On Fri Jan 13 18:24:59 EST 2012, ara...@mgk.ro wrote: > On Sat, Jan 14, 2012 at 12:14 AM, Francisco J Ballesteros > wrote: > > but if you insert extra music in front of your track dedup in venti won't > > help. > > or would it? > > It wouldn't. In practice it seems that it usually appends, prob

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread erik quanstrom
> given the fact that most disks are very large, and most people's non-media > storage requirements are very small, why is the compelling. shoot. ready. aim. i ment, "why is this compelling?". sorry. - erik

Re: [9fans] du vs. ls: duplication or not?

2012-01-14 Thread erik quanstrom
> du(1) stands for Disk Usage. But does this, with the "historical" use, > has any real sense now, with a namespace that can be, not only built by > presenting a same file via different paths, but a combination of local > storage (including memory one...) and remote storage? i agree with charles.

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread erik quanstrom
> 0) > use of rolling-checksum enables decent block-level deduplication on files > that > are modified in the middle; some info: > http://svana.org/kleptog/rgzip.html > http://blog.kodekabuki.com/post/11135148692/rsync-internals > > in short, a rolling checksum is used to find reasonable restart

[9fans] Plan9port GUI applications don't work in Nx session

2012-01-14 Thread Aram Hăvărneanu
I have a Windows machine that runs an X server and a Linux machine running plan9port. When I use the Windows machine to connect to the Linux machine via ssh, with X forwarding enabled, plan9port GUI applications like acme and sam work perfectly. In this case the environment is: SSH_CLIENT=19

Re: [9fans] Plan9port GUI applications don't work in Nx session

2012-01-14 Thread Aram Hăvărneanu
> I don't know if this is an X error, or a Plan9 environment error. If I set $NAMESPACE before starting acme, I only get the X errors, so I guess the problem is X related. -- Aram Hăvărneanu

Re: [9fans] Plan9port GUI applications don't work in Nx session

2012-01-14 Thread Aram Hăvărneanu
>> I don't know if this is an X error, or a Plan9 environment error. > > If I set $NAMESPACE before starting acme, I only get the X errors, so > I guess the problem is X related. I upgraded to the latest, beta, version of the commercial Nx server and the problem went away. I guess the proxy imple

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread hiro
venti is too big, buy bigger disks and forget venti.

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread Charles Forsyth
Although drives are larger now, even SSDs, there is great satisfaction in being able to make copies of large trees arbitrarily, without having to worry about them adding any more than just the changed files to the write-once stored set. I do this fairly often during testing. On 14 January 2012 15:

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread erik quanstrom
On Sat Jan 14 10:07:25 EST 2012, charles.fors...@gmail.com wrote: > Although drives are larger now, even SSDs, there is great satisfaction in > being able to make copies of large trees arbitrarily, without having to > worry about them adding any more than just the changed files to the > write-once

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread Aram Hăvărneanu
erik quanstrom wrote: > i think it would be fair to argue that source and executables are > negligeable users of storage.  media files, which are already compressed, > tend to dominate. What about virtual machine images? > the tradeoff for this compression is a large amount of memory, > fragmenta

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread erik quanstrom
> What about virtual machine images? > > > the tradeoff for this compression is a large amount of memory, > > fragmentation, and cpu usage.  that is to say, storage latency. > > I have 24GB RAM. My primary laptops have 8GB RAM. I have all this RAM > not because of dedup but because I do memory in

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread Aram Hăvărneanu
> russ posted some notes how how much memory and disk bandwidth are > required to write at a constant b/w of Xmb/s to venti.  venti requires > enormous resources to perform this capability. Maybe, I was talking generally about the concept of content-addressable storage, not venti in particular. I

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread Charles Forsyth
That only affects the directories, which are tiny, not the files. On 14 January 2012 15:29, erik quanstrom wrote: > (as an aside, one assumes changed files + directory tree as the > a/mtimes are changed.) >

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread erik quanstrom
> Maybe, I was talking generally about the concept of > content-addressable storage, not venti in particular. I believe it's > possible to do CAS without a major performance hit, look at ZFS, for > example. do you have any reference to ZFS being content-addressed storage? > >> I'm of a completely

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread Aram Hăvărneanu
> do you have any reference to ZFS being content-addressed storage? It's not purely content-addressed storage, but it implements deduplication in the same way venti does: http://blogs.oracle.com/bonwick/entry/zfs_dedup (the blog post offers only a high level overview, you have to dig into the code

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread erik quanstrom
On Sat Jan 14 16:40:46 EST 2012, ara...@mgk.ro wrote: > > do you have any reference to ZFS being content-addressed storage? > > It's not purely content-addressed storage, but it implements > deduplication in the same way venti does: > http://blogs.oracle.com/bonwick/entry/zfs_dedup (the blog post

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread Aram Hăvărneanu
> content addressed means given the content, you can generate the address. > this is NOT true of zfs at all. How come? With venti, the address is the SHA-1 hash, with ZFS, you get to chose the hash, but it can still be a hash. > you keep changing the subject.  your original claim was that random

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread erik quanstrom
On Sat Jan 14 17:12:49 EST 2012, ara...@mgk.ro wrote: > > content addressed means given the content, you can generate the address. > > this is NOT true of zfs at all. > > How come? With venti, the address is the SHA-1 hash, with ZFS, you get > to chose the hash, but it can still be a hash. becaus

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread Aram Hăvărneanu
>> How come? With venti, the address is the SHA-1 hash, with ZFS, you get >> to chose the hash, but it can still be a hash. > > because in zfs the hash is not used as an address (lba). But by this definition neither is venti. In venti, the hash is translated to a lba by the index cache. In ZFS, th

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread Bakul Shah
On Sat, 14 Jan 2012 17:42:12 EST erik quanstrom wrote: > On Sat Jan 14 17:12:49 EST 2012, ara...@mgk.ro wrote: > > > content addressed means given the content, you can generate the address. > > > this is NOT true of zfs at all. > > > > How come? With venti, the address is the SHA-1 hash, with ZF

Re: [9fans] fossil pb: FOUND!

2012-01-14 Thread Aram Hăvărneanu
>> > How come? With venti, the address is the SHA-1 hash, with ZFS, you get >> > to chose the hash, but it can still be a hash. >> >> because in zfs the hash is not used as an address (lba). > > True. As I said, neither in Venti. The hash is translated to a lba by the index, a table that can be re