Re: [PERFORM] PG-related ACM Article: "The Pathologies of Big Data"

2009-08-08 Thread Pierre Frédéric Caillau d
I don't see how on any recent hardware, random access to RAM is slower than sequential from disk.  RAM access, random or not, is measured in GB/sec... I don't think anybody's arguing that. http://www.anandtech.com/cpuchipsets/showdoc.aspx?i=2795&p=5 These guys mention about 50 ns memory

Re: [PERFORM] PG-related ACM Article: "The Pathologies of Big Data"

2009-08-07 Thread Scott Marlowe
On Fri, Aug 7, 2009 at 7:34 PM, Scott Carey wrote: > Well, there is CPU overhead for reading postgres pages and tuples.  On a > disk subsystem that gets 1GB/sec sequential reads, I can't get more than > about 700MB/sec of I/O and on a select count(*) query on very large tables > with large rows (60

Re: [PERFORM] PG-related ACM Article: "The Pathologies of Big Data"

2009-08-07 Thread Scott Carey
Well, there is CPU overhead for reading postgres pages and tuples. On a disk subsystem that gets 1GB/sec sequential reads, I can't get more than about 700MB/sec of I/O and on a select count(*) query on very large tables with large rows (600 bytes) and its closer to 300MB/sec if the rows are smalle

Re: [PERFORM] PG-related ACM Article: "The Pathologies of Big Data"

2009-08-07 Thread Scott Marlowe
Oh I just noticed his graphic is "values per second" but he had originally said they were 16 bit values. Even if they were 32 or 64 bit values, I'd expect way more than what he's getting there. On Fri, Aug 7, 2009 at 6:40 PM, Scott Marlowe wrote: > Well, from the article, I got the feeling he nev

Re: [PERFORM] PG-related ACM Article: "The Pathologies of Big Data"

2009-08-07 Thread Scott Marlowe
On Fri, Aug 7, 2009 at 2:17 PM, Josh Kupershmidt wrote: > Just stumbled across this recent article published in the > Communications of the ACM: > > http://cacm.acm.org/magazines/2009/8/34493-the-pathologies-of-big-data/fulltext > > The author shares some insights relating to difficulties processin

Re: [PERFORM] PG-related ACM Article: "The Pathologies of Big Data"

2009-08-07 Thread Greg Stark
On Fri, Aug 7, 2009 at 9:17 PM, Josh Kupershmidt wrote: > Just stumbled across this recent article published in the > Communications of the ACM: > > http://cacm.acm.org/magazines/2009/8/34493-the-pathologies-of-big-data/fulltext > > The author shares some insights relating to difficulties processin