Re: [PERFORM] Fw: [osdldbt-general] Running DBT5 on remote database server

2013-07-31 Thread Greg Smith
way as this is normally done with Postgres. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your sub

Re: [PERFORM] PostgreSQL settings for running on an SSD drive

2013-07-17 Thread Greg Smith
On 6/20/13 4:32 PM, Josh Berkus wrote: First, cc'ing Greg Smith to see if he can address this with the Fusion folks so that they stop giving out a bad guide. I'm working on a completely replacement of that guide, one that actually gives out a full set of advice. Right now I'

[PERFORM] Re: bgwriter autotuning might be unnecessarily penalizing bursty workloads

2013-07-17 Thread Greg Smith
r. Sample from a busy system with moderately tuned BGW and checkpoint_timeout at 15 minutes: -[ RECORD 1 ]---+--- alloc_mbps | 246.019686474412 checkpoint_mbps | 0.0621780475463596 clean_mbps | 2.38631188442859 backend_mbps| 0.777490109599045 write_mbps | 3.2

Re: [PERFORM] Check Pointer

2013-05-30 Thread Greg Smith
what you're seeing there. It doesn't actually use any significant amount of memory on its own. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pg

Re: [PERFORM] pgbench: spike in pgbench results(graphs) while testing pg_hint_plan performance

2013-05-24 Thread Greg Smith
. -Does this happen on every test run? Is it at the same time? -You can run "top -bc" to dump snapshots of what the system is doing every second. With some work you can then figure out what was actually happening during the two seconds around when the throughput dropped. --

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-24 Thread Greg Smith
es-and-ncq/ With that loose end sorted, I'll be kicking off a brand new round of SSD tests on a 24 core server here soon. All those will appear on my blog. The 320 drive is returning as the bang for buck champ, along with a DC S3700 and a Seagate 1TB Hybrid drive with NAND durable write

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
es to flash wear or bytes written, either. Several of the SMART attributes are labeled "Vendor-specific," but you'll need to guess what they track and read the associated values using third-party software." That's a serious problem for most business use of this s

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
better protocol for handling things. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http:

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
efore abandoning it altogether. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
et, and I haven't recommended anyone buy a 710 since the announcement. However, "hit the street" is still an issue. No one has been able to keep DC S3700 drives in stock very well yet. It took me three tries through Newegg before my S3700 drive actually shipped. -- Greg Smi

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
de 710 drives that aim for 900TB instead. But I do have standby servers using the low grade stuff, so anything I can do to decrease SSD burn rate without dropping performance is useful. And only the top tier of transaction rates will outrun a RAID1 pair of 15K drives dedicated to WAL. --

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
them on traditional drives can be cost effective in a few ways. That approach is lucky to hit 10K TPS though, so it can't compete against what a PCI-E card like the FusionIO drives are capable of. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Ser

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
gh, so I'm still waiting for more data before I feel comfortable saying exactly what the worst case looks like. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-22 Thread Greg Smith
almost free it's so fast? Sure. But presuming all systems will look like that is optimistic, and it sets unreasonable expectations. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgs

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-21 Thread Greg Smith
s. My test server's 100GB DC S3700 was $250. That's still not two orders of magnitude faster though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing li

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-19 Thread Greg Smith
thing that's happened since the last base backup happened. That can easily result in a week of downtime if you're only shipping backups once per month, for example. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Suppor

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

2013-05-19 Thread Greg Smith
flushed to disk. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.or

Re: [PERFORM] Deleting Rows From Large Tables

2013-05-19 Thread Greg Smith
t this type of work going on in the background. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subs

Re: [PERFORM] Hardware suggestions for maximum read performance

2013-05-19 Thread Greg Smith
systems are relatively small. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.p

Re: [PERFORM] sniff test on some PG 8.4 numbers

2013-03-10 Thread Greg Smith
ash and leave it on. Purchasing and management people tend to get annoyed if they discover the core count of the server is half what they thought they were buying. The potential downside of HT isn't so big that its worth opening that can of worms, unless you've run real application lev

Re: [PERFORM] New server setup

2013-03-10 Thread Greg Smith
safely and as fast as it should be, 2ndQuadrant does offer a hardware benchmarking service to do that sort of thing: http://www.2ndquadrant.com/en/hardware-benchmarking/ I think we're even generating those reports in German now. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Bal

Re: [PERFORM] sniff test on some PG 8.4 numbers

2013-03-10 Thread Greg Smith
ake a look at http://www.postgresql.org/support/versioning/ and you'll see 8.4 only has a little over a year before it won't get bug fixes anymore. Also, your server would really appreciate the performance gains added to 9.2. If that's a bit too leading edge for you, I don'

Re: [PERFORM] pgbench intriguing results: better tps figures for larger scale factor

2013-03-04 Thread Greg Smith
nge. Nowadays there really is no reason to consider running pgbench on a system with a smaller scale than that. I normally get a rough idea of things by running with scales 100, 250, 500, 1000, 2000. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training

Re: [PERFORM] exponential performance decrease in ISD transaction

2012-09-05 Thread Greg Smith
; line in /proc/meminfo as the test runs will give you an idea if write cache filling is actually an issue here. If that number just keeps going up and speeds keep on dropping, that's at least one cause here. This could easily be both that and an autovacuum related too thoug

Re: [PERFORM] [repost] Help me develop new commit_delay advice

2012-09-05 Thread Greg Smith
delay needed to vary a bit based on the number of the siblings was one of the challenges I kept butting into then. Making the GUC settings even more complicated for this doesn't seem a productive step forward for the average user. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com

Re: [PERFORM] ZFS vs. UFS

2012-07-26 Thread Greg Smith
how can I create a real-time replica of this data?" to protect databases, and the standby server for that doesn't need to be an expensive system. That said, there is no reason to set things up so that they only work with that Intel RAID controller, given that it's not a very good

[PERFORM] Linux memory zone reclaim

2012-07-17 Thread Greg Smith
ermittently, you'll see one incredibly slow query periodically while most are fast. All depends on exactly what core is executing, where the memory it needs is at, and whether the server wants to reclaim memory (and just what that means its own complicated topic) as part of tha

Re: [PERFORM] MemSQL the "world's fastest database"?

2012-07-05 Thread Greg Smith
their TPS numbers are useless without a contest of how big each transaction is, and we don't know. I can take MemSQL seriously when there's a press release describing how to replicate their benchmark independently. Then it's useful to look at the absolute number. -- Greg Smith

Re: [PERFORM] The need for clustered indexes to boost TPC-V performance

2012-07-05 Thread Greg Smith
of writing that feature?" and "how can we get them sponsored to focus on it?" I can tell from your comments yet what role(s) in that process VMWare wants to take on internally, and which it's looking for help with. The job of convincing people it's a useful feature isn&

Re: [PERFORM] Introducing the TPC-V benchmark, and its relationship to PostgreSQL

2012-07-05 Thread Greg Smith
was CPU bound, that would be an awesome world to live in. Anyway, guessing at causes here is premature speculation. When there's some code for the test kit published, at that point discussing the particulars of why it's not running well will get interesting. -- Greg Smith 2ndQua

Re: [PERFORM] SSD, Postgres and safe write cache

2012-06-30 Thread Greg Smith
t toward leading edge SSD only to discover their real working set fits in memory just fine, so it doesn't even matter. Or that the bottleneck is somewhere else entirely. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support ww

Re: [PERFORM] MemSQL the "world's fastest database"?

2012-06-30 Thread Greg Smith
e type MySQL is usually faster than PostgreSQL on (i.e. ones using simple operations and operators), or they could be ones where PostgreSQL is usually faster than MySQL (i.e. more complicated joins). All I can tell you for sure if that they used a query mix that makes MemSQL look much fast

Re: [PERFORM] SSD selection

2012-05-28 Thread Greg Smith
IOPS of writes that are done more efficiently in terms of flash longevity (710). You can't get both at the same time. The 710 may ultimately throttle its speed back to meet lifetime specifications as the drive fills, it's really hard to benchmark the differences between the tw

Re: [PERFORM] Configuration Recommendations

2012-04-25 Thread Greg Smith
performance, consider running your own more application-like benchmarks instead. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org

Re: [PERFORM] how to demonstrate the effect of direct I/O ?

2012-02-07 Thread Greg Smith
ented acceleration here and b) on Solaris. You won't find a compelling performance improvement listed at https://ext4.wiki.kernel.org/articles/c/l/a/Clarifying_Direct_IO%27s_Semantics_fd79.html and Linux has generally ignored direct I/O as something important to optimize for. -- Greg Smith 2ndQ

Re: [PERFORM] wal_level=archive gives better performance than minimal - why?

2012-01-16 Thread Greg Smith
chive archive To make sure the difference wasn't some variation on "gets slower after each run". pgbench suffers a lot from problems in that class. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

Re: [PERFORM] auto vacuum, not working?

2012-01-15 Thread Greg Smith
jects/reorg/ -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

[PERFORM] Slow nested loop execution on larger server

2011-12-16 Thread Greg Smith
15:39:55.489344 semop(16777323, 0x7fffd09dac00, 1) = 0 Process 12634 detached pg_locks for this 12634 shows all granted ones, nothing exciting there. I asked how well this executes with enable_nestloop turned off, hoping to see that next. This all seems odd, and I get interested and concerned w

Re: [PERFORM] SSD endurance calculations

2011-11-23 Thread Greg Smith
eliminate several of them. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.post

Re: [PERFORM] Benchmarking tools, methods

2011-11-19 Thread Greg Smith
ng, you may not find as much about ZFS tuning and SSDs as you'd like though. http://www.2ndquadrant.com/en/talks/ has some updated material about things discovered since the book was published. The "Bottom-Up Database Benchmarking" there shows the tests I'm running nowadays

Re: [PERFORM] SSD options, small database, ZFS

2011-11-18 Thread Greg Smith
sons I keep so many samples on that program's page is to help people navigate this whole maze, and have some data points to set expectations against. See https://github.com/gregs1104/stream-scaling for the code and the samples. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Bal

Re: [PERFORM] What's the state of postgresql on ext4 now?

2011-11-15 Thread Greg Smith
aluate RHEL6 to see how it's doing soon, just haven't had time/requests for it. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performanc

Re: [PERFORM] Slow queries / commits, mis-configuration or hardware issues?

2011-11-15 Thread Greg Smith
onds on one of their servers if I try to make that problem bad, you're only seeing the middle range of latency issues so far. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-perfor

Re: [PERFORM] What's the state of postgresql on ext4 now?

2011-11-15 Thread Greg Smith
y the ext4 changes. In just about every other way but commit performance, ext4 is faster than most other filesystems. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us

Re: [PERFORM] WAL partition filling up after high WAL activity

2011-11-11 Thread Greg Smith
t now there's no mention of that parameter in the section that talks about parameters to configure; there really should be. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-perform

Re: [PERFORM] WAL partition filling up after high WAL activity

2011-11-09 Thread Greg Smith
ts=16 is a low setting. You might as well set it to a large number, say 128, and let checkpoints get driven by time instead. The existing limit isn't working effectively anyway, and having more segments lets the checkpoint spreading code work more evenly. -- Greg Smith 2ndQuadrant US

Re: [PERFORM] Composite keys

2011-10-12 Thread Greg Smith
plenty of times that the reason behind "why isn't it using my index?" is "the index is too fat to navigate efficiently", because the actual number of blocks involved is factored into the cost computations. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimor

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-11 Thread Greg Smith
e trouble. I did a few blog entries and mailing list posts about this earlier this year, and that was as much documentation as I could justify at the time. If there's a user-visible behavior changes here, that's the point where an update to the manual would be in order. -- Greg Smi

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Greg Smith
ere have been beating out improvements made to the database fast enough that it's hard to innovate in this area within Postgres. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-pe

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Greg Smith
7;d have to put a kernel bug on the list of suspects with this unusual vmstat output. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@p

Re: [PERFORM] Adding more memory = hugh cpu load

2011-10-10 Thread Greg Smith
x27;s hard to do that in the position you're in now though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make chan

Re: [PERFORM] : PG9.0 - Checkpoint tuning and pg_stat_bgwriter

2011-10-04 Thread Greg Smith
ose things. The advantage to making them happen less often is that you get less total writes. People need to be careful about going a long *time* between checkpoints. But there's very few cases where you need to worry about the segment count going too high before another one is trigg

Re: [PERFORM] : PG9.0 - Checkpoint tuning and pg_stat_bgwriter

2011-10-04 Thread Greg Smith
useful. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [PERFORM] Suggestions for Intel 710 SSD test

2011-10-02 Thread Greg Smith
-scaling but it's still quite rough to use. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us <> -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to you

Re: [PERFORM] PostgreSQL-9.0 Monitoring System to improve performance

2011-09-28 Thread Greg Smith
and I haven't come across someone yet who wants to fund that size of project for this purpose yet. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-p

Re: [PERFORM] Query optimization using order by and limit

2011-09-21 Thread Greg Smith
more pieces, and see if one of them results in the query not excluding partitions anymore or not. I can't figure out if you're running into a basic error here, where constraint exclusion just isn't working at all, or if you are only having this problem because the query is too

Re: [PERFORM] : Performance Improvement Strategy

2011-09-21 Thread Greg Smith
to do that. A multi-gigabyte table can easily be unavailable for several hours if you execute VACUUM FULL against it. CLUSTER is almost always faster. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us --

Re: Show_database_bloat reliability? [was: Re: [PERFORM] REINDEX not working for wastedspace]

2011-09-21 Thread Greg Smith
determining if bloat is likely increasing or decreasing--which is the purpose of that query. The value returned is a rough estimate, and should not be considered useful as any sort of absolute measurement. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training

Re: [PERFORM] raid array seek performance

2011-09-14 Thread Greg Smith
r what people really want to know nowadays. That's going live to the world at the end of the month, at #PgWest: http://pgwest2011.sched.org/event/875b87d8d237bef3a53ab27ac9c8057c -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24

Re: [PERFORM] RAID Controller (HP P400) beat by SW-RAID?

2011-09-14 Thread Greg Smith
lot of your test results seem like they may be using different levels of write reliability, which makes things less fair than they should be too--in favor of the cheap IDE drives normally. Check out http://wiki.postgresql.org/wiki/Reliable_Writes for more information about that topic.

Re: [PERFORM] how delete/insert/update affects select performace?

2011-09-08 Thread Greg Smith
the shared_buffers parameters on Windows--no more than 512MB. That makes your server a bit less likely to run in the nasty checkpoint spike issues Kevin was alluding to. I don't think we've seen any reports of that on Windows. The problem is worst on Linux. -- Greg Smith 2ndQuadra

Re: [PERFORM] Intel 320 SSD info

2011-08-24 Thread Greg Smith
On 08/24/2011 01:42 PM, David Boreham wrote: On 8/24/2011 11:41 AM, Greg Smith wrote: I've measured the performance of this drive from a couple of directions now, and it always comes out the same. For PostgreSQL, reading or writing 8K blocks, I'm seeing completely random worklo

Re: [PERFORM] Intel 320 SSD info

2011-08-24 Thread Greg Smith
#x27;s just over 2500 IOPS. It's quite possible that number can go lower under pressure of things like internal drive garbage collection however, which I believe is going into the 600 IOPS figure. I haven't tried to force that yet--drive is too useful to me to try and burn it out

Re: [PERFORM] How to track number of connections and hosts to Postgres cluster

2011-08-24 Thread Greg Smith
h as looking at pg_stat_activity, will sometimes miss one. If you're willing to lose a connection sometimes, a cron job that polls pg_stat_activity and saves a summary of what it finds will normally use less resources. But connections that start and end between runs will be missed. -- Greg Sm

Re: [PERFORM] RAID Controllers

2011-08-23 Thread Greg Smith
, but it sounds like you're not planning to run one of those anyway. If I had 24 drives to connect, I'd prefer an LSI controller just because I know those scale fine to that level; I'm not sure how well Adaptec does there. Haven't found anyone brave enough to try that te

Re: [PERFORM] settings input for upgrade

2011-08-22 Thread Greg Smith
together in the future, that may be an option for improving performance one day. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us

Re: [PERFORM] settings input for upgrade

2011-08-21 Thread Greg Smith
t to find slow lock issues by turning on log_lock_waits, you're just going to put it right back to the default again--or lower it. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us

Re: [PERFORM] Reports from SSD purgatory

2011-08-19 Thread Greg Smith
On 08/15/2011 07:49 PM, Greg Smith wrote: News update for anyone else who's trapped like me, waiting for a fix to the Intel 320 SSD bug where they can truncate themselves to 8MB. Over the weekend Intel has announced a firmware fix for the problem is done, and is due to ship "withi

Re: [PERFORM] Raid 5 vs Raid 10 Benchmarks Using bonnie++

2011-08-17 Thread Greg Smith
barrier" just because I'm used to that name on XFS systems. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make

Re: [PERFORM] Raid 5 vs Raid 10 Benchmarks Using bonnie++

2011-08-17 Thread Greg Smith
esults. They should be between the ext3 and the XFS results, not far lower than either. Normally this only comes from using a bad set of mount options. With a battery-backed write cache, you'd want to use "nobarrier" for example; if you didn't do that, that can cru

[PERFORM] Reports from SSD purgatory

2011-08-15 Thread Greg Smith
The most common source of dead drives is sudden, catastrophic electronics failure. These are not predicted by SMART, and have nothing to do with hitting the drive's wear limits. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24

Re: [PERFORM] strange pgbench results (as if blocked at the end)

2011-08-15 Thread Greg Smith
hit as hard yet by users. My paranoid raving about this particular bug isn't strongly supported; I'll concede that with the additional data you've provided. But I don't think it's completely unfounded either. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltim

Re: [PERFORM] strange pgbench results (as if blocked at the end)

2011-08-14 Thread Greg Smith
;t assure you'll a checkpoint begin and end in all circumstances, but it is the expected case. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-perfor

Re: [PERFORM] strange pgbench results (as if blocked at the end)

2011-08-12 Thread Greg Smith
kernels to provide this more efficient mechanism of ensuring ordering...these future block layer improvements will change some kernel interfaces..." Yikes, that does not inspire confidence to me. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us

Re: [PERFORM] Recommended optimisations slows down PostgreSQL 8.4

2011-08-12 Thread Greg Smith
rom them is no fun. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.

Re: [PERFORM] Recommended optimisations slows down PostgreSQL 8.4

2011-08-11 Thread Greg Smith
read/written once, so the database cache is no better than the OS one). -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make cha

Re: [PERFORM] Suspected Postgres Datacorruption

2011-08-08 Thread Greg Smith
the situation, the version of PostgreSQL you use won't matter too much--the database will still be unreliable if the hardware is configured to do the wrong thing. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant

Re: [PERFORM] Postgres 8.4 memory related parameters

2011-08-08 Thread Greg Smith
ans ">8GB" right now. Note that the problems you can run into with too much buffer cache are much worse with a low setting for checkpoint_segments...and this configuration doesn't change it at all from the tiny default. That should go to at least 64 on a server this size.

Re: [PERFORM] Postgres performance on Linux and Windows

2011-08-08 Thread Greg Smith
x27;s commentary about network speed heavily influencing results. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make chan

Re: [PERFORM] benchmark woes and XFS options

2011-08-08 Thread Greg Smith
ly need to separate out "bad driver" from the other possibilities here given what you've described, and that's a low impact way to do it. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us

Re: [PERFORM] synchronous_commit off

2011-08-01 Thread Greg Smith
disk before returning to the client, but all the validation checks are done before that. The sole risk with synchronous_commit off is that a client will get COMMIT, but the server will lose the transaction completely--when there's a crash before it's written to disk. --

[PERFORM] Intel 320 series drives firmware bug

2011-07-21 Thread Greg Smith
newsroom/blog/2011/05/19/chip-shot-new-5-year-limited-warranty-on-intel-ssd-320 We just need to figure out how/where they're drawing the "enterprise usage levels" line at. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD -- Sent via pgsql-performance mailing

Re: [PERFORM] BBU still needed with SSD?

2011-07-19 Thread Greg Smith
8 18184 - 18159 = 25; exactly the count I used in 32MB blocks. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] BBU still needed with SSD?

2011-07-19 Thread Greg Smith
d up commits--you will only see minimal benefit from with these SSDs. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] cpu comparison

2011-07-18 Thread Greg Smith
ould happen is that total speed keeps going up as you add more threads on the newer system, while the old DDR2 model stays as the same basic total. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD -- Sent via pgsql-performance mailing list (pgsql-performance@postgres

Re: [PERFORM] BBU still needed with SSD?

2011-07-18 Thread Greg Smith
ssage-id/4d9d1fc3.4020...@2ndquadrant.com And there's a larger discussion of this topic at http://blog.2ndquadrant.com/en/2011/04/intel-ssd-now-off-the-sherr-sh.html that answers this question in a bit more detail. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD -

Re: [PERFORM] UPDATEDs slowing SELECTs in a fully cached database

2011-07-17 Thread Greg Smith
lucky one to find it free of a write backlog. So the best case is much better than a typical spinning drive with no such cache. The worst case is in the 100ms+ range though on EBS. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD -- Sent via pgsql-performance mailing

Re: [PERFORM] Hardware advice for scalable warehouse db

2011-07-15 Thread Greg Smith
ays gotten seriously good performance relative to the dollars spent with that series. Only one of these Dell storage arrays I've heard two disappointing results from (but not tested directly yet) is the MD3220. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD -- Sent

Re: [PERFORM] "VACUUM FULL ANALYZE" vs. Autovacuum Contention

2011-07-08 Thread Greg Smith
e), this can work much faster than a normal load. It's able to skip the pg_xlog WAL writes in this situation. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD Comprehensive and Customized PostgreSQL Training Classes: http://www.2ndquadrant.us/postgresql-training/ --

Re: [PERFORM] "VACUUM FULL ANALYZE" vs. Autovacuum Contention

2011-07-07 Thread Greg Smith
djust how much work autovacuum does at any moment in time, it isn't involved in any prioritization. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD Comprehensive and Customized PostgreSQL Training Classes: http://www.2ndquadrant.us/postgresql-training/ -- Sent via pgsq

Re: [PERFORM] bitmask index

2011-07-05 Thread Greg Smith
to coin a new acronym for: those are part of the Most Important Values in that column. The MIV set is the MCV information plus information about the rare but critical columns. And the easiest way to expose that data to the planner is with a partial index. I smell a blog post coming on th

Re: [PERFORM] Postgres bulkload without transaction logs

2011-07-05 Thread Greg Smith
l for this case is http://pgbulkload.projects.postgresql.org/ -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD Comprehensive and Customized PostgreSQL Training Classes: http://www.2ndquadrant.us/postgresql-training/ -- Sent via pgsql-performance mailing list (pg

Re: [PERFORM] Infinite Cache

2011-07-03 Thread Greg Smith
k transparently the application is a nice benefit of EDB's commercial product. But it's usually the case that if you really want to do the best possible implementation of an approach, optimizing very specifically for your application is what's needed. -- Greg Smith 2nd

Re: [PERFORM] Infinite Cache

2011-07-01 Thread Greg Smith
er performance, I aim for a 10X speedup and often do much better than that. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD Comprehensive and Customized PostgreSQL Training Classes: http://www.2ndquadrant.us/postgresql-training/ -- Sent via pgsql-performance mailin

Re: [PERFORM] Slow performance when querying millions of rows

2011-06-28 Thread Greg Smith
ed table, but the way PostgreSQL does it may make that hard to realize on Windows. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD Comprehensive and Customized PostgreSQL Training Classes: http://www.2ndquadrant.us/postgresql-training/ -- Sent via pgsql-performance maili

Re: [PERFORM] Slow performance when querying millions of rows

2011-06-28 Thread Greg Smith
the vacuum part of that is the painful one. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD Comprehensive and Customized PostgreSQL Training Classes: http://www.2ndquadrant.us/postgresql-training/ -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org

Re: [PERFORM] Slow performance when querying millions of rows

2011-06-28 Thread Greg Smith
on makers who feel they should be able to query any way they want, regardless of its impact on the database. Finding a middle ground there is usually challenging. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD Comprehensive and Customized PostgreSQL Training Classes: http:

Re: [PERFORM] Long Running Update - My Solution

2011-06-27 Thread Greg Smith
the set of data to be processed to become much larger than RAM, it can be a bad decision. The performance drop when things stop fitting in memory is not a slow one, it's like a giant cliff you fall off. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Tra

Re: [PERFORM] Cost of creating an emply WAL segment

2011-06-24 Thread Greg Smith
On 06/24/2011 02:55 PM, Shaun Thomas wrote: On 06/24/2011 11:18 AM, Greg Smith wrote: sync=14525.296 s, total=14786.868 s What!? 6% of 8GB is just shy of 500MB. That's not a small amount, exactly, but it took 14525 seconds to call syncs for those writes? What kind of ridiculo

Re: [PERFORM] Cost of creating an emply WAL segment

2011-06-24 Thread Greg Smith
in 8.3-9.0, as a bug fix because this turns into a "system is too slow to be considered operational" performance problem when it flares up. This was a rare thing I'd never seen before back in September when I started working on this area again; now I see it once a month on a new system

  1   2   3   4   5   6   7   8   9   10   >