Re: [PERFORM] Cursor + upsert (astronomical data)

2014-07-26 Thread Vitalii Tymchyshyn
I am not sure I understand the problem fully, e.g. what to do if there are observations A,B and C with A to B and B to C less then treshold and A to C over treshold, but anyway. Could you first apply a kind of grid to your observations? What I mean is to round your coords to, say, 1/2 arcsec on ea

[PERFORM] Re: High rate of transaction failure with the Serializable Isolation Level

2014-07-26 Thread Ryan Johnson
That does sound pretty similar, modulo the raw performance difference. I have no idea how many MEE threads there were; it was just a quick run with exactly zero tuning, so I use whatever dbt5 does out of the box. Actually, though, if you have any general tuning tips for TPC-E I'd be interested

[PERFORM] Re: High rate of transaction failure with the Serializable Isolation Level

2014-07-26 Thread Reza Taheri
Hi Ryan, Thanks a lot for sharing this. When I run with 12 CE threads and 3-5 MEE threads (how many MEE threads do you have?) @ 80-90 tps, I get something in the 20-30% of trade-result transactions rolled back depending on how I count. E.g., in a 5.5-minute run with 3 MEE threads, I saw 87.5 tp

[PERFORM] Re: High rate of transaction failure with the Serializable Isolation Level

2014-07-26 Thread Ryan Johnson
Dredging through some old run logs, 12 dbt-5 clients gave the following when everything was run under SSI (fully serializable, even the transactions that allow repeatable read isolation). Not sure how that translates to your results. Abort rates were admittedly rather high, though perhaps lower

[PERFORM] Re: High rate of transaction failure with the Serializable Isolation Level

2014-07-26 Thread Reza Taheri
Hi Ryan, That's a very good point. We are looking at dbt5. One question: what throughput rate, and how many threads of execution did you use for dbt5? The failure rates I reported were at ~120 tps with 15 trade-result threads. Thanks, Reza > -Original Message- > From: pgsql-performance

[PERFORM] Cursor + upsert (astronomical data)

2014-07-26 Thread Jiří Nádvorník
Hello guys. My issue kind of hits multiple topics, but the main question is about performance. I think you need to understand the background a little bit to be able to help me. So I will firstly define the problem and my solutions to it and place the questions for you to the end of this message