On Fri, Mar 16, 2012 at 4:38 AM, Dmytrii Nagirniak wrote:
> To "fix" it I open a transaction before each test and roll it back at the
> end.
>
> Some numbers for ~700 tests.
>
> - Truncation: SQLite - 34s, PG - 76s.
> - Transaction: SQLite - 17s, PG - 18s.
>
> 2x speed increase for SQLite.
> 4x s
Hi all,
Just a follow-up.
I found the biggest bottleneck and now my specs run as fast as the SQLite ones.
TL;DR - the issue was the database cleanup that did the truncation. Apparently
SQLite is way too fast there.
To "fix" it I open a transaction before each test and roll it back at the end.
Hi Guys,
Sorry for the late reply.
Thanks to all of you for the help. Appreciate all your suggestions.
So far (with my pretty limited knowledge of PG) I could speed it up a little
bit (~20% or so comparing to the original installation) only by "tweaking" the
settings.
I think it is relatively
On Fri, Feb 24, 2012 at 12:16 AM, Dmytrii Nagirniak wrote:
> That's totally fine if PG can't beat SQLite on speed in **this particular
> case**.
The point is that PG can beat SQLite in this test *easily* if you
choose to use the main architectural difference as an advantage:
running tests concur
On 24 Feb 2012, at 1:00, Dmytrii Nagirniak wrote:
>> What are the specs?
> A typical DB spec (test) does the following:
> 1. Creates a number of records (commonly about 5-ish, but may vary from 1 to
> ~40 across all tables).
> 2. Executes some queries against the dataset (**MOST** of them are pre
On Thursday, February 23, 2012 4:00:08 pm Dmytrii Nagirniak wrote:
> On 24/02/2012, at 2:06 AM, Adrian Klaver wrote:
> >> Would appreciate some suggestions.
> >
> > Not enough information to make suggestions.
>
> Jack Christensen pretty nailed it very well. But I'll answer the particulars
here t
On Thu, Feb 23, 2012 at 5:22 PM, Dmytrii Nagirniak wrote:
> On 24/02/2012, at 5:15 AM, Simon Riggs wrote:
>
> Now all you have to do is parallelise the tests and everything can
> work 10 times quicker and it would be much faster than the time SQLite
> produced.
>
> So using PostgreSQL for testing
On 02/23/2012 07:16 PM, Dmytrii Nagirniak wrote:
That's totally fine if PG can't beat SQLite on speed in **this
particular case**.
I just want to try to tune it to be as fast as it can (for **this
particular case**, see my reply to Adrian).
You can find all of the big tunable parameters at
htt
On 24/02/2012, at 5:15 AM, Simon Riggs wrote:
> Now all you have to do is parallelise the tests and everything can
> work 10 times quicker and it would be much faster than the time SQLite
> produced.
>
> So using PostgreSQL for testing would be both quicker and more
> accurate, if you set the tes
On 24/02/2012, at 2:22 AM, Simon Riggs wrote:
>> Would appreciate some suggestions.
>
> You really need to explain why this matters...
I've just replied to Adrian with more details. I suppose you don't mind to take
a look there so I won't copy-paste it :)
> You mention a "typical Ruby on Rail
On 24/02/2012, at 2:06 AM, Adrian Klaver wrote:
>>
>> Would appreciate some suggestions.
>
> Not enough information to make suggestions.
Jack Christensen pretty nailed it very well. But I'll answer the particulars
here too:
> Why are you switching databases?
Need FTS from PG. No other reason
On 23/02/2012, at 7:35 PM, Marti Raudsepp wrote:
> If you have lots of very simple queries, then usually much of the
> overhead is in query planning. There are a few tricks to dumb down the
> planner to make it faster -- although that may come at the cost of
> slowing down execution.
>
> * If you
On Thu, Feb 23, 2012 at 11:15 AM, Simon Riggs wrote:
> On Thu, Feb 23, 2012 at 5:20 PM, Scott Marlowe
> wrote:
>
>> He's probably doing automated continuous integration testing. Two
>> jobs ago we had a setup to do that and had 40k tests. The whole test
>> suite took about 30 minutes to runm a
On Thu, Feb 23, 2012 at 5:20 PM, Scott Marlowe wrote:
> He's probably doing automated continuous integration testing. Two
> jobs ago we had a setup to do that and had 40k tests. The whole test
> suite took about 30 minutes to runm and kicked off automatically when
> the last one finished and an
On Thu, Feb 23, 2012 at 10:05 AM, David Salisbury wrote:
>
>
> On 2/23/12 9:06 AM, Jack Christensen wrote:
>>
>> As another Rails developer using PostgreSQL I think I can explain the
>> use case. In standard Rails usage, the ORM handles all SQL query
>> generation and thus the application is datab
On 2/23/12 9:06 AM, Jack Christensen wrote:
As another Rails developer using PostgreSQL I think I can explain the
use case. In standard Rails usage, the ORM handles all SQL query
generation and thus the application is database agnostic. It is typical
to use SQLite in development and testing and
On 2/23/2012 9:22 AM, Simon Riggs wrote:
On Thu, Feb 23, 2012 at 5:13 AM, Dmytrii Nagirniak wrote:
I wonder if you can suggest me how to speed-up PG when running specs.
I asked it at SO here:
http://stackoverflow.com/questions/9407442/optimise-postgresql-for-fast-testing
But briefly. PG spec
On Thu, Feb 23, 2012 at 5:13 AM, Dmytrii Nagirniak wrote:
> I wonder if you can suggest me how to speed-up PG when running specs.
> I asked it at SO here:
>
> http://stackoverflow.com/questions/9407442/optimise-postgresql-for-fast-testing
>
> But briefly. PG specs are >2x slower than SQLite.
> I
On Wednesday, February 22, 2012 9:13:06 pm Dmytrii Nagirniak wrote:
> Hi guys,
>
> I wonder if you can suggest me how to speed-up PG when running specs.
> I asked it at SO here:
>
> http://stackoverflow.com/questions/9407442/optimise-postgresql-for-fast-tes
> ting
>
> But briefly. PG specs are >
On Thu, Feb 23, 2012 at 08:02, Dmytrii Nagirniak wrote:
> Thanks. So far I tried:
>
> fsync = off
> full_page_writes = off
>
> It seems it got a *little* faster (down to ~65 seconds from ~76) but is till
> too far from my target of ~34 secs.
If you have lots of very simple queries, then usually m
2012/2/23 Dmytrii Nagirniak :
> On 23/02/2012, at 5:05 PM, Pavel Stehule wrote:
>
>> SQLite should be faster in single user test - it is optimized for this
>> purpose. So you cannot to get same speed from PostgreSQL
>
> That's unfortunate to hear.
> But hoped with a bit of tuning to get PG close to
On 23/02/2012, at 5:05 PM, Pavel Stehule wrote:
> SQLite should be faster in single user test - it is optimized for this
> purpose. So you cannot to get same speed from PostgreSQL
That's unfortunate to hear.
But hoped with a bit of tuning to get PG close to SQLite by the fact that I can
change t
Hello
SQLite should be faster in single user test - it is optimized for this
purpose. So you cannot to get same speed from PostgreSQL
Pavel
2012/2/23 Dmytrii Nagirniak :
>
>
> On 23/02/2012, at 4:38 PM, Jan Kesten wrote:
>
> Hi Dmytrii,
>
> just as short idea, put "fsync = off" in your postgre
On 23/02/2012, at 4:38 PM, Jan Kesten wrote:
> Hi Dmytrii,
>
> just as short idea, put "fsync = off" in your postgres.conf. That turns off
> that after a commit data is forcilby written to disk - if the database
> crashes there might be dataloss.
Thanks. So far I tried:
fsync = off
full_pa
Hi Dmytrii,
just as short idea, put "fsync = off" in your postgres.conf. That turns off
that after a commit data is forcilby written to disk - if the database crashes
there might be dataloss.
Von meinem iPhone gesendet
Am 23.02.2012 um 06:13 schrieb Dmytrii Nagirniak :
> Hi guys,
>
> I wonde
Hi guys,
I wonder if you can suggest me how to speed-up PG when running specs.
I asked it at SO here:
http://stackoverflow.com/questions/9407442/optimise-postgresql-for-fast-testing
But briefly. PG specs are >2x slower than SQLite.
I want it to be on par (don't care about reliability or anything
26 matches
Mail list logo
