Re: [HACKERS] Concurrency testing

2009-10-13 Thread Simon Riggs
On Wed, 2009-10-07 at 13:07 -0400, Alvaro Herrera wrote: > David Fetter wrote: > > > I seem to recall that there were some patches to get psql to help with > > such things, but they didn't go in. Time to revive them? > > Yeah, the API they implemented wasn't ideal, so there was some > discussion

Re: [HACKERS] Concurrency testing

2009-10-09 Thread David Fetter
On Fri, Oct 09, 2009 at 08:34:55PM +0200, Markus Schiltknecht wrote: > Hi, > > David Fetter wrote: >> 1. Test situations which require more than one connection, possibly >> to different clusters, for example in the cases of Hot Standby and >> Streaming Replication. This further divides into event

Re: [HACKERS] Concurrency testing

2009-10-09 Thread Andrew Dunstan
Markus Wanner wrote: Hi, Dimitri Fontaine wrote: I even started a DBT2 implementation as tsung modules, back when returning from pgcon 2006: http://tapoueh.org/misc.html#sec7 darcs get http://pgsql.tapoueh.org/dbt2-tsung/ Now *that* is very cool! I definitely need to have a look at it.

Re: [HACKERS] Concurrency testing

2009-10-09 Thread Markus Wanner
Hi, Dimitri Fontaine wrote: I even started a DBT2 implementation as tsung modules, back when returning from pgcon 2006: http://tapoueh.org/misc.html#sec7 darcs get http://pgsql.tapoueh.org/dbt2-tsung/ Now *that* is very cool! I definitely need to have a look at it. Have you tried Sun Faban

Re: [HACKERS] Concurrency testing

2009-10-09 Thread Markus Wanner
Hi, David Fetter wrote: 1. Test situations which require more than one connection, possibly to different clusters, for example in the cases of Hot Standby and Streaming Replication. This further divides into event-based and time-based tests. It's this situation I had in mind when I posted, an

Re: [HACKERS] Concurrency testing

2009-10-08 Thread Peter Eisentraut
On Thu, 2009-10-08 at 13:00 -0400, Greg Smith wrote: > Perl is the best tool available that runs on the buildfarm, which makes > something written in it useful here, while Tsung is not. Unfortunately. But you're not going to do performance testing on the buildfarm anyway. -- Sent via pgsql-ha

Re: [HACKERS] Concurrency testing

2009-10-08 Thread Dimitri Fontaine
Le 8 oct. 2009 à 19:00, Greg Smith a écrit : Perl is the best tool available that runs on the buildfarm, which makes something written in it useful here, while Tsung is not. Unfortunately. I though we were talking about 2 separate issues here: david's one, for concurrent regression testi

Re: [HACKERS] Concurrency testing

2009-10-08 Thread Joshua D. Drake
On Thu, 2009-10-08 at 13:00 -0400, Greg Smith wrote: > On Thu, 8 Oct 2009, Dimitri Fontaine wrote: > > > sorry guys, perl is not made for concurrent programming, it's not going > > to be easy reaching current tsung level (years of work with ad-hoc > > tools) using a general hobbyist programming

Re: [HACKERS] Concurrency testing

2009-10-08 Thread Greg Smith
On Thu, 8 Oct 2009, Dimitri Fontaine wrote: sorry guys, perl is not made for concurrent programming, it's not going to be easy reaching current tsung level (years of work with ad-hoc tools) using a general hobbyist programming language. Perl is the best tool available that runs on the buildfa

Re: [HACKERS] Concurrency testing

2009-10-08 Thread David Fetter
On Thu, Oct 08, 2009 at 10:28:53AM +0200, Dimitri Fontaine wrote: > http://tsung.erlang-projects.org/ > http://tsung.erlang-projects.org/user_manual.html > http://archives.postgresql.org/pgsql-admin/2008-12/msg00032.php It looks like we're talking about two orthogonal projects here: 1. Test s

Re: [HACKERS] Concurrency testing

2009-10-08 Thread Dimitri Fontaine
Andrew Dunstan writes: > Last time I built something to drive a huge client load (many > thousands of simultaneous connections to a web app) I did it in highly > threaded Java using HttpUnit from a number of separate client machines. You > wouldn't believe what that managed to do to MySQL on the

Re: [HACKERS] Concurrency testing

2009-10-08 Thread Dimitri Fontaine
Greg Smith writes: > The stuff I've been building lately takes "how many sessions at once?" as an > input, and the useful range on that (particularly as we wander further > toward multi-core land) is much higher than you'd want to manage one at a > time manually. I'd rather see a "session sim

Re: [HACKERS] Concurrency testing

2009-10-07 Thread Robert Haas
On Wed, Oct 7, 2009 at 10:41 PM, Jeff Janes wrote: >> I don't really see what's wrong with using Perl modules that are >> likely to be installed most places and easy to obtain where not, if it >> makes writing a test framework much easier.  But I also think that we >> should not get bogged down on

Re: [HACKERS] Concurrency testing

2009-10-07 Thread Jeff Janes
On Wed, Oct 7, 2009 at 5:33 PM, David E. Wheeler wrote: > On Oct 7, 2009, at 5:18 PM, Jeff Janes wrote: > >> I'd much rather live without Test::More and use DBD::Pg, then have >> Test::More but need to open pipes to psql to talk to the database, >> rather than using DBI to do it.  But I guess we w

Re: [HACKERS] Concurrency testing

2009-10-07 Thread Jeff Janes
On Wed, Oct 7, 2009 at 6:01 PM, Robert Haas wrote: > On Wed, Oct 7, 2009 at 8:33 PM, David E. Wheeler wrote: >>> Do we need to restrict ourselves to core?  Developers already need >>> flex and bison, which aren't needed when installing from the tarball. >>> Couldn't we also have "make dev-check"

Re: [HACKERS] Concurrency testing

2009-10-07 Thread Andrew Dunstan
David E. Wheeler wrote: On Oct 7, 2009, at 5:18 PM, Jeff Janes wrote: I'd much rather live without Test::More and use DBD::Pg, then have Test::More but need to open pipes to psql to talk to the database, rather than using DBI to do it. But I guess we would need to worry about whether we can

Re: [HACKERS] Concurrency testing

2009-10-07 Thread Robert Haas
On Wed, Oct 7, 2009 at 8:33 PM, David E. Wheeler wrote: >> Do we need to restrict ourselves to core?  Developers already need >> flex and bison, which aren't needed when installing from the tarball. >> Couldn't we also have "make dev-check" that has higher requirements >> than "make check" does, b

Re: [HACKERS] Concurrency testing

2009-10-07 Thread David E. Wheeler
On Oct 7, 2009, at 5:18 PM, Jeff Janes wrote: I'd much rather live without Test::More and use DBD::Pg, then have Test::More but need to open pipes to psql to talk to the database, rather than using DBI to do it. But I guess we would need to worry about whether we can make DBD::Pg work with the

Re: [HACKERS] Concurrency testing

2009-10-07 Thread Jeff Janes
On Wed, Oct 7, 2009 at 10:38 AM, David Fetter wrote: > On Wed, Oct 07, 2009 at 10:28:08AM -0700, David Wheeler wrote: >> On Oct 7, 2009, at 9:53 AM, David Fetter wrote: >> >>> At the moment, we have no way to test this, although with certain >>> Perl modules, it would be pretty trivial. >> >> No n

Re: [HACKERS] Concurrency testing

2009-10-07 Thread David Fetter
On Wed, Oct 07, 2009 at 08:06:50PM -0400, Greg Smith wrote: > On Wed, 7 Oct 2009, Alvaro Herrera wrote: > >> I don't find this a compelling argument against concurrent psql. >> Sure there are things you can't do with it, but it doesn't mean >> it's not useful. Are we going to need further tools to

Re: [HACKERS] Concurrency testing

2009-10-07 Thread Greg Smith
On Wed, 7 Oct 2009, Alvaro Herrera wrote: I don't find this a compelling argument against concurrent psql. Sure there are things you can't do with it, but it doesn't mean it's not useful. Are we going to need further tools to find "the good concurrent bugs"? No doubt. Don't get me wrong, I

Re: [HACKERS] Concurrency testing

2009-10-07 Thread Alvaro Herrera
Greg Smith wrote: > On Wed, 7 Oct 2009, Alvaro Herrera wrote: > > >Yeah, the API they implemented wasn't ideal, so there was some > >discussion that ended up with a specification everyone was happy with, > >but then nobody got around to implementing it. > > I needed something like this and didn't

Re: [HACKERS] Concurrency testing

2009-10-07 Thread Greg Smith
On Wed, 7 Oct 2009, Alvaro Herrera wrote: Yeah, the API they implemented wasn't ideal, so there was some discussion that ended up with a specification everyone was happy with, but then nobody got around to implementing it. I needed something like this and didn't implement those suggestions be

Re: [HACKERS] Concurrency testing

2009-10-07 Thread David E. Wheeler
On Oct 7, 2009, at 10:38 AM, David Fetter wrote: When did Test::More become core? I believe we support back to Perl 5.6 :/ Module::CoreList says 5.006002, though I would have sworn it was added much earlier than that. You could always use Test.pm, I suppose, which has been in core since 5

Re: [HACKERS] Concurrency testing

2009-10-07 Thread David Fetter
On Wed, Oct 07, 2009 at 10:28:08AM -0700, David Wheeler wrote: > On Oct 7, 2009, at 9:53 AM, David Fetter wrote: > >> At the moment, we have no way to test this, although with certain >> Perl modules, it would be pretty trivial. > > No non-core modules necessary. Just use Test::More and file handle

Re: [HACKERS] Concurrency testing

2009-10-07 Thread David E. Wheeler
On Oct 7, 2009, at 9:53 AM, David Fetter wrote: At the moment, we have no way to test this, although with certain Perl modules, it would be pretty trivial. No non-core modules necessary. Just use Test::More and file handles opened via pipes to two or more psql sessions. Best, David -- Se

Re: [HACKERS] Concurrency testing

2009-10-07 Thread Alvaro Herrera
David Fetter wrote: > I seem to recall that there were some patches to get psql to help with > such things, but they didn't go in. Time to revive them? Yeah, the API they implemented wasn't ideal, so there was some discussion that ended up with a specification everyone was happy with, but then n

Re: [HACKERS] Concurrency testing

2009-10-07 Thread David Fetter
On Wed, Oct 07, 2009 at 09:17:52AM -0700, David Fetter wrote: > Folks, > > As we move forward, we run into increasingly complex situations under > the general rubric of concurrency. > > What test frameworks are already out there that we can use in our > regression test suite? If there aren't any

[HACKERS] Concurrency testing

2009-10-07 Thread David Fetter
Folks, As we move forward, we run into increasingly complex situations under the general rubric of concurrency. What test frameworks are already out there that we can use in our regression test suite? If there aren't any, how might we build one? Cheers, David. -- David Fetter http://fetter.or