Re: [PERFORM] dbt-2 tuning results with postgresql-8.3.5

2009-02-06 Thread Greg Smith

On Thu, 5 Feb 2009, Mark Wong wrote:

One of the problems is that my scripts are listing the OS of the main 
driver system, as opposed to the db system.


That's not a fun problem to deal with.  Last time I ran into it, I ended 
up writing a little PL/PerlU function that gathered all the kernel-level 
info I wanted, and then just pulled the info over by running that on the 
driver system.


--
* Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Matt Burke
Rajesh Kumar Mallah wrote:
>> I've checked out the latest Areca controllers, but the manual 
>> available on their website states there's a limitation of 32 disks 
>> in an array...
> 
> Where exactly is there  limitation of 32 drives. the datasheet of 
> 1680 states support upto 128drives using enclosures.

The 1680 manual:
http://www.areca.us//support/download/RaidCards/Documents/Manual_Spec/SAS_Manual.zip

Page 25:

> Note:
> 
> 1. The maximum no. is 32 disk drived included in a single RAID set

Page 49:

> 1. Up to 32 disk drives can be included in a single RAID set.
> 2. Up to 8 RAID sets can be created per controller

(point 2 meaning you can't do s/w RAID over umpteen h/w RAID1 pairs)

Page 50:

> To create RAID 30/50/60 volume, you need create multiple RAID sets
> first with the same disk members on each RAID set. The max no. disk
> drives per volume set: 32 for RAID 0/1/10/3/5/6 and 128 for RAID
> 30/50/60.

...and a few more times saying the same thing

-- 

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Matt Burke
Scott Carey wrote:
> You probably don’t want a single array with more than 32 drives anyway,
> its almost always better to start carving out chunks and using software
> raid 0 or 1 on top of that for various reasons. I wouldn’t put more than
> 16 drives in one array on any of these RAID cards, they’re just not
> optimized for really big arrays and tend to fade between 6 to 16 in one
> array, depending on the quality.

This is what I'm looking at now. The server I'm working on at the moment
currently has a PERC6/e and 3xMD1000s which needs to be tested in a few
setups.  I need to code a benchmarker yet (I haven't found one yet that
can come close to replicating our DB usage patterns), but I intend to try:

1. 3x h/w RAID10 (one per shelf), sofware RAID0
2. lots x h/w RAID1, software RAID0 if the PERC will let me create
enough arrays
3. Pure s/w RAID10 if I can convince the PERC to let the OS see the disks
4. 2x h/w RAID30, software RAID0

I'm not holding much hope out for the last one :)


I'm just glad work on a rewrite of my inherited backend systems should
start soon; get rid of the multi-TB MySQL hell and move to a distributed
PG setup on dirt cheap Dell R200s/blades


> You can do direct-attached storage to 100+ drives or more if you want.
>  The price and manageability cost go up a lot if it gets too big
> however.  Having global hot spare drives is critical.  Not that the cost
> of using SAN’s and such is low...  SAS expanders have made DAS with
> large arrays very accessible though.

For large storage arrays (RAID60 or similar) you can't beat a RAID
controller and disk shelf(s), especially if you keep the raidsets small
and use cheap ludicrous capacity SATA disks

You just need to be aware that performance doesn't scale well/easily
over 1-2 shelves on the things


-- 



-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Scott Marlowe
On Fri, Feb 6, 2009 at 2:04 AM, Matt Burke  wrote:
> Scott Carey wrote:
>> You probably don't want a single array with more than 32 drives anyway,
>> its almost always better to start carving out chunks and using software
>> raid 0 or 1 on top of that for various reasons. I wouldn't put more than
>> 16 drives in one array on any of these RAID cards, they're just not
>> optimized for really big arrays and tend to fade between 6 to 16 in one
>> array, depending on the quality.
>
> This is what I'm looking at now. The server I'm working on at the moment
> currently has a PERC6/e and 3xMD1000s which needs to be tested in a few
> setups.  I need to code a benchmarker yet (I haven't found one yet that
> can come close to replicating our DB usage patterns), but I intend to try:
>
> 1. 3x h/w RAID10 (one per shelf), sofware RAID0

Should work pretty well.

> 2. lots x h/w RAID1, software RAID0 if the PERC will let me create
> enough arrays

I don't recall the max number arrays.  I'm betting it's less than that.

> 3. Pure s/w RAID10 if I can convince the PERC to let the OS see the disks

Look for JBOD mode.

> 4. 2x h/w RAID30, software RAID0
>
> I'm not holding much hope out for the last one :)

Me either. :)

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Number of occurrence of characters?

2009-02-06 Thread Ivan Voras
hubert depesz lubaczewski wrote:
> On Thu, Feb 05, 2009 at 02:31:24PM +0100, Ivan Voras wrote:
>> The problem is how to do it efficiently. I see there isn't a built-in
>> function that counts character occurrences so I'd have to write it
>> myself. An additional constraint is that it must be implemented with
>> built-in capabilities, i.e. SQL and plpsql languages. I can do it the
>> brute force way, looping over the string and processing one by one
>> character with substring(), but is there a faster way?
> 
> # select length(regexp_replace('/some/string/with/slashes', '[^/]+', '', 
> 'g'));
>  length
> 
>   4
> (1 row)

Thank you (and Heikki), I had a feeling I was missing an approach.



signature.asc
Description: OpenPGP digital signature


[PERFORM] Can't locate Test/Parser/Dbt2.pm in DBT2 tests

2009-02-06 Thread Rohan Pethkar
Hi All,

I am conductingDBT2 tests on PostgreSQL. After completing the test while
analyzing and creating the results I am getting following error:

./dbt2-run-workload: line 514:   731 Terminated  dbt2-client
${CLIENT_COMMAND_ARGS} -p ${PORT} -o ${CDIR} >
${CLIENT_OUTPUT_DIR}/`hostname`/client-${SEG}.out 2>&1
waiting for server to shut down done
server stopped
Can't locate Test/Parser/Dbt2.pm in @INC (@INC contains:
/usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi
/usr/lib64/perl5/site_perl/5.8.7/x86_64-linux-thread-multi
/usr/lib64/perl5/site_perl/5.8.6/x86_64-linux-thread-multi
/usr/lib64/perl5/site_perl/5.8.5/x86_64-linux-thread-multi
/usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.7
/usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_perl/5.8.5
/usr/lib/perl5/site_perl
/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi
/usr/lib64/perl5/vendor_perl/5.8.7/x86_64-linux-thread-multi
/usr/lib64/perl5/vendor_perl/5.8.6/x86_64-linux-thread-multi
/usr/lib64/perl5/vendor_perl/5.8.5/x86_64-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl/5.8.7
/usr/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.5
/usr/lib/perl5/vendor_perl /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi
/usr/lib/perl5/5.8.8 .) at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-post-process line 13.
BEGIN failed--compilation aborted at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-post-process line 13.
The authenticity of host 'localhost (127.0.0.1)' can't be established.
RSA key fingerprint is 87:1e:a0:e0:4f:b7:32:42:3d:5c:85:10:f3:25:63:8d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
Write failed: Broken pipe
Creating chart for Index Scans...
Creating chart for Index Blocks Read...
Creating chart for Index Blocks Hit...
Creating chart for Table Blocks Read...
Creating chart for Table Blocks Hit...
Creating chart for Sequential Scans...
Creating chart for Rows Inserted...
Creating chart for Rows Updated...
Creating chart for Row Deleted...
Creating chart for HOT Rows Updated...
Creating chart for Dead Tuples...
Can't exec "gnuplot": No such file or directory at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
113.
Can't exec "gnuplot": No such file or directory at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
113.
Can't exec "gnuplot": No such file or directory at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
113.
Can't exec "gnuplot": No such file or directory at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
113.
Can't exec "gnuplot": No such file or directory at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
113.
Can't exec "gnuplot": No such file or directory at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
113.
Can't exec "gnuplot": No such file or directory at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
113.
Can't exec "gnuplot": No such file or directory at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
113.
Can't exec "gnuplot": No such file or directory at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
113.
Can't exec "gnuplot": No such file or directory at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
113.
Can't exec "gnuplot": No such file or directory at
/home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
113.
Test completed.
Results are in: OUTPUT

I ma not sure why it doesn't find Test/Parser/Dbt2.pm even if I have
installed DBT2 completely. Did I miss any steps? Do I need to install some
extra packages? If any then please let me know.

Thanks in advance for your help.

Thanks,
Rohan


Re: [PERFORM] Number of occurrence of characters?

2009-02-06 Thread Grzegorz Jaśkiewicz
It all depends at the end of a day, how crucial is that functionality
to your app ?
If very, I would add to every insert/update a trigger, that would
update info on other table(s) with stats per character.

Other option, is to write a function in C that would parse word and
chop it in C,O (character, number of occurrences ) result.


-- 
GJ

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Bruce Momjian
Matt Burke wrote:
> Scott Carey wrote:
> > You probably don?t want a single array with more than 32 drives anyway,
> > its almost always better to start carving out chunks and using software
> > raid 0 or 1 on top of that for various reasons. I wouldn?t put more than
> > 16 drives in one array on any of these RAID cards, they?re just not
> > optimized for really big arrays and tend to fade between 6 to 16 in one
> > array, depending on the quality.
> 
> This is what I'm looking at now. The server I'm working on at the moment
> currently has a PERC6/e and 3xMD1000s which needs to be tested in a few
> setups.  I need to code a benchmarker yet (I haven't found one yet that
> can come close to replicating our DB usage patterns), but I intend to try:

Stupid question, but why do people bother with the Perc line of cards if
the LSI brand is better?  It seems the headache of trying to get the
Perc cards to perform is not worth any money saved.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Matt Burke
Glyn Astill wrote:
>> Stupid question, but why do people bother with the Perc line of
>> cards if the LSI brand is better?  It seems the headache of trying 
>> to get the Perc cards to perform is not worth any money saved.
> 
> I think in most cases the dell cards actually cost more, people end
> up stuck with them because they come bundled with their servers -
> they find out too late that they've got a lemon.

That's what's been happening with me... The fact Dell prices can have a
fair bit of downward movement when you get the account manager on the
phone makes them especially attractive to the people controlling the
purse strings.

The biggest reason for me however is the lack of comparative reviews. I
struggled to get the LSI card to replace the PERC3 because all I had to
go on was qualitative mailing list/forum posts from strangers. The only
way I got it was to make the argument that other than trying the LSI,
we'd have no choice other than replacing the server+shelf+disks.

I want to see just how much better a high-end Areca/Adaptec controller
is, but I just don't think I can get approval for a £1000 card "because
some guy on the internet said the PERC sucks". Would that same person
say it sucked if it came in Areca packaging? Am I listening to the
advice of a professional, or a fanboy?



-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Bruce Momjian
Matt Burke wrote:
> Glyn Astill wrote:
> >> Stupid question, but why do people bother with the Perc line of
> >> cards if the LSI brand is better?  It seems the headache of trying 
> >> to get the Perc cards to perform is not worth any money saved.
> > 
> > I think in most cases the dell cards actually cost more, people end
> > up stuck with them because they come bundled with their servers -
> > they find out too late that they've got a lemon.
> 
> That's what's been happening with me... The fact Dell prices can have a
> fair bit of downward movement when you get the account manager on the
> phone makes them especially attractive to the people controlling the
> purse strings.
> 
> The biggest reason for me however is the lack of comparative reviews. I
> struggled to get the LSI card to replace the PERC3 because all I had to
> go on was qualitative mailing list/forum posts from strangers. The only
> way I got it was to make the argument that other than trying the LSI,
> we'd have no choice other than replacing the server+shelf+disks.
> 
> I want to see just how much better a high-end Areca/Adaptec controller
> is, but I just don't think I can get approval for a ?1000 card "because
> some guy on the internet said the PERC sucks". Would that same person
> say it sucked if it came in Areca packaging? Am I listening to the
> advice of a professional, or a fanboy?

The experiences I have heard is that Dell looks at server hardware in
the same way they look at their consumer gear, "If I put in a cheaper
part, how much will it cost Dell to warranty replace it".  Sorry, but I
don't look at my performance or downtime in the same way Dell does.  ;-)

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


[PERFORM] Postgres not willing to use an index?

2009-02-06 Thread Mario Splivalo

I have a table, like this:

CREATE TABLE transactions
(
  transaction_id integer NOT NULL DEFAULT 
nextval('transactions_seq'::regclass),

  transaction_type integer NOT NULL,
  transaction_client_id integer NOT NULL,
  transaction_destination_id integer NOT NULL,
  transaction_operator_id integer NOT NULL,
  transaction_application_id integer NOT NULL,
  transaction_application_service character varying NOT NULL,
  transaction_quantity integer NOT NULL,
  transaction_time_commit timestamp with time zone NOT NULL,
  transaction_time_received timestamp with time zone NOT NULL,
  transaction_gateway_id character(36) NOT NULL,
  transaction_payment_amount integer NOT NULL DEFAULT 0,
  CONSTRAINT transactions_pk PRIMARY KEY (transaction_id),
  CONSTRAINT transactions_uq__gateway_id UNIQUE (transaction_gateway_id)
)
WITH (OIDS=FALSE);

Now, all the _type, client_id, destination_id, operator_id, and 
application_id are foreigen-keyed to coresponding tables. There are no 
indices on those columns.


Besides PK and uq-constraint indices I have this index:

CREATE INDEX transactions_idx__client_data ON transactions
USING btree (transaction_client_id, transaction_destination_id, 
transaction_operator_id, transaction_application_id, 
transaction_time_commit)


The table_count is like this:

jura=# select count(*) from transactions;
  count
--
 13751457
(1 row)

There are roughly 500.000 - 600.000 transactions for each month. There 
are also transactions from past two years in the table.


I often SELECT data from the table for specified time period - usualy 
from begining to the end of the month, like this:


SELECT  FROM transactions WHERE transaction_time_commit 
BETWEEN '2009-01-01' AND '2009-01-31 23:59:59';


The problem is that postgres is never using an index:

jura=# explain analyze select * from transactions where 
transaction_time_commit between '2009-01-01' and '2009-01-31 23:59:59';


QUERY PLAN 


-
 Seq Scan on transactions  (cost=0.00..416865.85 rows=593713 width=91) 
(actual time=4.067..3918.629 rows=525051 loops=1)
   Filter: ((transaction_time_commit >= '2009-01-01 
00:00:00+01'::timestamp with time zone) AND (transaction_time_commit <= 
'2009-01-31 23:59:59+01'::timestamp with time zone))

 Total runtime: 4026.404 ms
(3 rows)

Time: 4068.521 ms


If I force it not to use sequential scans, it is using index, with 
benefits of shorter execution time:

jura=# set enable_seqscan to false;
SET
Time: 0.103 ms
jura=# explain analyze select * from transactions where 
transaction_time_commit between '2009-01-01' and '2009-01-31 23:59:59';


 QUERY PLAN 


---
 Bitmap Heap Scan on transactions  (cost=410369.98..629869.67 
rows=593713 width=91) (actual time=1060.569..1280.500 rows=525051 loops=1)
   Recheck Cond: ((transaction_time_commit >= '2009-01-01 
00:00:00+01'::timestamp with time zone) AND (transaction_time_commit <= 
'2009-01-31 23:59:59+01'::timestamp with time zone))
   ->  Bitmap Index Scan on transactions_idx__client_data 
(cost=0.00..410221.55 rows=593713 width=0) (actual 
time=1058.992..1058.992 rows=525051 loops=1)
 Index Cond: ((transaction_time_commit >= '2009-01-01 
00:00:00+01'::timestamp with time zone) AND (transaction_time_commit <= 
'2009-01-31 23:59:59+01'::timestamp with time zone))

 Total runtime: 1388.882 ms
(5 rows)

Time: 1396.737 ms


Now, I found interesting is that if I create index just on 
transaction_time_commit column (and I leave 
transactions_idx__client_data index), then postgres is using that new index.


Also, if I change idx__client_data index like this (first I drop it, and 
then I create new one):


CREATE INDEX transactions_idx__client_data ON transactions
USING btree (transaction_client_id, transaction_destination_id, 
transaction_time_commit);


then postgres is using that index:

jura=# explain analyze select * from transactions where 
transaction_time_commit between '2009-01-01' and '2009-01-31 23:59:59';


 QUERY PLAN 


---
 Bitmap Heap Scan on transactions  (cost=349473.37..568973.06 
rows=593713 width=91) (actual time=949.224..1128.848 rows=525051 loops=1)
   Recheck Cond: ((transaction_time_commit >= '2009-01-01 
00:00:00+01'::timestamp with time zone) AND (transaction_time_commit <= 
'2009-01-31 23:59:59+01'::timestamp with time zone))
   ->  Bitmap Index Scan on transactions_idx__client_data 
(cost=0.00..349324.94 rows=593713 width=0) (actual time=947.678..94

Re: [PERFORM] Postgres not willing to use an index?

2009-02-06 Thread Grzegorz Jaśkiewicz
On Fri, Feb 6, 2009 at 3:43 PM, Mario Splivalo
 wrote:

> Besides PK and uq-constraint indices I have this index:
>
> CREATE INDEX transactions_idx__client_data ON transactions
> USING btree (transaction_client_id, transaction_destination_id,
> transaction_operator_id, transaction_application_id,
> transaction_time_commit)

I think it is because it isn't just a simple index, and for some
reason planner decides - that going through every
transaction_application_id, etc, etc just to find right
transaction_time_commit isn't worth the hassle.
Try using few more indexes, on less columns.
Also, if I may - I don't think it is quite usefull to have column
names that include table name in every single one of them, makes
things so much less readable.

-- 
GJ

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread justin




Bruce Momjian wrote:

  Matt Burke wrote:
  
  

we'd have no choice other than replacing the server+shelf+disks.

I want to see just how much better a high-end Areca/Adaptec controller
is, but I just don't think I can get approval for a ?1000 card "because
some guy on the internet said the PERC sucks". Would that same person
say it sucked if it came in Areca packaging? Am I listening to the
advice of a professional, or a fanboy?

  
  
The experiences I have heard is that Dell looks at server hardware in
the same way they look at their consumer gear, "If I put in a cheaper
part, how much will it cost Dell to warranty replace it".  Sorry, but I
don't look at my performance or downtime in the same way Dell does.  ;-)
  

It always boils down to money.  To communicate to the ones controlling
the purse strings talk dollar bills.  To get what one wants from the
purse string holders give examples like this. 
Buying cheap hardware can result in a complete shut down resulting in
lost sales and/or non productive labor being spent.  

Example would be a company generates 100 sales orders an hour average
$100 = $10,000 if the server is down for 8 hours 1 business day thats
$80,000 lost in business.  now lets throw in labor average hourly rate
lets say $15.00 an hour for 10 people = $150.00 for 8 hours = $1200 in
lost labor.  Now throw in overtime to get caught up  $1800  total labor
cost  = $3000

The $200 to $300 saved on the card  was a good decision :-(

Now the argument can be made hardware failures are low so that goes out
the door

Your next best argument is showing the waste in lost productivity. 
Lets say  because of the cheap hardware purchased the users must sit
idle  3 seconds per transactions  times 100 transactions per day = 300
seconds lost X 10 people = 3000 Seconds per day X 235 working days  =
705000/60/60 = 196 hours lost per year  times 3years for average life
span of the server = 588 hours X  average pay rate $15 = $8820.00 lost
labor

Again smart thinking.  

There are all kind of ways to win these arguments to push for higher
quality hardware.




Re: [PERFORM] Postgres not willing to use an index?

2009-02-06 Thread Tom Lane
Mario Splivalo  writes:
> Besides PK and uq-constraint indices I have this index:

> CREATE INDEX transactions_idx__client_data ON transactions
> USING btree (transaction_client_id, transaction_destination_id, 
> transaction_operator_id, transaction_application_id, 
> transaction_time_commit)

> SELECT  FROM transactions WHERE transaction_time_commit 
> BETWEEN '2009-01-01' AND '2009-01-31 23:59:59';
> The problem is that postgres is never using an index:

Hardly surprising --- a search on the index's lowest-order column would
require scanning practically all of the index.  (If you think about the
ordering of the index entries you'll see why.)  If this is a typical
query then you need a separate index on transaction_time_commit.

The fine manual goes into some detail about how to design indexes;
http://www.postgresql.org/docs/8.3/static/indexes.html
particularly 11.3, 11.5.

regards, tom lane

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Glyn Astill
--- On Fri, 6/2/09, Bruce Momjian  wrote:

> Stupid question, but why do people bother with the Perc
> line of cards if
> the LSI brand is better?  It seems the headache of trying
> to get the
> Perc cards to perform is not worth any money saved.

I think in most cases the dell cards actually cost more, people end up stuck 
with them because they come bundled with their servers - they find out too late 
that they've got a lemon.

Up until recently those in charge of buying hardware where I work insisted 
everything be supplied from dell. Fortunately that policy is no more; I have 
enough paperweights.




-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Postgres not willing to use an index?

2009-02-06 Thread Robert Haas
On Fri, Feb 6, 2009 at 11:14 AM, Tom Lane  wrote:
> Mario Splivalo  writes:
>> Besides PK and uq-constraint indices I have this index:
>
>> CREATE INDEX transactions_idx__client_data ON transactions
>> USING btree (transaction_client_id, transaction_destination_id,
>> transaction_operator_id, transaction_application_id,
>> transaction_time_commit)
>
>> SELECT  FROM transactions WHERE transaction_time_commit
>> BETWEEN '2009-01-01' AND '2009-01-31 23:59:59';
>> The problem is that postgres is never using an index:
>
> Hardly surprising --- a search on the index's lowest-order column would
> require scanning practically all of the index.  (If you think about the
> ordering of the index entries you'll see why.)  If this is a typical
> query then you need a separate index on transaction_time_commit.
>
> The fine manual goes into some detail about how to design indexes;
> http://www.postgresql.org/docs/8.3/static/indexes.html
> particularly 11.3, 11.5.

What's weird about this example is that when he sets enable_seqscan to
off, the bitmap index scan plan is actually substantially faster, even
though it in fact does scan nearly the entire heap.  I don't
understand how it can be faster to scan the index and the heap than to
just scan the heap.

...Robert

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Can't locate Test/Parser/Dbt2.pm in DBT2 tests

2009-02-06 Thread Richard Huxton
Rohan Pethkar wrote:
> Hi All,
> 
> I am conductingDBT2 tests on PostgreSQL. After completing the test while
> analyzing and creating the results I am getting following error:
> 
> ./dbt2-run-workload: line 514:   731 Terminated  dbt2-client
> ${CLIENT_COMMAND_ARGS} -p ${PORT} -o ${CDIR} >
> ${CLIENT_OUTPUT_DIR}/`hostname`/client-${SEG}.out 2>&1
> waiting for server to shut down done
> server stopped
> Can't locate Test/Parser/Dbt2.pm in @INC (@INC contains:
> /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi
> /usr/lib64/perl5/site_perl/5.8.7/x86_64-linux-thread-multi
> /usr/lib64/perl5/site_perl/5.8.6/x86_64-linux-thread-multi
> /usr/lib64/perl5/site_perl/5.8.5/x86_64-linux-thread-multi
> /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.7
> /usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_perl/5.8.5
> /usr/lib/perl5/site_perl
> /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi
> /usr/lib64/perl5/vendor_perl/5.8.7/x86_64-linux-thread-multi
> /usr/lib64/perl5/vendor_perl/5.8.6/x86_64-linux-thread-multi
> /usr/lib64/perl5/vendor_perl/5.8.5/x86_64-linux-thread-multi
> /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl/5.8.7
> /usr/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.5
> /usr/lib/perl5/vendor_perl /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi
> /usr/lib/perl5/5.8.8 .) at
> /home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-post-process line 13.

Well, if Test::Parser::Dbt2 isn't in somewhere in that list of
directories, you'll need to tell perl where to look. Simplest is
probably just to:
  export PERL5LIB="/path/to/extra/libs"
before running your tests.

> Can't exec "gnuplot": No such file or directory at
> /home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
> 113.

It also looks like you're missing gnuplot for your charts.

> I ma not sure why it doesn't find Test/Parser/Dbt2.pm even if I have
> installed DBT2 completely. Did I miss any steps? Do I need to install some
> extra packages? If any then please let me know.

You can always "perldoc perlrun" for more info (google it if you don't
have docs installed locally).

-- 
  Richard Huxton
  Archonet Ltd

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Scott Marlowe
On Fri, Feb 6, 2009 at 8:19 AM, Matt Burke  wrote:
> Glyn Astill wrote:
>>> Stupid question, but why do people bother with the Perc line of
>>> cards if the LSI brand is better?  It seems the headache of trying
>>> to get the Perc cards to perform is not worth any money saved.
>>
>> I think in most cases the dell cards actually cost more, people end
>> up stuck with them because they come bundled with their servers -
>> they find out too late that they've got a lemon.
>
> That's what's been happening with me... The fact Dell prices can have a
> fair bit of downward movement when you get the account manager on the
> phone makes them especially attractive to the people controlling the
> purse strings.
>
> The biggest reason for me however is the lack of comparative reviews. I
> struggled to get the LSI card to replace the PERC3 because all I had to
> go on was qualitative mailing list/forum posts from strangers. The only
> way I got it was to make the argument that other than trying the LSI,
> we'd have no choice other than replacing the server+shelf+disks.
>
> I want to see just how much better a high-end Areca/Adaptec controller
> is, but I just don't think I can get approval for a £1000 card "because
> some guy on the internet said the PERC sucks". Would that same person
> say it sucked if it came in Areca packaging? Am I listening to the
> advice of a professional, or a fanboy?

The best reviews I've seen have been on Tweakers first, then
tomshardware.  I am both a professional who recommends the Areca 16xx
series and a bit of a fanboy, mainly because they saved out bacon this
last year, compared to the crapware we'd have had to buy from Dell at
twice to price to come even close to it in performance. A $11.5k white
box with the Areac is a match for over $20k worth of Dell hardware,
and it just works.  Can you get an evaluation unit from a supplier?

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Postgres not willing to use an index?

2009-02-06 Thread Kevin Grittner
>>> Robert Haas  wrote: 
> What's weird about this example is that when he sets enable_seqscan
to
> off, the bitmap index scan plan is actually substantially faster,
even
> though it in fact does scan nearly the entire heap.  I don't
> understand how it can be faster to scan the index and the heap than
to
> just scan the heap.
 
It's cached in the second test, maybe?
 
-Kevin

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Arjen van der Meijden

On 4-2-2009 22:36 Scott Marlowe wrote:

We purhcased the Perc 5E, which dell wanted $728 for last fall with 8
SATA disks in an MD-1000 and the performance is just terrible.  No
matter what we do the best throughput on any RAID setup was about 30
megs/second write and 60 Megs/second read.  I can get that from a
mirror set of the same drives under linux kernel software RAID.  This
was with battery backed cache enabled. Could be an interaction issue
with the MD-1000, or something, but the numbers are just awful.  We
have a Perc 6(i or e not sure) on a 6 disk SAS array and it's a little
better, getting into the hundred meg/second range, but nothing
spectacular.  They're stable, which is more than I can say for a lot
of older PERCs and the servers they came in (x600 series with Perc 3i
for instance).


When we purchased our Perc 5/e with MD1000 filled with 15 15k rpm sas 
disks, my colleague actually spend some time benchmarking the PERC and a 
ICP Vortex (basically a overclocked Adaptec) on those drives. 
Unfortunately he doesn't have too many comparable results, but it 
basically boiled down to quite good scores for the PERC and a bit less 
for the ICP Vortex.
IOMeter sequential reads are above 300MB/s for the RAID5 and above 
240MB/s for a RAID10 (and winbench99 versions range from 400+ to 
600+MB/s). The results for a 10, 12 and to 14 disk configuration also 
showed nice increments in performance.


So we've based our purchase on my colleague's earlier bad experience 
with Adaptec (much worse results than LSI) and weren't dissapointed by 
Dell's scores. I have no idea whether Adaptec's results have increased 
over time, unfortunately we haven't had a larger scale disk IO-benchmark 
for quite some time.


If you're able to understand Dutch, you can click around here:
http://tweakers.net/benchdb/test/90

Best regards,

Arjen

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Scott Carey


> 3. Pure s/w RAID10 if I can convince the PERC to let the OS see the disks

Look for JBOD mode.



PERC 6 does not have JBOD mode exposed.  Dell disables the feature from the LSI 
firmware in their customization.
However, I have been told that you can convince them to tell you the 'secret 
handshake' or whatever that allows JBOD to be enabled.  The more adventurous 
flash the card with the LSI firmware, though I'm sure that voids all sorts of 
support from DELL.


[PERFORM] inheritance, and plans

2009-02-06 Thread Grzegorz Jaśkiewicz
Hey folks,

I have few tables, that inherit from table X.
The query I perform, tries to obtain information about changes in all
tables that inherit from X,
aside from that, I have table Y that keeps another information related
to changes, but in bit different schema.
Anyway, there is one unique id field, shared amongst them.

When I want to obtain all that information, I do:


select updateid from (
  select updateid from r.skel
union all
  select updateid from r.history
) as foo
  where  updateid > 1232634919168805;

And what amazes me, is that no matter what value I choose in where X >
, postgres will always think this is the best plan:

  QUERY PLAN
---
 Subquery Scan foo  (cost=0.00..167736.75 rows=978726 width=8)
   Filter: (foo.updateid > 1232634919168805::bigint)
   ->  Append  (cost=0.00..131034.54 rows=2936177 width=8)
 ->  Subquery Scan "*SELECT* 1"  (cost=0.00..130999.94
rows=2934947 width=8)
   ->  Result  (cost=0.00..101650.47 rows=2934947 width=8)
 ->  Append  (cost=0.00..101650.47 rows=2934947 width=8)
   ->  Seq Scan on skel  (cost=0.00..24.80
rows=1480 width=8)
   ->  Seq Scan on a skel
(cost=0.00..22028.96 rows=923596 width=8)
   ->  Seq Scan on b skel  (cost=0.00..8.01
rows=201 width=8)
   ->  Seq Scan on c skel  (cost=0.00..1.81
rows=81 width=8)
   ->  Seq Scan on d skel
(cost=0.00..22117.94 rows=923594 width=8)
   ->  Seq Scan on e skel  (cost=0.00..6.03
rows=303 width=8)
   ->  Seq Scan on f skel  (cost=0.00..6.02
rows=202 width=8)
   ->  Seq Scan on g skel  (cost=0.00..1987.40
rows=85140 width=8)
   ->  Seq Scan on h skel  (cost=0.00..1.01
rows=1 width=8)
   ->  Seq Scan on i skel
(cost=0.00..55454.99 rows=99 width=8)
   ->  Seq Scan on j skel  (cost=0.00..13.50
rows=350 width=8)
 ->  Seq Scan on history  (cost=0.00..22.30 rows=1230 width=8)
(18 rows)

so my question is, why isn't postgres use index on some tables , and
search for the X > N individually ?

Because, yet - I tried to recreate problem, but I wasn't able. I have
this test db:

create schema r;

create sequence fooseq;
create domain r.fooint AS bigint NOT NULL default  nextval('fooseq');

create table skel(aid r.fooint, cd timestamp default now() not null);

create table one( a bigserial, aid r.fooint, cd timestamp not null);
create table two( a bigserial, aid r.fooint, cd timestamp not null);
create table three( a bigserial, aid r.fooint, cd timestamp not null);
create table four( a bigserial, aid r.fooint, cd timestamp not null);
create table five( a bigserial, aid r.fooint, cd timestamp not null);

create unique index one_aid on one(aid);
create unique index two_aid on two(aid);
create unique index three_aid on three(aid);
create unique index four_aid on four(aid);
create unique index five_aid on five(aid);


create table numbers( something int default random()*666, aid_foo r.fooint);

create unique index numbers_aid on numbers(aid_foo);

insert into one(a, cd) select generate_series(1,200), now();
insert into two(a, cd) select generate_series(1,20), now();
insert into three(a, cd) select generate_series(1,220), now();
insert into four(a, cd) select generate_series(1,220), now();
insert into five(a, cd) select generate_series(1,220), now();

insert into numbers(something) select generate_series(1,87);

alter table one inherit skel;
alter table two inherit skel;
alter table three inherit skel;
alter table four inherit skel;
alter table five inherit skel;


But no matter how many tables I throw in ( and I got to 20 ) - it will
always do it right:
gjaskie=# explain  select aid from (select aid from skel union all
select aid_foo as aid from numbers) AS foo where  aid > 999000;
 QUERY PLAN
-
 Result  (cost=0.00..178034.88 rows=8661268 width=8)
   ->  Append  (cost=0.00..178034.88 rows=8661268 width=8)
 ->  Seq Scan on skel  (cost=0.00..32.12 rows=590 width=8)
   Filter: ((aid)::bigint > 999000)
 ->  Index Scan using one_aid on one skel
(cost=0.00..34549.76 rows=991445 width=8)
   Index Cond: ((aid)::bigint > 999000)
 ->  Seq Scan on two skel  (cost=0.00..3774.00 rows=199980 width=8)
   Filter: ((aid)::bigint > 999000)
 ->  Seq Scan on three skel  (cost=0.00..41513.00 rows=2199780 width=8)
   Filter: ((aid)::bigint > 999000)
 ->  Seq Scan on four skel  (cost=0.00..41513.00 rows=2199780 width=8)
   Filter: ((aid)::bigint > 

Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Arjen van der Meijden

On 6-2-2009 16:27 Bruce Momjian wrote:

The experiences I have heard is that Dell looks at server hardware in
the same way they look at their consumer gear, "If I put in a cheaper
part, how much will it cost Dell to warranty replace it".  Sorry, but I
don't look at my performance or downtime in the same way Dell does.  ;-)


I'm pretty sure all major server-suppliers will have some form of 
risk-analysis for their servers, especially in the high volume x86 
market where most servers are replaced in three years time anyway.
And although Dell's image for quality hardware isn't too good, the 
servers we have from them all reached high uptimes before we did 
hardware unrelated reboots.
Our Dell-desktops/workstations have seen a bit more support-technician's 
though, so we're not becoming fanboys any time soon ;-) They seem to be 
much more serious on quality for their servers compared to the other stuff.


Best regards,

Arjen

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Postgres not willing to use an index?

2009-02-06 Thread Robert Haas
On Fri, Feb 6, 2009 at 12:41 PM, Kevin Grittner
 wrote:
 Robert Haas  wrote:
>> What's weird about this example is that when he sets enable_seqscan to
>> off, the bitmap index scan plan is actually substantially faster, even
>> though it in fact does scan nearly the entire heap.  I don't
>> understand how it can be faster to scan the index and the heap than to
>> just scan the heap.
>
> It's cached in the second test, maybe?

I gather that the results were repeatable, but perhaps Mario could
double-check that?

...Robert

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] inheritance, and plans

2009-02-06 Thread Robert Haas
Just guessing here, but what values are you using for
join_collapse_limit and from_collapse_limit, and what happens if you
make them much bigger (like 100)?

...Robert

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] inheritance, and plans

2009-02-06 Thread Grzegorz Jaśkiewicz
On Fri, Feb 6, 2009 at 6:15 PM, Robert Haas  wrote:
> Just guessing here, but what values are you using for
> join_collapse_limit and from_collapse_limit, and what happens if you
> make them much bigger (like 100)?
both default values, afair = 8.

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] inheritance, and plans

2009-02-06 Thread Grzegorz Jaśkiewicz
On Fri, Feb 6, 2009 at 6:20 PM, Grzegorz Jaśkiewicz  wrote:
> On Fri, Feb 6, 2009 at 6:15 PM, Robert Haas  wrote:
>> Just guessing here, but what values are you using for
>> join_collapse_limit and from_collapse_limit, and what happens if you
>> make them much bigger (like 100)?

changing them to 100 doesn't help a bit.



-- 
GJ

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Greg Smith

On Fri, 6 Feb 2009, Bruce Momjian wrote:


Stupid question, but why do people bother with the Perc line of cards if
the LSI brand is better?


Because when you're ordering a Dell server, all you do is click a little 
box and you get a PERC card with it.  There aren't that many places that 
carry the LSI cards either, so most people are looking at "get the PERC as 
part of a supported package from Dell" vs. "have one rogue component I 
bought from random reseller mixed in, complicating all future support 
calls".


The failure in this logic is assuming that "support" from Dell includes 
making sure the box actually performs well.  If it works at any speed, 
that's good enough for Dell.


--
* Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] inheritance, and plans

2009-02-06 Thread Tom Lane
=?UTF-8?Q?Grzegorz_Ja=C5=9Bkiewicz?=  writes:
> so my question is, why isn't postgres use index on some tables , and
> search for the X > N individually ?

The UNION arms have to all be the same data type in order to have
restrictions pushed down through the UNION.  You did not show us
the table declarations for your first example, but I bet that updateid
isn't the same type in both.  (And yes, a domain is different from its
underlying type for this purpose.)

In the second example, "1" isn't even the right base type let alone
the same domain.

regards, tom lane

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Scott Carey
On 2/6/09 9:53 AM, "Arjen van der Meijden"  wrote:

When we purchased our Perc 5/e with MD1000 filled with 15 15k rpm sas
disks, my colleague actually spend some time benchmarking the PERC and a
ICP Vortex (basically a overclocked Adaptec) on those drives.
Unfortunately he doesn't have too many comparable results, but it
basically boiled down to quite good scores for the PERC and a bit less
for the ICP Vortex.

Adaptec's previous generation stuff was not very good.  Its the '5IE5' (I = 
internal port count, E = external) series that is much improved, and very much 
like the Areca 16xxx series.  Maybe not as good - I haven't seen a head to head 
on those two but they are built very similarly.
I have no idea if the Vortex referred to correlates with this newer generation 
card or the old generation.  But if it was in relation to a Perc 5, which is 
also an older (PCI-X, not PCIe) generation, then I'm not sure how much this 
relates to the Perc 6 and new Adaptecs or Areca 16xxx series which are all PCIe.

One manufacturer may be good in one generation and stink in another.  For those 
long established providers, every generation tends to yield new winners and 
losers.  Some are more often on the top or the bottom, but it wouldn't be a 
total shocker a recent low performer like LSI's or 3Ware had a next generation 
product that ended up on or near the top, or if the next PERC is one of the 
better ones.  What is more consistent between generations is the management 
software and recovery process.


[PERFORM] explanation of some configs

2009-02-06 Thread Thomas Finneid

Hi

I am having some problem understanding the details of a couple of 
configuration directives. Here  is what I think I understand it so far:


effective_cache_size
- specifies in how many B/KB/MB will be used to cache data
  between pg and the OS?
- what is actually cached and how it is actually used by the
  planner and how does it affect the planner and the queries?
  -in other words, when and why do I need to set this parameter?

checkpoint_segments
- specifies the number of segments?
  size: number*16MB?
- means when number of WAL segments has filled up and matches
  checkpoint segment, dirty data is written to disk?
- so it specifies how much data is stored in memory/wal before
  written to disk and therefore affects recoverability?

regards

thomas

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] explanation of some configs

2009-02-06 Thread Joshua D. Drake
On Sat, 2009-02-07 at 01:52 +0100, Thomas Finneid wrote:
> Hi
> 
> I am having some problem understanding the details of a couple of 
> configuration directives. Here  is what I think I understand it so far:
> 
> effective_cache_size
>   - specifies in how many B/KB/MB will be used to cache data
> between pg and the OS?
>   - what is actually cached and how it is actually used by the
>planner and how does it affect the planner and the queries?
> -in other words, when and why do I need to set this parameter?

This is just a hint to tell the planner how much cache will generally be
available. The number should be reflective of your shared buffers +
available operating system cache. If you database is postgresql only you
can generally set this very high 75% of available ram. If not then you
need to tone it down.

> 
> checkpoint_segments
>   - specifies the number of segments?

The number of segments that will be used before a checkpoint is forced.

> size: number*16MB?

Yes but they are not preallocated.

>   - means when number of WAL segments has filled up and matches
>checkpoint segment, dirty data is written to disk?

A checkpoint occurs.

Joshua D. Drake

-- 
PostgreSQL - XMPP: jdr...@jabber.postgresql.org
   Consulting, Development, Support, Training
   503-667-4564 - http://www.commandprompt.com/
   The PostgreSQL Company, serving since 1997


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Getting error while running DBT2 test for PostgreSQL

2009-02-06 Thread Mark Wong
On Wed, Feb 4, 2009 at 2:42 AM, Rohan Pethkar  wrote:
> Hi All,
>
> I am trying to conduct DBT2 test for PostgreSQL. I am getting following
> errors when I run client and driver separately(Instead of running
> dbt2-run-workload script). I am also attaching exact error log file for the
> same.
>
> tid:1073875280 /home/rohan/NEW_DBT2/Installer/dbt2/src/driver.c:496
> connect_to_client() failed, thread exiting...
> Wed Feb  4 20:00:52 2009
> tid:1074010448 /home/rohan/NEW_DBT2/Installer/dbt2/src/driver.c:496
> connect_to_client() failed, thread exiting...
> .
> .
> .
>
> Can someone please provide inputs on this? Do I need to make specific
> changes if any then please let me know.
>
> Any help on this will be appreciated.
>
> Thanks in advance for your help.

Hi Rohan,

As I mentioned on the osdldbt list, for questions like this it would
be more appropriate to cc the osdldbt-gene...@lists.sourceforge.net
instead of the -performance list.

It's not clear why the driver can't connect to the client.  Can you
provide all of the log files somewhere?

Regards,
Mark

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Can't locate Test/Parser/Dbt2.pm in DBT2 tests

2009-02-06 Thread Mark Wong
On Fri, Feb 6, 2009 at 3:46 AM, Richard Huxton  wrote:
> Rohan Pethkar wrote:
>> Hi All,
>>
>> I am conductingDBT2 tests on PostgreSQL. After completing the test while
>> analyzing and creating the results I am getting following error:
>>
>> ./dbt2-run-workload: line 514:   731 Terminated  dbt2-client
>> ${CLIENT_COMMAND_ARGS} -p ${PORT} -o ${CDIR} >
>> ${CLIENT_OUTPUT_DIR}/`hostname`/client-${SEG}.out 2>&1
>> waiting for server to shut down done
>> server stopped
>> Can't locate Test/Parser/Dbt2.pm in @INC (@INC contains:
>> /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi
>> /usr/lib64/perl5/site_perl/5.8.7/x86_64-linux-thread-multi
>> /usr/lib64/perl5/site_perl/5.8.6/x86_64-linux-thread-multi
>> /usr/lib64/perl5/site_perl/5.8.5/x86_64-linux-thread-multi
>> /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.7
>> /usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_perl/5.8.5
>> /usr/lib/perl5/site_perl
>> /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi
>> /usr/lib64/perl5/vendor_perl/5.8.7/x86_64-linux-thread-multi
>> /usr/lib64/perl5/vendor_perl/5.8.6/x86_64-linux-thread-multi
>> /usr/lib64/perl5/vendor_perl/5.8.5/x86_64-linux-thread-multi
>> /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl/5.8.7
>> /usr/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.5
>> /usr/lib/perl5/vendor_perl /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi
>> /usr/lib/perl5/5.8.8 .) at
>> /home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-post-process line 13.
>
> Well, if Test::Parser::Dbt2 isn't in somewhere in that list of
> directories, you'll need to tell perl where to look. Simplest is
> probably just to:
>  export PERL5LIB="/path/to/extra/libs"
> before running your tests.
>
>> Can't exec "gnuplot": No such file or directory at
>> /home/rohan/NEW_DBT2/Installer/DBT2_SETUP/bin/dbt2-pgsql-analyze-stats line
>> 113.
>
> It also looks like you're missing gnuplot for your charts.
>
>> I ma not sure why it doesn't find Test/Parser/Dbt2.pm even if I have
>> installed DBT2 completely. Did I miss any steps? Do I need to install some
>> extra packages? If any then please let me know.
>
> You can always "perldoc perlrun" for more info (google it if you don't
> have docs installed locally).

Hi Rohan,

In addition to what Richard said, I'm guessing you don't have those
perl modules installed.  In the main README file lists the perl
modules required for the post processing of the data colelcted, which
is what failed here:

"The data analysis scripts requires two additional Perl packages to be
installed,
which are not checked by configure.  They are Statistics::Descriptive and
Test::Parser.  To generate HTML reports, Test::Reporter is required."

Sorry, it's still a work in progress., as slow as that may be.

Regards,
Mark

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance