Re: [PERFORM] Heavy contgnous load

2011-11-13 Thread Pavel Stehule
Hello 2011/11/9 kzsolt : > I try to found out more information. Looks like the COUNT(*) is not the > strongest part of pgr therfore I do a worakround. After this I have the > folwing result: > > Below the 1Mrec the row insert time is ~23msec. Above the 7Mrec the insert > time is ~180msec. > * use

Re: [PERFORM] Heavy contgnous load

2011-11-13 Thread kzsolt
I try to found out more information. Looks like the COUNT(*) is not the strongest part of pgr therfore I do a worakround. After this I have the folwing result: Below the 1Mrec the row insert time is ~23msec. Above the 7Mrec the insert time is ~180msec. I belive I use the fastest index type (def

Re: [PERFORM] Heavy contgnous load

2011-11-11 Thread kzsolt
Looks like I found more magic. My table is: each record near 1kbyte, contain dozen col some text some numeric, some of the numeric columns are indexed. The database located at ramdisk (tmpfs) ((I hope)). The table is contignously filled with rows. If the table has less than 4Mrec then looks like

Re: [PERFORM] Heavy contgnous load

2011-10-23 Thread kzsolt
So guys, lot of thank you for all of the explanation and ideas! Jeff Janes wrote: > What happens if the database has a hiccup and can't accept records for > a few seconds or minutes? Craig Ringer wrote: > If you really need absolutely maximum insert performance, you should > just use a flat fil

Re: [PERFORM] Heavy contgnous load

2011-10-20 Thread Jeff Janes
On Tue, Oct 18, 2011 at 5:09 AM, kzsolt wrote: > Dear Anybody! > > I use pgr to store records. But the characterisitc of the record traffic are > special. For example 50 of them arrived in one sec contignously trough weeks > and aligned interally trough tables. What happens if the database has a

Re: [PERFORM] Heavy contgnous load

2011-10-19 Thread Craig Ringer
On 10/20/2011 02:55 AM, kzsolt wrote: "try to have as few indexes as possible and do your inserts inside transactions in batches, rather than one-by-one as individual statements. " That is the main problem. I use now few index as possible. Unfortunately the one-by-one INSERT is nature of or syste

Re: [PERFORM] Heavy contgnous load

2011-10-19 Thread kzsolt
Kevin Grittner wrote: > Rollback activity? What rollback activity? When you're doing what? > What is the exact message? I mean here some kind of option to save reources. For example mysql has table (storage) type where no transaction support (rollback) in. This make the all server faster and us

Re: [PERFORM] Heavy contgnous load

2011-10-19 Thread kzsolt
"try to have as few indexes as possible and do your inserts inside transactions in batches, rather than one-by-one as individual statements. " That is the main problem. I use now few index as possible. Unfortunately the one-by-one INSERT is nature of or system. To join (batch) inserts is require sp

Re: [PERFORM] Heavy contgnous load

2011-10-18 Thread Craig Ringer
On 10/18/2011 08:09 PM, kzsolt wrote: What is important think for this task I do not need any tranasction. So the COMMIT and ROLLBACK feature is useless. The question is how I minimize the rollback activity to free resoureces? Actually, you do need transactions, because they're what prevents y

Re: [PERFORM] Heavy contgnous load

2011-10-18 Thread Kevin Grittner
kzsolt wrote: > I use pgr to store records. But the characterisitc of the record > traffic are special. For example 50 of them arrived in one sec > contignously trough weeks and aligned interally trough tables. > To absorb this traffic I put the pgr database to ramdisk (fast as > possible). C

[PERFORM] Heavy contgnous load

2011-10-18 Thread kzsolt
Dear Anybody! I use pgr to store records. But the characterisitc of the record traffic are special. For example 50 of them arrived in one sec contignously trough weeks and aligned interally trough tables. To absorb this traffic I put the pgr database to ramdisk (fast as possible). But after more