Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Ciprian Dorin Craciun
On Sun, Nov 23, 2008 at 12:32 AM, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > >> On 21 Lis, 13:50, [EMAIL PROTECTED] ("Ciprian Dorin Craciun") >> wrote: > >> > What have I observed / tried: >> > * I've tested without the primary key and the index, and the >> > results were the best for inse

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Ciprian Dorin Craciun
On Sun, Nov 23, 2008 at 3:09 AM, Scott Marlowe <[EMAIL PROTECTED]> wrote: > On Sat, Nov 22, 2008 at 5:54 PM, Scara Maccai <[EMAIL PROTECTED]> wrote: >> Since you always need the timestamp in your selects, have you tried indexing >> only the timestamp field? >> Your selects would be slower, but sin

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Ciprian Dorin Craciun
On Sun, Nov 23, 2008 at 12:26 AM, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > Ciprian Dorin Craciun escribió: > >> I've tested also Sqlite3 and it has the same behavior as >> Postgres... Meaning at beginning it goes really nice 20k inserts, >> drops to about 10k inserts, but after a few million

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Ciprian Dorin Craciun
On Sun, Nov 23, 2008 at 1:02 AM, Tom Lane <[EMAIL PROTECTED]> wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: >> The problem is, most likely, on updating the indexes. Heap inserts >> should always take more or less the same time, but index insertion >> requires walking down the index struct fo

Re: [GENERAL] Postgres mail list traffic over time

2008-11-22 Thread Bruce Momjian
Magnus Hagander wrote: > Bruce Momjian wrote: > > Ron Mayer wrote: > >> Joshua D. Drake wrote: > >>> On Fri, 2008-11-21 at 08:18 -0800, Ron Mayer wrote: > Bruce Momjian wrote: > > Tom Lane wrote: > >> ... harder to keep > >> up with the list traffic; so something is happening that

[GENERAL] [Q]updating multiple rows with Different values

2008-11-22 Thread V S P
Hello, searched documentation, FAQ and mailing list archives (mailing list archive search is volumous :-) ) but could not find an answer: I would like to be able to update several rows to different values at the same time In oracle this used to be called Array update or 'collect' update or 'bul

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Scott Marlowe
On Sat, Nov 22, 2008 at 5:54 PM, Scara Maccai <[EMAIL PROTECTED]> wrote: > Since you always need the timestamp in your selects, have you tried indexing > only the timestamp field? > Your selects would be slower, but since client and sensor don't have that > many distinct values compared to the nu

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Scara Maccai
Since you always need the timestamp in your selects, have you tried indexing only the timestamp field? Your selects would be slower, but since client and sensor don't have that many distinct values compared to the number of rows you are inserting maybe the difference in selects would not be that

Re: [GENERAL] delete commands fails silently to delete primary key

2008-11-22 Thread Tom Lane
"Andrus" <[EMAIL PROTECTED]> writes: > I have table in 8.1.4 which tracks users logged into db There have been a number of index-corruption bugs fixed since 8.1.4 ... In particular, if it's possible that any of these clients abort before committing these insertions, the vacuum race condition bug

Re: [GENERAL] date stamp on update?

2008-11-22 Thread Martin Gainty
Create Table TableName( ColumnName DATE Sysdate; ); ALTER TABLE TableName ALTER COLUMN ColumnName { SET DEFAULT newdefaultvalue | DROP OLDDEFAULT } HTH Martin __ Disclaimer and confidentiality note Everything in this e-mail and any attachmen

Re: [GENERAL] date stamp on update?

2008-11-22 Thread Tom Lane
"blackwater dev" <[EMAIL PROTECTED]> writes: > Is there a datatype in postgres that will automatically update the date when > the row is updated? No, and it's conceptually impossible to make that happen at the datatype level. Use a trigger. regards, tom lane -- Sent via

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > The problem is, most likely, on updating the indexes. Heap inserts > should always take more or less the same time, but index insertion > requires walking down the index struct for each insert, and the path to > walk gets larger the more data you have.

Re: [GENERAL] Postgres mail list traffic over time

2008-11-22 Thread Gregory Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Bruce Momjian <[EMAIL PROTECTED]> writes: >> Tom Lane wrote: >>> So, to a first approximation, the PG list traffic has been constant >>> since 2000. Not the result I expected. > >> I also was confused by its flatness. I am finding the email traffic >> almos

Re: [GENERAL] Postgres mail list traffic over time

2008-11-22 Thread Gregory Stark
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Richard Huxton wrote: > >> Some of the EXPLAINs on the performance list are practically impossible >> to read unless you've got the time to cut+paste and fix line-endings. > > Maybe we should start recommending people to post those via > http://explain-

[GENERAL] date stamp on update?

2008-11-22 Thread blackwater dev
Is there a datatype in postgres that will automatically update the date when the row is updated? I know I can do a timestamp and set the default to now() but once the row is inserted, and then edited, I want the column updated without editing my application code or adding a trigger. Is this possi

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Alvaro Herrera
> On 21 Lis, 13:50, [EMAIL PROTECTED] ("Ciprian Dorin Craciun") > wrote: > >     What have I observed / tried: > >     * I've tested without the primary key and the index, and the > > results were the best for inserts (600k inserts / s), but the > > readings, worked extremly slow (due to the lack

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Alvaro Herrera
Ciprian Dorin Craciun escribió: > I've tested also Sqlite3 and it has the same behavior as > Postgres... Meaning at beginning it goes really nice 20k inserts, > drops to about 10k inserts, but after a few million records, the HDD > led starts to blink non-stop, and then it drops to unde 1k

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread David Wilson
On Sat, Nov 22, 2008 at 4:54 PM, Ciprian Dorin Craciun <[EMAIL PROTECTED]> wrote: > On Sat, Nov 22, 2008 at 11:51 PM, Scott Marlowe <[EMAIL PROTECTED]> wrote: >> On Sat, Nov 22, 2008 at 2:37 PM, Ciprian Dorin Craciun >> <[EMAIL PROTECTED]> wrote: >>> >>>Hello all! >> SNIP >>>So I would conc

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Ciprian Dorin Craciun
On Sat, Nov 22, 2008 at 11:51 PM, Scott Marlowe <[EMAIL PROTECTED]> wrote: > On Sat, Nov 22, 2008 at 2:37 PM, Ciprian Dorin Craciun > <[EMAIL PROTECTED]> wrote: >> >>Hello all! > SNIP >>So I would conclude that relational stores will not make it for >> this use case... > > I was wondering y

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Scott Marlowe
On Sat, Nov 22, 2008 at 2:37 PM, Ciprian Dorin Craciun <[EMAIL PROTECTED]> wrote: > >Hello all! SNIP >So I would conclude that relational stores will not make it for > this use case... I was wondering you guys are having to do all individual inserts or if you can batch some number together

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Ciprian Dorin Craciun
On Sat, Nov 22, 2008 at 8:04 PM, Shane Ambler <[EMAIL PROTECTED]> wrote: > Ciprian Dorin Craciun wrote: > >> >>I would try it if I would know that it could handle the load... Do >> you have some info about this? Any pointers about the configuration >> issues? >> >>Ciprian. >> > > > Apart fr

[GENERAL] delete commands fails silently to delete primary key

2008-11-22 Thread Andrus
I have table in 8.1.4 which tracks users logged into db CREATE TABLE "session" ( workplace character(16) NOT NULL, ipaddress character(20), logintime character(28), loggeduser character(10), CONSTRAINT session_pkey PRIMARY KEY (workplace) ); Commands executed at logon in same transaction ar

Re: [GENERAL] SQL query

2008-11-22 Thread Tom Lane
"Michael Thorsen" <[EMAIL PROTECTED]> writes: > ... I gave a simple example above, but the query runs over 2 tables > with about a million entries in each. So I am unable to verify what is > wrong, but I know the count is incorrect as I should not have more than what > is in the user_table. You co

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Shane Ambler
Ciprian Dorin Craciun wrote: I would try it if I would know that it could handle the load... Do you have some info about this? Any pointers about the configuration issues? Ciprian. Apart from the configure options at build time you should read - http://www.sqlite.org/pragma.html I

Re: [GENERAL] SQL query

2008-11-22 Thread Raymond O'Donnell
On 22/11/2008 16:07, Michael Thorsen wrote: > For the most part yes. The price and value were "real" columns, > otherwise the rest of it is the same. On a small data set I seem to get That's almost certainly the problem, so - rounding errors are causing the equality test in the join to fail. You s

Re: [GENERAL] SQL query

2008-11-22 Thread Raymond O'Donnell
On 22/11/2008 04:33, Michael Thorsen wrote: > select count(*) > from user_table u, locations l > where u.user_code = l.code > and u.price = l.price > and u.value = l.value; > > The answer to this should be 2, but when I run my query I get 4 (in fact Are you sure that's the query that'

Fwd: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Ciprian Dorin Craciun
(I'm adding the discussion also to the Postgres list.) On Fri, Nov 21, 2008 at 11:19 PM, Dann Corbit <[EMAIL PROTECTED]> wrote: > What is the schema for your table? > If you are using copy rather than insert, 1K rows/sec for PostgreSQL seems > very bad unless the table is extremely wide. T

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Ciprian Dorin Craciun
On Fri, Nov 21, 2008 at 3:12 PM, Michal Szymanski <[EMAIL PROTECTED]> wrote: > On 21 Lis, 13:50, [EMAIL PROTECTED] ("Ciprian Dorin Craciun") > wrote: >> Hello all! >> >> I would like to ask some advice about the following problem >> (related to the Dehems project:http://www.dehems.eu/): >>

Re: [GENERAL] Using Postgres to store high volume streams of sensor readings

2008-11-22 Thread Michal Szymanski
On 21 Lis, 13:50, [EMAIL PROTECTED] ("Ciprian Dorin Craciun") wrote: >     Hello all! > >     I would like to ask some advice about the following problem > (related to the Dehems project:http://www.dehems.eu/): >     * there are some clients; (the clients are in fact house holds;) >     * each devi

[GENERAL] strange commit behavior

2008-11-22 Thread Flavio Palumbo
Hi all, I'm developing a little tool in Java that manages database update throught external text files. In this tool there is an option that allows the user accepts a defined amount of errors during the update and save the data well formed. To do this I start commitment control when the proces

[GENERAL] Connecting to old 7.1 Database

2008-11-22 Thread Andy Greensted
Hi All, I need to connect to a version 7.1 PostgreSQL database. Unfortunately, I cannot get the 7.1.3 source to compile. configure gives this error: checking types of arguments for accept()... configure: error: could not determine argument types I'm not posting to pgsql-bugs because I think

[GENERAL] Best way to import a plpythonu module

2008-11-22 Thread Chris McDonald
I am using plpythonu on linux version postgresql-plpython-8.2.9-1.fc7. Consider a python class called Wibble which is written into a python module wibble.py. I wish to use Wibble from within a plpythonu procedure. If I simply do: from wibble import Wibble then I am told the object was not found

[GENERAL] PgAdminIII - RAISE WARNING prints in one line

2008-11-22 Thread Michal Szymanski
Hi, When I call pgsql procedure in Postgres 8.3 from pgAdmin III everything is printed in one line: WARNING: Odczytany token>'+48'b- zb-z< Całkowity czas wykonania zapytania:16 ms. 1 wierszy zwróconych. It is happening in all our computers and it was started after migration to 8.3. It is small

[GENERAL] how to cite postgresql?

2008-11-22 Thread Tomas Lanczos
I am using Postgresql to store all my research related data. At the moment I am just finishing my PhD thesis and I want to cite postgresql correctly but can't find how to do it. Could somebody give me an advice? Many thanks tomas -- Sent via pgsql-general mailing list (pgsql-general@postgresq

[GENERAL] SQL query

2008-11-22 Thread Michael Thorsen
I am running the Postgres(8.2.11) on Windows. I have 2 tables, one with users and one with locations. user_table --- user_id user_code price value 1 245.23 -97.82 2 3 42.67 -98.32 3

Re: [GENERAL] Database access over the Internet...

2008-11-22 Thread Scott Ribe
> You did not understand the problem 1) You did not explain the problem. 2) It's your problem, not ours. Your demand that thousands of other people adapt to your unusual problem is absurdly self-absorbed. Get decent email service, then subscribe from there. (Or go away. Either works for t

Re: [GENERAL] Database access over the Internet...

2008-11-22 Thread Bayless Kirtley
Michelle, I don't think the list is going to change its operations for one disgruntled user. Since you seem unwilling or unable to employ the advice already given, maybe your only acceptable option is to unsubscribe from the list. At least that would eliminate much of the noise that currently i

Re: [GENERAL] Postgres mail list traffic over time

2008-11-22 Thread Magnus Hagander
Bruce Momjian wrote: > Ron Mayer wrote: >> Joshua D. Drake wrote: >>> On Fri, 2008-11-21 at 08:18 -0800, Ron Mayer wrote: Bruce Momjian wrote: > Tom Lane wrote: >> ... harder to keep >> up with the list traffic; so something is happening that a simple >> volume count doesn't ca

Re: [GENERAL] Postgres mail list traffic over time

2008-11-22 Thread Bruce Momjian
Ron Mayer wrote: > Joshua D. Drake wrote: > > On Fri, 2008-11-21 at 08:18 -0800, Ron Mayer wrote: > >> Bruce Momjian wrote: > >>> Tom Lane wrote: > ... harder to keep > up with the list traffic; so something is happening that a simple > volume count doesn't capture. > >> If measured

Re: [GENERAL] Postgres mail list traffic over time

2008-11-22 Thread Stefan Kaltenbrunner
Alvaro Herrera wrote: Sam Mason wrote: the following has links to more: http://markmail.org/search/?q=list:org.postgresql Wow, the spanish list is the 3rd in traffic after hackers and general! yeah and that tom lane guy sent over 77000(!!!) mails to the lists up to now ... Stefan --

Re: [GENERAL] pgAdmin error

2008-11-22 Thread Magnus Hagander
Dave Page wrote: > On Sat, Nov 22, 2008 at 6:35 AM, David <[EMAIL PROTECTED]> wrote: >> I am trying to use pgAdmin 1.8.4 to edit the pg_hba.conf file on a >> PostgreSQL 8.3 database running on Ubuntu 8.10. I get the following >> error message: >> >> An error has occurred: >> >> ERROR: absolute pa

Re: [GENERAL] Postgres mail list traffic over time

2008-11-22 Thread Peter Eisentraut
On Friday 21 November 2008 19:10:45 Tom Lane wrote: > Yeah, I think this is most probably explained by repeat postings > of successive versions of large patches. Still, Ron might be on to > something. I had not considered message lengths in my previous > numbers ... Also consider that since we s

Re: [GENERAL] pgAdmin error

2008-11-22 Thread Dave Page
On Sat, Nov 22, 2008 at 6:35 AM, David <[EMAIL PROTECTED]> wrote: > I am trying to use pgAdmin 1.8.4 to edit the pg_hba.conf file on a > PostgreSQL 8.3 database running on Ubuntu 8.10. I get the following > error message: > > An error has occurred: > > ERROR: absolute path not allowed > CONTEXT: