Searching for big differences between values

2017-11-29 Thread Durumdara
Hello! Somewhere the users made mistakes on prices (stock). I need to search for big differences between values. For example: 20 21 21,5 30 28 .. 46392 <- 46392 <- But it could be: 42300 43100 44000 43800 65000 <- 42100 Human eye could locate these values, but there we need to check 30.000 ar

Re: pg_replication_slots

2017-11-29 Thread Emanuel Alvarez
On Wed, Nov 29, 2017 at 12:39 PM, Nicola Contu wrote: > Hello, Hi! > I just have few questions on the replication slots : > > - is it possible to get size of the slot? I use something like this to know where my slot is at: SELECT pg_xlog_location_diff(pg_current_xlog_location(), restart_lsn)

Re: seq vs index scan in join query

2017-11-29 Thread Jeff Janes
On Tue, Nov 28, 2017 at 10:55 PM, Emanuel Alvarez wrote: > hi all, > > we're in the process of optimizing some queries and we've noted a case > where the planner prefers a sequential scan instead of using an index, > while the index scan is actually much faster. to give you some > context: we hav

Re: seq vs index scan in join query

2017-11-29 Thread Emanuel Alvarez
Thank you all for your responses! On Wed, Nov 29, 2017 at 7:31 AM, legrand legrand wrote: > Hi, > > Could you give us the partitions (ranges values) and indexes definition for > result table ? We partition by month, they usually start the 20th of each month (this was the date we partitioned the

Partition pruning / agg push down for star schema in pg v11

2017-11-29 Thread legrand legrand
Hello, Working on Oracle migration POCs, I'm very interested in v11 and declarative partitioning optimizations. I have a typical star schema, having dimension tables "product", "calendar" and "country" and a fact table "sales". This fact table is partitionned by time (range by month) and country

Re: seq vs index scan in join query

2017-11-29 Thread Laurenz Albe
Andres Freund wrote: > On 2017-11-29 18:17:18 +0100, Laurenz Albe wrote: > > That is because the execution with the sequential scan touched > > 26492 + 80492 = 106984 blocks, while the second execution touched > > 311301 + 48510 = 359811 blocks, more than three times as many. > > That's not neces

Re: large numbers of inserts out of memory strategy

2017-11-29 Thread Rory Campbell-Lange
On 28/11/17, Rob Sargent (robjsarg...@gmail.com) wrote: > > On 11/28/2017 10:50 AM, Ted Toth wrote: > > On Tue, Nov 28, 2017 at 11:19 AM, Rob Sargent wrote: > > > > On Nov 28, 2017, at 10:17 AM, Ted Toth wrote: > > > > > > > > I'm writing a migration utility to move data from non-rdbms data > >

Re: seq vs index scan in join query

2017-11-29 Thread Andres Freund
On 2017-11-29 18:17:18 +0100, Laurenz Albe wrote: > That is because the execution with the sequential scan touched > 26492 + 80492 = 106984 blocks, while the second execution touched > 311301 + 48510 = 359811 blocks, more than three times as many. That's not necessarily said. What those count are

Re: seq vs index scan in join query

2017-11-29 Thread Laurenz Albe
Emanuel Alvarez wrote: > the problematic query looks like this: > > SELECT keywords.strategy_id, results.position, results.created_at FROM results > JOIN keywords ON results.keyword_id = keywords.id > WHERE results.account_id = 1 > AND results.created_at >= '2017-10-25 00:00:00.00' >

Re: large numbers of inserts out of memory strategy

2017-11-29 Thread Steven Lembark
> I'm pretty new to postgres so I haven't changed any configuration > setting and the log is a bit hard for me to make sense of :( Diving into the shark tank is a helluva way to learn how to swim :-) Are you interested in finding doc's on how to deal with the tuning? -- Steven Lembark

Re: large numbers of inserts out of memory strategy

2017-11-29 Thread Steven Lembark
> > what tools / languages ate you using? > > I'm using python to read binary source files and create the text files > contains the SQL. Them I'm running psql -f . Then chunking the input should be trivial. There are a variety of techniques you can use to things like disable indexes during loa

pg_replication_slots

2017-11-29 Thread Nicola Contu
Hello, I just have few questions on the replication slots : - is it possible to get size of the slot? - if the slave is down, the table grows, when the slave comes up again, will the table be flushed after pushing wals? - will they impact performances on the master? I'm just worried about the siz

Re: seq vs index scan in join query

2017-11-29 Thread Marti Raudsepp
Hi On Wed, Nov 29, 2017 at 8:55 AM, Emanuel Alvarez wrote: > on the other hand, if we disable sequential scans (SET enable_seqscan > = 0), we see than not only the query runs faster but the cost seems to > be lower, as seen in the query plan [2]. True, the cost of the scan itself is lower, but

Re: large numbers of inserts out of memory strategy

2017-11-29 Thread Tom Lane
Ted Toth writes: > On Tue, Nov 28, 2017 at 9:59 PM, Tom Lane wrote: >> So whatever's going on here, there's more to it than a giant client-issued >> INSERT (or COPY), or for that matter a large number of small ones. What >> would seem to be required is a many-megabyte-sized plpgsql function body

Re: large numbers of inserts out of memory strategy

2017-11-29 Thread Ted Toth
On Tue, Nov 28, 2017 at 9:59 PM, Tom Lane wrote: > Brian Crowell writes: >> On Tue, Nov 28, 2017 at 12:38 PM, Tomas Vondra >> wrote: >>> So what does the script actually do? Because psql certainly is not >>> running pl/pgsql procedures on it's own. We need to understand why >>> you're getting OOM

SV: SV: Refreshing materialized views

2017-11-29 Thread Henrik Uggla
I finally managed to sort out all needed permissions and mappings. Thanks for all replies! cheers Henrik Från: Ben Primrose Skickat: den 29 november 2017 12:42:56 Till: Daevor The Devoted Kopia: hendrik.ug...@kristianstad.se; pgsql-general@lists.postgresql

Re: vacuumdb fails with error pg_statistic_relid_att_inh_index constraint violation after upgrade to 9.6

2017-11-29 Thread Swapnil Vaze
Hello, Thanks for the reply. We deleted those rows and re run vacuum and analyze and it worked fine. Thanks, Thanks, Swapnil Vaze

Re: SV: Refreshing materialized views

2017-11-29 Thread Ben Primrose
It may be simpler to just run the query from the materialized view definition as the user that you want to refresh the mv. On Tue, Nov 28, 2017 at 10:30 PM, Daevor The Devoted wrote: > > > On 28 Nov 2017 5:18 pm, "Tom Lane" wrote: > > Henrik Uggla writes: > > The underlying tables are foreign

Re: seq vs index scan in join query

2017-11-29 Thread legrand legrand
Hi, Could you give us the partitions (ranges values) and indexes definition for result table ? Regards PAscal -- Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html