Re: Bitmap scan seem like such a strange choice when "limit 1"

2020-09-04 Thread Michael Lewis
Index Scan using bigtable_y2020_cars_ref_idx on bigtable_y2020 bigtable (cost=0.57..2966738.51 rows=2873818 width=636) (actual time=0.026..0.026 rows=1 loops=1) Given the system expects to get almost 3 million rows when it should be just 1, it seems like a stats problem to me. How is ndistinct on

Re: 12.3 replicas falling over during WAL redo

2020-09-04 Thread Ben Chobot
Alvaro Herrera wrote on 8/3/20 4:54 PM: On 2020-Aug-03, Ben Chobot wrote: Alvaro Herrera wrote on 8/3/20 2:34 PM: On 2020-Aug-03, Ben Chobot wrote: dd if=16605/16613/60529051 bs=8192 count=1 seek=6501 of=/tmp/page.6501 If I use skip instead of seek Argh, yes, I did correct that in my tes

Re: Check for duplicates before inserting new rows

2020-09-04 Thread Rich Shepard
On Fri, 4 Sep 2020, Chris Sterritt wrote: Assuming some simple table structures (I've not included PK or FK definitions for simplicity): Chris, Thanks very much. Stay well, Rich

Can I get some advice regarding logical replication server?

2020-09-04 Thread Sang Gyu Kim
Hello Ranjan, This is Sang Kim and I recently set up a logical replication server on postgresql 11 but got an error. While google for the issue, I found some pages and this email address is the responder. Please give me some idea how to fix the issue on postgresql 11. I set up logical replicatio

Re: Dependency problem using community repo on Redhat 7

2020-09-04 Thread Devrim Gündüz
Hi, On Thu, 2020-09-03 at 18:31 +, Lawrence Layhee wrote: > We are having a dependency problem when using the community repo on > redhat 7 > When we install postgresql12-devel-12.4-1PGDG.rhel7.x86_64 we get the > issue below. https://yum.postgresql.org/news/devel-rpms-require-a-new-reposit

Re: Check for duplicates before inserting new rows

2020-09-04 Thread Chris Sterritt
On 04/09/2020 14:21, Rich Shepard wrote: This is a new issue for me: I've received data from a different source and need to add non-duplicates to two tables in the database. Each row in the new data has a station ID and associated measurements. The existing database includes a table for statio

Re: Check for duplicates before inserting new rows

2020-09-04 Thread Chris Sterritt
On 04/09/2020 15:46, Rich Shepard wrote: On Fri, 4 Sep 2020, Olivier Gautherot wrote: First of all, what version of PostgreSQL are you using? Olivier, 12.2. One way would be to add a UNIQUE constraint and perform for each row of the source table a INSERT ... ON CONFLICT DO NOTHING (see h

Re: Dependency problem using community repo on Redhat 7

2020-09-04 Thread Adrian Klaver
On 9/3/20 11:31 AM, Lawrence Layhee wrote: Hi, We are having a dependency problem when using the community repo on redhat 7 When we install postgresql12-devel-12.4-1PGDG.rhel7.x86_64 we get the issue below. Redhat doesn’t support the dependencies. Any ideas? We are trying to get the ruby ge

Re: Check for duplicates before inserting new rows

2020-09-04 Thread Rich Shepard
On Fri, 4 Sep 2020, Olivier Gautherot wrote: First of all, what version of PostgreSQL are you using? Olivier, 12.2. One way would be to add a UNIQUE constraint and perform for each row of the source table a INSERT ... ON CONFLICT DO NOTHING (see https://www.postgresql.org/docs/current/sql-i

Re: Bitmap scan seem like such a strange choice when "limit 1"

2020-09-04 Thread Tom Lane
Klaudie Willis writes: > I'd like to add, that when I do the same query DIRECTLY on the bigtable_y2020 > (instead of the partition parent) it does change to "index scan" again. Yeah. I think the issue here is that add_paths_to_append_rel only considers cheapest-total paths for the member relati

Re: Check for duplicates before inserting new rows

2020-09-04 Thread Rich Shepard
On Fri, 4 Sep 2020, George Woodring wrote: I would suggest creating a temp table based on the original table and loading the data into it first. You can then purge the duplicates. George, I hadn't thought of this. Using a duplicate table without a PK would work well if there's only one attrib

Re: Check for duplicates before inserting new rows

2020-09-04 Thread George Woodring
I would suggest creating a temp table based on the original table and loading the data into it first. You can then purge the duplicates. George Woodring iGLASS Networks www.iglass.net On Fri, Sep 4, 2020 at 9:21 AM Rich Shepard wrote: > This is a new issue for me: I've received data from a di

Check for duplicates before inserting new rows

2020-09-04 Thread Rich Shepard
This is a new issue for me: I've received data from a different source and need to add non-duplicates to two tables in the database. Each row in the new data has a station ID and associated measurements. The existing database includes a table for station information and another for measurements m

Re: Bitmap scan seem like such a strange choice when "limit 1"

2020-09-04 Thread Klaudie Willis
> PostgreSQL estimates that 2817675 rows satisfy the index condition and expects > that it will have to scan many of them before it finds one that satisfies the > filter condition. That turns out to be a wrong guess. > > You could create an index on (cars_ref, t), then PostgreSQL will certainly > p

Re: Bitmap scan seem like such a strange choice when "limit 1"

2020-09-04 Thread Laurenz Albe
On Fri, 2020-09-04 at 11:42 +, Klaudie Willis wrote: > Postgres 13 beta3 > > set enable_bitmapscan=1; -- default > explain (analyze,buffers) > select * > from bigtable > where cars_ref = 1769854207 and t > '2020-01-01'::timestamp and t < > '2021-01-01'::timestamp > limit 1 > > Short sto

Bitmap scan seem like such a strange choice when "limit 1"

2020-09-04 Thread Klaudie Willis
Postgres 13 beta3 set enable_bitmapscan=1; -- default explain (analyze,buffers) select * from bigtable where cars_ref = 1769854207 and t > '2020-01-01'::timestamp and t < '2021-01-01'::timestamp limit 1 Short story. Big table > 100M rows. b-tree index on cars_ref, the t constraints limits it to

Re: how to get top plan of GatherMerge in OSS10

2020-09-04 Thread David Rowley
On Fri, 4 Sep 2020 at 20:21, Yang, Rong wrote: > in OSS 10, how to make the top plan of the plan tree to GatherMerge with > ‘except all’ in sql? (I guess you're talking about PostgreSQL 10. I'm not sure what OSS 10 is.) The ability for Gather Merge to work with Init Plan values was only added

Aw: Implement a new data type

2020-09-04 Thread Karsten Hilbert
> If I want to create a new type Person (firstname varchar, lastname varchar, > address varchar ...) > what is the best way to procede in Postgresql   The best way is to re-evaluate the "I want". Karsten

Implement a new data type

2020-09-04 Thread mohand oubelkacem makhoukhene
Hello, If I want to create a new type Person (firstname varchar, lastname varchar, address varchar ...) what is the best way to procede in Postgresql Create type or create extension ? What are advantages /disadvantages of both solutions ? Thank you and best regards Mohand

Re: How to enumerate partitions from a window function?

2020-09-04 Thread Alban Hertroys
On Thu, 3 Sep 2020 at 20:59, Michael Lewis wrote: > It seems like you are maybe wanting this- If the previous row is the same, > then get the previous row's run_nr. If it is different, then increment. > > case when lag( property_A ) over() = property_A and lag( property_B ) > over() = property_B

how to get top plan of GatherMerge in OSS10

2020-09-04 Thread Yang, Rong
Hello~ When I look at the plan for a query in OSS11 and OSS10, the plan tree structure is different. OSS10.13: postgres=# explain (costs off) select C,((select c_int from table1 where C<=2)except all(select c_int from table1 where C=2)) from table1 where C < 100 order by C;