Re: converting text to bytea

2021-02-25 Thread Yambu
Hi Is there a reason why i'm getting text when i run the below select convert_to('some_text', 'UTF8')i get back 'some_text' regards On Mon, Feb 22, 2021 at 9:09 AM Pavel Stehule wrote: > Hi > > po 22. 2. 2021 v 7:37 odesílatel Yambu napsal: > >> Hello >> >> This sounds simple, but im not

Re: Batch update million records in prd DB

2021-02-25 Thread Yi Sun
Hi Michael, Thank you for your reply We found that each loop take time is different, it will become slower and slower, as our table is big table and join other table, even using index the last 1000 records take around 15 seconds, will it be a problem? Will other concurrent update have to wait for

Re: Deleting takes days, should I add some index?

2021-02-25 Thread Alexander Farber
Hello, revisiting an older mail on the too long deletion times (in PostgreSQL 13.2)... I have followed the advices here, thank you - On Fri, Nov 27, 2020 at 4:15 PM Guillaume Lelarge wrote: > Le ven. 27 nov. 2020 à 16:05, Alvaro Herrera a > écrit : > >> On 2020-Nov-27, Alexander Farber wrote:

Re: Deleting takes days, should I add some index?

2021-02-25 Thread Pavel Stehule
čt 25. 2. 2021 v 14:06 odesílatel Alexander Farber < alexander.far...@gmail.com> napsal: > Hello, revisiting an older mail on the too long deletion times (in > PostgreSQL 13.2)... > > I have followed the advices here, thank you - > > On Fri, Nov 27, 2020 at 4:15 PM Guillaume Lelarge > wrote: > >>

Re: Deleting takes days, should I add some index?

2021-02-25 Thread Alexander Farber
Hi Pavel, trying to follow your advice "You should check so all foreign keys have an index" I look at the table where I want to delete older records: # \d words_games Table "public.words_games" Column | Type | Collation | Nullable |

Extension intarray and null values

2021-02-25 Thread Eric Brison
Hello, I use tables with several "int[]" columns. I use the "@>" operator to filter my data. To increase speed , i install the "intarray" extension. Now queries are very fast with the specific indexes (i use "gin__int_ops"). But, in few columns, i have null values in my intarray and i cannot use

Postgres Analog of Oracle APPEND hint

2021-02-25 Thread Rumpi Gravenstein
All, Using PostgreSQL 13.1 I am new to PostgreSQL transitioning from Oracle. One of the many Oracle tricks I learned is that large inserts can be sped up by adding the direct path load hint /*+APPEND*/ . I am faced with having to perform many large inserts (100K->100M rows) in my PostgreSQL dat

PostgreSQL URI

2021-02-25 Thread Paul Förster
Hi, in https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING it says that the syntax for a PostgreSQL URI is: postgresql://[user[:password]@][host][:port][,...][/dbname][?param1=value1&...] What I don't understand is the [,...] part, i.e. optionally repeating argument. I

Re: converting text to bytea

2021-02-25 Thread Tom Lane
Yambu writes: > Is there a reason why i'm getting text when i run the below > select convert_to('some_text', 'UTF8')i get back 'some_text' You must have bytea_output set to "escape". regards, tom lane

Re: Deleting takes days, should I add some index?

2021-02-25 Thread Pavel Stehule
čt 25. 2. 2021 v 14:36 odesílatel Alexander Farber < alexander.far...@gmail.com> napsal: > Hi Pavel, > > trying to follow your advice "You should check so all foreign keys have an > index" I look at the table where I want to delete older records: > > # \d words_games >

Server hangs on pg_repack

2021-02-25 Thread Roman Liverovskiy
Hello. I have postgresql 12 with a 3.0 GB database with a table containing 10 millions of rows, this table also has 4 indexes. I have an AWS EC2 server with two AMD EPYC 7571 cores, 2 GB of RAM and SSD disk. Because of index bloating I use pg_repack. When I use default postgresql.conf file I have n

Re: converting text to bytea

2021-02-25 Thread Pavel Stehule
čt 25. 2. 2021 v 16:01 odesílatel Tom Lane napsal: > Yambu writes: > > Is there a reason why i'm getting text when i run the below > > select convert_to('some_text', 'UTF8')i get back 'some_text' > > You must have bytea_output set to "escape". > yes set bytea_output TO escape ; postgres=#

Re: Extension intarray and null values

2021-02-25 Thread Tom Lane
Eric Brison writes: > I use tables with several "int[]" columns. > I use the "@>" operator to filter my data. > To increase speed , i install the "intarray" extension. Now queries are > very fast with the specific indexes (i use "gin__int_ops"). > But, in few columns, i have null values in my inta

Re: PostgreSQL URI

2021-02-25 Thread Tom Lane
=?utf-8?Q?Paul_F=C3=B6rster?= writes: > in > https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING > it says that the syntax for a PostgreSQL URI is: > postgresql://[user[:password]@][host][:port][,...][/dbname][?param1=value1&...] > What I don't understand is the [,...]

Re: PostgreSQL URI

2021-02-25 Thread Paul Förster
Hi Tom, > On 25. Feb, 2021, at 16:09, Tom Lane wrote: > > =?utf-8?Q?Paul_F=C3=B6rster?= writes: >> in >> https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING >> it says that the syntax for a PostgreSQL URI is: > >> postgresql://[user[:password]@][host][:port][,...][/db

Re: PostgreSQL URI

2021-02-25 Thread Tom Lane
=?utf-8?Q?Paul_F=C3=B6rster?= writes: > I suspected this already. Still the position of the closing angle bracket > behind the "host" part in the syntax is IMHO wrong in the doc. Hmm. Maybe postgresql://[user[:password]@][host[:port][,...]][/dbname][?param1=value1&...] ? Seems like that woul

Re: PostgreSQL URI

2021-02-25 Thread Paul Förster
Hi Tom, > On 25. Feb, 2021, at 16:22, Tom Lane wrote: > > Hmm. Maybe > > postgresql://[user[:password]@][host[:port][,...]][/dbname][?param1=value1&...] > > ? Seems like that would clarify how much you can repeat. yes, that looks better, thanks. Cheers, Paul

Re: Postgres Analog of Oracle APPEND hint

2021-02-25 Thread Jayadevan M
> > Using PostgreSQL 13.1 > > I am new to PostgreSQL transitioning from Oracle. One of the many Oracle > tricks I learned is that large inserts can be sped up by adding the direct > path load hint /*+APPEND*/ . I am faced with having to perform many large > inserts (100K->100M rows) in my Postgre

Re: Postgres Analog of Oracle APPEND hint

2021-02-25 Thread Mark Johnson
Since INSERT /*+APPEND*/ is generally used when bulk loading data into Oracle from external files you should probably look at the PostgreSQL COPY command (https://www.postgresql.org/docs/13/sql-copy.html) and additional utilities like pg_bulkload (https://github.com/ossc-db/pg_bulkload) . On Thu

Re: Server hangs on pg_repack

2021-02-25 Thread Michael Lewis
Why not use reindex concurrently?

Re: PostgreSQL URI

2021-02-25 Thread Tom Lane
=?utf-8?Q?Paul_F=C3=B6rster?= writes: >> On 25. Feb, 2021, at 16:22, Tom Lane wrote: >> Hmm. Maybe >> postgresql://[user[:password]@][host[:port][,...]][/dbname][?param1=value1&...] >> ? Seems like that would clarify how much you can repeat. > yes, that looks better, thanks. Experimenting, it

Re: Extension intarray and null values

2021-02-25 Thread Eric Brison
yes i a have a specific query with many rows and big int array These data not contains null values. And in this case , i use the specific GIN index. The query time decrease from 30s to 100ms with the index. I found that i can use the "arraycontains" builtin function for int[] with null values. Bu

Re: PostgreSQL URI

2021-02-25 Thread Paul Förster
Hi Tom, > On 25. Feb, 2021, at 16:43, Tom Lane wrote: > > Experimenting, it does let you omit the host and specify a port: > > $ psql -d postgresql://:5433 > psql: error: could not connect to server: No such file or directory >Is the server running locally and accepting >connect

Re: Server hangs on pg_repack

2021-02-25 Thread Roman Liverovskiy
Thanks, I will try it, but it is not an answer on my question. On Thu, Feb 25, 2021, 19:38 Michael Lewis wrote: > Why not use reindex concurrently? >

Re: Postgres Analog of Oracle APPEND hint

2021-02-25 Thread Rumpi Gravenstein
Unfortunately, I am not looking to load from an external source. My process is moving data from source PostgreSQL tables to target PostgreSQL tables. On Thu, Feb 25, 2021 at 10:36 AM Mark Johnson wrote: > Since INSERT /*+APPEND*/ is generally used when bulk loading data into > Oracle from exter

Re: Extension intarray and null values

2021-02-25 Thread Tom Lane
Eric Brison writes: > yes i a have a specific query with many rows and big int array These data > not contains null values. And in this case , i use the specific GIN index. > The query time decrease from 30s to 100ms with the index. My point is that you can also make a gin index on an integer ar

Re: Postgres Analog of Oracle APPEND hint

2021-02-25 Thread Tom Lane
Rumpi Gravenstein writes: > Unfortunately, I am not looking to load from an external source. My > process is moving data from source PostgreSQL tables to target PostgreSQL > tables. The hints in https://www.postgresql.org/docs/current/populate.html would still largely apply, though of course n

Re: Postgres Analog of Oracle APPEND hint

2021-02-25 Thread Rob Sargent
On 2/25/21 9:26 AM, Rumpi Gravenstein wrote: Unfortunately, I am not looking to load from an external source.  My process is moving data from source PostgreSQL tables to target PostgreSQL tables. Are you trying to duplicate the source tables in the target tables? If so, there are replication

serializability and unique constraint violations

2021-02-25 Thread Jonathan Amsterdam
I found that I can get a "duplicate key value violates unique constraint" error under certain conditions which I don't think the documentation describes. I don't know if this is a documentation bug, a product bug, or if I'm holding something wrong. My table is created with CREATE TABLE paths (

Re: Deleting takes days, should I add some index?

2021-02-25 Thread Alexander Farber
Pavel, thank you for asking! I have put the anonymized dump of my database at: http://wordsbyfarber.com/words_dev.sql.gz (beware, it is a 1.3 GB download) The question is why does the command take days (when I tried last time): delete from words_games where created < now() - interval '12 month'

Code of Conduct: Hebrew Translation for Review

2021-02-25 Thread Stacey Haysler
The PostgreSQL Community Code of Conduct Committee has received a draft of the Hebrew translation of the Code of Conduct Policy updated August 18, 2020 for review.The English version of the Policy is at:https://www.postgresql.org/about/policies/coc/The patch was created by:Michael GoldbergThe patch

Re: getting tables list of other schema too

2021-02-25 Thread Francisco Olarte
Atul. Due to your top posting style and not being a native english speaker I'm unable to understand your question. As all the quotes at the bottom seemed to belong to me, I'm assuming you referred to some of my postings. As the last one said I tried to point a thing after checking some docs, th

Re: Postgres Analog of Oracle APPEND hint

2021-02-25 Thread Rumpi Gravenstein
My use case involves complicated joins on source tables in one schema loading a target table in the same or a different schema. On Thu, Feb 25, 2021 at 11:41 AM Rob Sargent wrote: > On 2/25/21 9:26 AM, Rumpi Gravenstein wrote: > > Unfortunately, I am not looking to load from an external source.

Re: Deleting takes days, should I add some index?

2021-02-25 Thread David Rowley
On Fri, 26 Feb 2021 at 02:06, Alexander Farber wrote: > However the deletion still takes forever and I have to ctrl-c it: > > # delete from words_games where created < now() - interval '12 month'; > > Do you please have any further suggestions? > > When I try to prepend "explain analyze" to the ab

Re: Code of Conduct: Hebrew Translation for Review

2021-02-25 Thread Valeria Kaplan
Had a read through, no comments. Thank you, Michael and Emil! On Thu, Feb 25, 2021 at 6:54 PM Stacey Haysler wrote: > The PostgreSQL Community Code of Conduct Committee has received a draft of > the Hebrew translation of the Code of Conduct Policy updated August 18, > 2020 for review. > > The En

Re: Deleting takes days, should I add some index?

2021-02-25 Thread Pavel Stehule
Hi čt 25. 2. 2021 v 19:39 odesílatel Alexander Farber < alexander.far...@gmail.com> napsal: > Pavel, thank you for asking! > > I have put the anonymized dump of my database at: > > http://wordsbyfarber.com/words_dev.sql.gz (beware, it is a 1.3 GB > download) > > The question is why does the comma

Re: Deleting takes days, should I add some index?

2021-02-25 Thread Pavel Stehule
čt 25. 2. 2021 v 22:02 odesílatel Pavel Stehule napsal: > Hi > > čt 25. 2. 2021 v 19:39 odesílatel Alexander Farber < > alexander.far...@gmail.com> napsal: > >> Pavel, thank you for asking! >> >> I have put the anonymized dump of my database at: >> >> http://wordsbyfarber.com/words_dev.sql.gz (be

Re: Deleting takes days, should I add some index?

2021-02-25 Thread Alexander Farber
Thank you, Pavel! I didn't even think about trying to "explain analyze" deletion of just 1 record - On Thu, Feb 25, 2021 at 10:04 PM Pavel Stehule wrote: > čt 25. 2. 2021 v 19:39 odesílatel Alexander Farber < >> alexander.far...@gmail.com> napsal: >> >>> The question is why does the command tak

Re: Deleting takes days, should I add some index?

2021-02-25 Thread Pavel Stehule
čt 25. 2. 2021 v 22:33 odesílatel Alexander Farber < alexander.far...@gmail.com> napsal: > Thank you, Pavel! > > I didn't even think about trying to "explain analyze" deletion of just 1 > record - > > On Thu, Feb 25, 2021 at 10:04 PM Pavel Stehule > wrote: > >> čt 25. 2. 2021 v 19:39 odesílatel A

Re: Deleting takes days, should I add some index?

2021-02-25 Thread Alexander Farber
Thank you for the explanation, David On Thu, Feb 25, 2021 at 9:49 PM David Rowley wrote: > > Since your foreign keys perform a cascade delete on the tables > referencing the tables you're deleting from, any records in those > referencing tables will be deleted too. You must also look at those >

PostgreSQL Licensing Question for pg_crypto and tablefunc extensions

2021-02-25 Thread Rumpi Gravenstein
All, Postgres 13.1 I am new to PostgreSQL and am unclear on how licensing works for PostgreSQL extensions. Are pg_crypto and tablefunc licensed with the PostgreSQL community edition or do PostgreSQL extensions fall under a separate license? I've looked for documentation on this and haven't foun

Re: PostgreSQL Licensing Question for pg_crypto and tablefunc extensions

2021-02-25 Thread Tom Lane
Rumpi Gravenstein writes: > I am new to PostgreSQL and am unclear on how licensing works for PostgreSQL > extensions. Are pg_crypto and tablefunc licensed with the PostgreSQL > community edition or do PostgreSQL extensions fall under a separate > license? I've looked for documentation on this an