Building lots of indices in parallel

2021-02-11 Thread Ron
We're migrating a big database (200ish tables 30 of which have bytea or xml fields; the Oracle size is 10TB) to Postgresql 12.5 (RDS, if it matters), and after the loads are completed, it will be time to create the secondary indices (tables currently only have PK constraints), and they of cou

Re: Encryption of Data Specific to a Tenant in PostgreSQL database | General Idea

2021-02-11 Thread Martin Ritchie
I have had good luck with security barrier views and performance. The main thing security barrier does is ensure that where statements are processed correctly. The big consideration IMHO is how many tenants are you dealing with. A couple of tenants, then best to give them separate databases. More

Re: Encryption of Data Specific to a Tenant in PostgreSQL database | General Idea

2021-02-11 Thread Rob Sargent
DBA=~super user If your clients have dba privs you need separate (vertical)servers

Re: Encryption of Data Specific to a Tenant in PostgreSQL database | General Idea

2021-02-11 Thread Jagmohan Kaintura
Hi Stephen, Note that views aren't actually guaranteed to provide the isolation you're looking for unless you mark them as being a security barrier, see: https://www.postgresql.org/docs/current/rules-privileges.html By using Security Barrier we had a huge impact on performance , it was not consi

Re: Consequence of changes to CTE's in 12

2021-02-11 Thread Steve Baldwin
Thanks Tom. This optimization fences concept is a new one to me, so great to know about. This does indeed give me a nice version-independent solution, and make me a very happy camper ;-) Steve On Fri, Feb 12, 2021 at 11:45 AM Tom Lane wrote: > Steve Baldwin writes: > > Is there a chance that

Re: Consequence of changes to CTE's in 12

2021-02-11 Thread Tom Lane
Steve Baldwin writes: > Is there a chance that the query optimiser should 'notice' the > pg_try_advisory_xact_lock function, and not be so clever when it sees it? The general policy with respect to volatile functions in WHERE quals is "here be dragons". You don't have enough control over when a

Re: Consequence of changes to CTE's in 12

2021-02-11 Thread Steve Baldwin
Thanks all. The fact that this is a view is not really relevant. I only bundled as a view here to make testing simpler. The underlying query still behaves differently pre-12 and 12+. Is there a chance that the query optimiser should 'notice' the pg_try_advisory_xact_lock function, and not be so cl

Re: Consequence of changes to CTE's in 12

2021-02-11 Thread Tom Lane
Michael Lewis writes: > If you move the limit 1 to the first CTE, does it not give you the same > behavior in both versions? Not sure if that's exactly the same, but certainly adding a traditional optimization fence (OFFSET 0) to the first CTE should do the trick. regards

Re: Consequence of changes to CTE's in 12

2021-02-11 Thread Michael Lewis
This functionality seems more a candidate for a set-returning function rather than a view, but I like my views to be side effect free and read only. It would be trivial to implement in plpgsql I believe. If you move the limit 1 to the first CTE, does it not give you the same behavior in both versi

Re: Consequence of changes to CTE's in 12

2021-02-11 Thread David G. Johnston
On Thu, Feb 11, 2021 at 5:07 PM Steve Baldwin wrote: > My 'dilemma' is that this functionality is packaged and the database it is > bundled into could be running on a pre-12 version or 12+. Is there any way > I can rewrite my view to achieve the same outcome (i.e. only creating 0 or > 1 advisory

Consequence of changes to CTE's in 12

2021-02-11 Thread Steve Baldwin
Hi, I realise this is probably an edge case, but would appreciate some advice or suggestions. I have a table that has rows to be processed: postgres=# create table lock_test (id uuid primary key default gen_random_uuid(), lock_id bigint); CREATE TABLE postgres=# insert into lock_test (lock_id) v

Re: Compiler warnings on Debian 10

2021-02-11 Thread Tom Lane
Looking at things more closely, I realize that the warnings are not showing up in most compiles (if they were, you'd have many more of them). They are only showing up in the llvmjit_*.cpp files, which need to be built with a C++ compiler not gcc. I can see in your make log that those are getting

Re: Compiler warnings on Debian 10

2021-02-11 Thread Daniel Westermann (DWE)
>> Nothing special. Configure and make log attached. The same procedure works >> fine on my local Debian 10.7. >Quite odd.  The configure output clearly shows that it thought gnu_printf >is fine: >checking for printf format archetype... gnu_printf >Possibly the corresponding section of config.l

Re: Unable to execute Query in parallel for partitioned table

2021-02-11 Thread Brajendra Pratap
Hi Albe, We have checked as per your suggestion and we are good now. Thank you !!! On Thu, 11 Feb, 2021, 8:49 PM Brajendra Pratap, < brajendra.pratap...@gmail.com> wrote: > Hi Albe, > > Thank you so much for information, will check this and get back to you if > any help required. > > I have a

Re: Compiler warnings on Debian 10

2021-02-11 Thread Tom Lane
"Daniel Westermann (DWE)" writes: > "Daniel Westermann (DWE)" writes: >>> I was wondering if someone already has seen these warnings on Debian 10 >>> (PostgreSQL 13.1): >>> ../../../../src/include/port.h:176:70: warning: 'format' attribute argument >>> not supported: gnu_printf >>>    [-Wi

Re: Problem with trigger function

2021-02-11 Thread David G. Johnston
On Thursday, February 11, 2021, Steve Baldwin wrote: > David, from what I can see of the docs, for 9.6 it is PROCEDURE. It seems > FUNCTION didn't appear until 11. > Indeed. I didn’t pay attention to the version. David J.

Re: Problem with trigger function

2021-02-11 Thread Steve Baldwin
David, from what I can see of the docs, for 9.6 it is PROCEDURE. It seems FUNCTION didn't appear until 11. Steve On Fri, Feb 12, 2021 at 7:05 AM David G. Johnston < david.g.johns...@gmail.com> wrote: > > On Thursday, February 11, 2021, Steve Baldwin > wrote: > >> Try ... EXECUTE PROCEDURE cust

Re: Problem with trigger function

2021-02-11 Thread David G. Johnston
On Thursday, February 11, 2021, Steve Baldwin wrote: > Try ... EXECUTE PROCEDURE customer_num_informix() > FUNCTION, not PROCEDURE David J.

Re: Problem with trigger function

2021-02-11 Thread Dave Cramer
FWIW, messing with serial numbers like this is pretty risky.Sequences have transactional semantics for a reason. Dave Cramer www.postgres.rocks On Thu, 11 Feb 2021 at 14:57, Steve Baldwin wrote: > Try ... EXECUTE PROCEDURE customer_num_informix() > > Steve > > On Fri, Feb 12, 2021 at 6:47 AM J

Re: Encryption of Data Specific to a Tenant in PostgreSQL database | General Idea

2021-02-11 Thread Stephen Frost
Greetings, * Jagmohan Kaintura (jagmo...@tecorelabs.com) wrote: > Yup right now data is being accessed in this manner only. > application access using tenant user only who have specific tenantId in > that session and can see its own data only. It doesn't know about anyone > else's data and neither

Re: Problem with trigger function

2021-02-11 Thread Steve Baldwin
Try ... EXECUTE PROCEDURE customer_num_informix() Steve On Fri, Feb 12, 2021 at 6:47 AM James B. Byrne wrote: > I am trying to implement a trigger in a PostgreSQL-9.6.17 database: > > CREATE OR REPLACE FUNCTION customer_num_informix() > RETURNS trigger AS $$ > BEGIN > -- An Aubit/Informix 4

Problem with trigger function

2021-02-11 Thread James B. Byrne
I am trying to implement a trigger in a PostgreSQL-9.6.17 database: CREATE OR REPLACE FUNCTION customer_num_informix() RETURNS trigger AS $$ BEGIN -- An Aubit/Informix 4GL INSERT statement passes the value 0 to Informix -- DBMS for columns that have the SERIAL data type. Informix will then

Re: Compiler warnings on Debian 10

2021-02-11 Thread Tom Lane
"Daniel Westermann (DWE)" writes: > I was wondering if someone already has seen these warnings on Debian 10 > (PostgreSQL 13.1): > ../../../../src/include/port.h:176:70: warning: 'format' attribute argument > not supported: gnu_printf > [-Wignored-attributes] Huh. What compiler are you

Re: PostgreSQL occasionally unable to rename WAL files (NTFS)

2021-02-11 Thread Thorsten Schöning
Guten Tag Guy Burgess, am Donnerstag, 11. Februar 2021 um 01:21 schrieben Sie: > What appears to be happening is the affected WAL files (which is > usually only 2 or 3 WAL files at a time) are somehow "losing" their > NTFS permissions, so the PG process can't rename them - though of > course the P

Re: Which partition scheme makes sense for my time based IoT-datagrams?

2021-02-11 Thread Thorsten Schöning
Guten Tag Peter J. Holzer, am Donnerstag, 11. Februar 2021 um 15:03 schrieben Sie: > If you get one datagram per day, how it is useful to query all datagrams > received in the last 15 minutes?[...] There's a special installation mode during which users are setting up their IoT-devices. During tha

Re: Unable to execute Query in parallel for partitioned table

2021-02-11 Thread Brajendra Pratap
Hi Albe, Thank you so much for information, will check this and get back to you if any help required. I have a doubt why didn't the parallelism works here ,could u plz guide me? Thank you so much again. On Thu, 11 Feb, 2021, 1:23 PM Laurenz Albe, wrote: > On Thu, 2021-02-11 at 05:09 +0530, Br

Re: Which partition scheme makes sense for my time based IoT-datagrams?

2021-02-11 Thread Peter J. Holzer
On 2021-02-10 16:09:30 +0100, Thorsten Schöning wrote: > I have a table storing datagrams from some IoT-devices, with one > datagram per device per day most likely for around 75'000 devices > currently. I want to test query performance with a partitioned table > and am interested in the following q

Re: SV: Insertion time is very high for inserting data in postgres

2021-02-11 Thread Peter J. Holzer
On 2021-02-10 11:10:41 +, Niels Jespersen wrote: > >Fra: prachi surangalikar > >We are using Postgres 12.2.1 for fetching per minute data for about 25 > machines but running parallely via a single thread in python. > >But suddenly the insertion time has increased to a very high level, about 30

Re: Assistance on PostgreSQL DB

2021-02-11 Thread Laurenz Albe
On Thu, 2021-02-11 at 08:57 +, arunkumar.sampathku...@cognizant.com wrote: > We would like to know the best practice to maintain PostgreSQL DB health so > that there is no delay in Jabber chat messages. You should hire a knowledgeable DBA. Yours, Laurenz Albe -- Cybertec | https://www.cyber

Re: PostgreSQL occasionally unable to rename WAL files (NTFS)

2021-02-11 Thread Lionel Bouton
Hi, Le 11/02/2021 à 01:21, Guy Burgess a écrit : > > Hello, > > Running 13.1 on Windows Server 2019, I am getting the following log > entries occasionally: > >     2021-02-11 12:34:10.149 NZDT [6072] LOG:  could not rename file > "pg_wal/0001009900D3": Permission denied >     2021-02-1

Assistance on PostgreSQL DB

2021-02-11 Thread ArunKumar.SampathKumar
Hi Team, We would like to know the best practice to maintain PostgreSQL DB health so that there is no delay in Jabber chat messages. Regards Arun This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged informat