Re: pg_dump: [archiver (db)] query failed: FATAL: semop(id=10649641) failed: Identifier removed

2018-10-08 Thread Tom Lane
Raghavendra Rao J S V writes: > Receiving below error while taking the backup using pg_dump. Please help me > why and how to resolve this. > pg_dump: [archiver (db)] query failed: FATAL: semop(id=10649641) failed: > Identifier removed Are you running on a platform that uses systemd? If so, see

pg_dump: [archiver (db)] query failed: FATAL: semop(id=10649641) failed: Identifier removed

2018-10-08 Thread Raghavendra Rao J S V
Receiving below error while taking the backup using pg_dump. Please help me why and how to resolve this. pg_dump: [archiver (db)] query failed: ERROR: could not open relation with OID 14132724 pg_dump: [archiver (db)] query was: SELECT pg_catalog.pg_get_viewdef('14132724'::pg_catalog.oid) AS vie

Re: pg9.6: no backup history file (*.backup) created on hot standby

2018-10-08 Thread Michael Paquier
On Tue, Oct 09, 2018 at 11:45:50AM +0800, magodo wrote: > Yet, I am still not so clear how does the bloat of pg_wal happen? Do > you mean pg_wal will be filled up by many .backup(s)? If your archive_command is for example a simple cp (which it should not be by the way), and if you try to archive t

Re: pg9.6: no backup history file (*.backup) created on hot standby

2018-10-08 Thread Michael Paquier
(Please do not forget to add the community mailing list in CC.) On Tue, Oct 09, 2018 at 10:33:56AM +0800, magodo wrote: > Since the backup history aims to aid administrator to identify the > point from which wal archive should be kept and before which the > archive could be cleaned. It is very hel

Re: FTS trigger works 1 at a time, but fails with bulk insert script

2018-10-08 Thread Adrian Klaver
On 10/8/18 3:54 PM, Adrian Klaver wrote: On 10/8/18 1:58 PM, Malik Rumi wrote: So what is the script you used to do the bulk INSERT? There's actually three, but they are all basically the same. The differences have to do with the source material being inserted: # usr/local/bin/python3.6 # co

Re: FTS trigger works 1 at a time, but fails with bulk insert script

2018-10-08 Thread Adrian Klaver
On 10/8/18 1:58 PM, Malik Rumi wrote: So what is the script you used to do the bulk INSERT? There's actually three, but they are all basically the same. The differences have to do with the source material being inserted: # usr/local/bin/python3.6 # coding: utf-8 from os import environ enviro

Re: Why the index is not used ?

2018-10-08 Thread Paul McGarry
Hi Didier, Yes, credit cards are a very specific space that probably gets people who are familiar with it going a bit. By the time you factor in general security practices, specific PCI requirements, your threat model and likely business requirements (needing relatively free access to parts of

Re: FTS trigger works 1 at a time, but fails with bulk insert script

2018-10-08 Thread Malik Rumi
So what is the script you used to do the bulk INSERT? There's actually three, but they are all basically the same. The differences have to do with the source material being inserted: # usr/local/bin/python3.6 # coding: utf-8 from os import environ environ['DJANGO_SETTINGS_MODULE'] = 'chronicle.s

Re: FTS trigger works 1 at a time, but fails with bulk insert script

2018-10-08 Thread Adrian Klaver
On 10/8/18 1:25 PM, Malik Rumi wrote: I hope this comes out readable. If not I can do a separate attachment. I notice it says 'BEFORE INSERT'. Maybe that should be after? No as the return value would be ignored: https://www.postgresql.org/docs/10/static/plpgsql-trigger.html "The return value

Re: FTS trigger works 1 at a time, but fails with bulk insert script

2018-10-08 Thread Malik Rumi
I hope this comes out readable. If not I can do a separate attachment. I notice it says 'BEFORE INSERT'. Maybe that should be after? Table "public.ktab_entry" Column | Type | Collation | Nullable | Default ---+--

Re: Why the index is not used ?

2018-10-08 Thread Tomas Vondra
Hi, On 10/08/2018 04:10 PM, ROS Didier wrote: > Hi Tomas >     >     Thank you for your answer and recommendation which is very > interesting. I'm going to study the PCI DSS document right now. > > * Here are my answer to your question : > > />>/ > /What is your threat model?/ > /< we

Re: FTS trigger works 1 at a time, but fails with bulk insert script

2018-10-08 Thread Adrian Klaver
On 10/8/18 12:29 PM, Malik Rumi wrote: 1. This code is entry_search_vector_trigger(), one of 3 trigger functions based on the Django model that created the site. 2. So this is the trigger definition (as far as I know) and it is on the Entry table. There is also a Tag table and the Tags intersect

Re: FTS trigger works 1 at a time, but fails with bulk insert script

2018-10-08 Thread Malik Rumi
1. This code is entry_search_vector_trigger(), one of 3 trigger functions based on the Django model that created the site. 2. So this is the trigger definition (as far as I know) and it is on the Entry table. There is also a Tag table and the Tags intersection table. 3. Uhh, I'm not sure. I assume

RE: Why the index is not used ?

2018-10-08 Thread ROS Didier
Dear Jean-Paul Thank you very much for this link which is actually very interesting. I am going to study it carefully. But my problem is more generic: How to set up the encryption of sensitive data and have good performance (using an index by example) ?. Apparen

RE: Why the index is not used ?

2018-10-08 Thread ROS Didier
Hi Paul Thank you very much for your feedback which is very informative. I understand that concerning the encryption of credit card numbers, it is imperative to respect the PCI DSS document. I am going to study it. However, I would like to say that I

RE: Why the index is not used ?

2018-10-08 Thread ROS Didier
Hi Tomas Thank you for your answer and recommendation which is very interesting. I'm going to study the PCI DSS document right now. - Here are my answer to your question : >> What is your threat model? << we want to prevent access to sensitive data for everyone except those who have

RE: Why the index is not used ?

2018-10-08 Thread Phil Endecott
ROS Didier wrote: Can you give some examples of these encryption function that doesn't salt the data. encrypt(data, 'motdepass', 'aes') Regards, Phil.

RE: Why the index is not used ?

2018-10-08 Thread ROS Didier
Hi Vlad OK, I take into account your remark about the need to do research on encrypted data. My answers to your remarks : >> you can use a deterministic algorithm for it (without salt) << Can you give me on of these deterministic algorithms(without salt) ? Best Regards Didier De :

RE: Why the index is not used ?

2018-10-08 Thread ROS Didier
Hi Phil Thank you for this recommendation, but I posted on this public list only generic examples that have nothing to do with the works done in my company. These examples serve me only to discuss about the subject of data encryption and performance My answers to your rem

RE: Why the index is not used ?

2018-10-08 Thread ROS Didier
Hi Vlad Sorry for this delay, but apparently the subject is of interest to many people in the community. I received a lot of comments and answers. I wrote my answers in the body of your message below Best Regards Didier De : greatvo...@gmail.com [mailto:greatvo...@gmail.com] Envoyé : samedi 6 oc

Re: pg9.6: no backup history file (*.backup) created on hot standby

2018-10-08 Thread Michael Paquier
On Mon, Oct 08, 2018 at 03:23:47PM +0800, magodo wrote: > Is this as expected? Yes. > If so, in which case should I do backup on primary and in which case > should I do it on standby? This depends on your use cases, sometimes you don't want to make the production server, the primary use more CPU

pg9.6: no backup history file (*.backup) created on hot standby

2018-10-08 Thread magodo
Hello, I am using PostgreSQL-9.6, trying to do basebackup on a hot standby with archive mode set to always. However, I found there is no backup history file (*.backup) generated in the archive destination directory after `pg_basebackup`, which is not the case if I do the same thing on the primar

Re: Why the index is not used ?

2018-10-08 Thread Jean-Paul Argudo
Dear Didier, Le lundi 08 octobre 2018 à 08:32 +, ROS Didier a écrit : > Hi Virendra > You think that outside encryption of the database is the best solution > ? >How do you manage the encryption key ? > Can you give me some examples of this kind of solution. >

RE: Why the index is not used ?

2018-10-08 Thread ROS Didier
Hi Virendra You think that outside encryption of the database is the best solution ? How do you manage the encryption key ? Can you give me some examples of this kind of solution. Best Regards Didier ROS -Message d'origine- De : virendra.ku...@guycarp.c

Re: pg_controldata: could not read file "/opt/postgres/9.2/data//global/pg_control": Success

2018-10-08 Thread Rajni Baliyan
Hi Raghvendra, Free up some space before restarting. Old archive/backup/logs could be the candidate for clean up. You do not need much space to restart instance. Once started, login to db and execute checkpoint. Thanks Rajni On Mon, 8 Oct 2018 at 4:40 pm Raghavendra Rao J S V < raghavendra...@gm