Re: vacuum is time consuming

2021-02-02 Thread Atul Kumar
Ok Martin, I got ur ur point of max limit of maintenance_work_mem is 1 GB but there is nothing mentioned about the same in postgresql.conf as remarks for this specific parameter. Is there any other option to increase the speed of vacuum? Regards Atul On Tuesday, February 2, 2021, Martín Mar

Re: vacuum is time consuming

2021-02-02 Thread Laurenz Albe
On Tue, 2021-02-02 at 13:44 +0530, Atul Kumar wrote: > Is there any other option to increase the speed of vacuum? For autovacuum, decrease "autovacuum_vacuum_cost_delay". Yours, Laurenz Albe -- Cybertec | https://www.cybertec-postgresql.com

Re: count(*) vs count(id)

2021-02-02 Thread Laurenz Albe
On Mon, 2021-02-01 at 19:14 -0600, Matt Zagrabelny wrote: > > > What is count(*) counting then? I thought it was rows. > > > > Yeah, but count(id) only counts rows where id isn't null. > > I guess I'm still not understanding it... > > I don't have any rows where id is null: Then the *result* of

Re: permission denied for large object 200936761

2021-02-02 Thread Laurenz Albe
On Tue, 2021-02-02 at 00:20 +0200, Andrus wrote: > > Obviously large objects *are* used. > > How to figure out what is this large object ? You can extract it with \lo_export 200936761 'somefile' in psql and examine the file. Ask the people who use that database! Yours, Laurenz Albe -- Cyber

Re: permission denied for large object 200936761

2021-02-02 Thread Andrus
Hi! You can extract it with \lo_export 200936761 'somefile' in psql and examine the file. Ask the people who use that database! Tried root@c202-76:~# ./pgsqlkaiv.sh psql (12.2 (Debian 12.2-2.pgdg100+1)) Type "help" for help. sba=#  \lo_export 200936761 'large200936761' lo_export sba=# \

Re: permission denied for large object 200936761

2021-02-02 Thread Andrus
Hi! >I have imported data from other clusters and executed lot of different sql commands.  I have used grant, revoke, reassign commands to change privileges for other users and have deleted and added users. I don't suppose this was done in a structured way that could be gone back over? E

Re: permission denied for large object 200936761

2021-02-02 Thread Andrus
Hi! I don't suppose this was done in a structured way that could be gone back over? Accidently  '200936767'::lo cast was issued : INSERT INTO report ( ...  ) values (.. , '200936767'::lo, ... ) server throws error   type "lo" does not exist for this. Maybe this causes orphan large object

Re: permission denied for large object 200936761

2021-02-02 Thread Laurenz Albe
On Tue, 2021-02-02 at 10:48 +0200, Andrus wrote: > > You can extract it with > > \lo_export 200936761 'somefile' > > > > in psql and examine the file. > > > > Ask the people who use that database! > > Tried > > root@c202-76:~# ./pgsqlkaiv.sh > > psql (12.2 (Debian 12.2-2.pgdg100+1)) > Type "

Re: libpq and mysterious "invalid byte sequence for encoding UTF8".

2021-02-02 Thread Jiří Pavlovský
On 02.02.2021 3:53, Tom Lane wrote: > rob stone writes: >> Columns:- maincontact boolean DEFAULT false, >> publdatacontact boolean DEFAULT false, >> invcontact boolean DEFAULT false, >> queries_recipient boolean, >> fakelastname boolean NOT NULL DEFAULT false, >> ar

Index created with PGSQL11 on ubuntu 18.04 corrupt with PGSQL11 on ubuntu 20.04

2021-02-02 Thread unilynx
I'm using postgresql 11 builds from http://apt.postgresql.org/pub/repos/apt/ - I've got a database created under Ubuntu 18.04, and recently updated to Ubuntu 20.04. These are all docker builds I've got an index defined as follows CREATE UNIQUE INDEX entity_settings_wh_unique_rawdata ON wrd.entity

Re: Index created with PGSQL11 on ubuntu 18.04 corrupt with PGSQL11 on ubuntu 20.04

2021-02-02 Thread Magnus Hagander
On Tue, Feb 2, 2021 at 11:20 AM unilynx wrote: > > I'm using postgresql 11 builds from http://apt.postgresql.org/pub/repos/apt/ > - I've got a database created under Ubuntu 18.04, and recently updated to > Ubuntu 20.04. These are all docker builds This is a known problem when upgrading Ubuntu (an

Re: Index created with PGSQL11 on ubuntu 18.04 corrupt with PGSQL11 on ubuntu 20.04

2021-02-02 Thread unilynx
Thanks, that was quick. Didn't realise the distribution's locale tables would have been a source of differences too -- Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

RE: Foreign table performance issue / PostgreSQK vs. ORACLE

2021-02-02 Thread Markhof, Ingolf
Hi! My PostgreSQL version is 11.8. The query I am running is referring to a number of foreign tables. The first one (table1) has to IDs, let's say ID_A and ID_B. While ID_A is unique, ID_B is not. In my case, I am pulling formation for a value of IB_B for which about 800 rows (with unique ID_

Re: permission denied for large object 200936761

2021-02-02 Thread Andrus
Hi! So? What is your point? Somebody created a large object of size 0. report table has bytea column. It looks like  psqlodbc driver adds ::lo cast  when inserting binary data: https://github.com/hlinnaka/psqlodbc/blob/master/convert.c#L4564 and this adds row to pg_largeobject_metadata t

Re: vacuum is time consuming

2021-02-02 Thread Martín Marqués
Hi, > > Is there any other option to increase the speed of vacuum? > > For autovacuum, decrease "autovacuum_vacuum_cost_delay". He mentioned in the original message that his problems was with a global VACUUM after upgrading, so cost_delay should be zero by default. Regards, -- Martín Marqués I

Re: ransomware

2021-02-02 Thread Marc Millas
Hi, I know its quite general. It is as I dont know what approaches may exist. Requirement is extremely simple: Is there anyway, from a running postgres standpoint, to be aware that a ransomware is currently crypting your data ? answer can be as simple as: when postgres do crash. something e

Re: ransomware

2021-02-02 Thread Peter J. Holzer
On 2021-02-02 15:44:31 +0100, Marc Millas wrote: > I know its quite general. It is as I dont know what approaches may exist. > > Requirement is extremely simple: Is there anyway, from a running postgres > standpoint, to be aware that a ransomware is currently crypting your data ? PostgreSQL can b

Re: libpq and mysterious "invalid byte sequence for encoding UTF8".

2021-02-02 Thread Tom Lane
=?UTF-8?B?SmnFmcOtIFBhdmxvdnNrw70=?= writes: > Client and server encoding is UTF8. Locales are set to C.UTF8. If lc_messages is C then no translation of error strings would happen, so that theory goes out the window. Oh well. Perhaps you could attach gdb to the backend and get a stack trace fro

Re: permission denied for large object 200936761

2021-02-02 Thread Adrian Klaver
On 2/2/21 4:12 AM, Andrus wrote: Hi! So? What is your point? Somebody created a large object of size 0. report table has bytea column. It looks like  psqlodbc driver adds ::lo cast  when inserting binary data: https://github.com/hlinnaka/psqlodbc/blob/master/convert.c#L4564 and this add

pg_dumpall and tablespaces

2021-02-02 Thread Joao Miguel Ferreira
Hello all, I have a dump file obtained from pg_dumpall on a MAC computer. I need to load in onto my Linux laptop running postgres. My scenario is software development. I'm trying to load the dump onto my Pg installation running on Linux (and later possibly on Linux over Docker) in order to perfor

Re: pg_dumpall and tablespaces

2021-02-02 Thread Adrian Klaver
On 2/2/21 8:43 AM, Joao Miguel Ferreira wrote: Hello all, I have a dump file obtained from pg_dumpall on a MAC computer. I need to load in onto my Linux laptop running postgres. My scenario is software development. I'm trying to load the dump onto my Pg installation running on Linux (and lat

Re: permission denied for large object 200936761

2021-02-02 Thread Andrus
Hi! So at that point the deed has been done. The questions to ask: 1) Why the driver thinks it is being  passed a large object in the first place? Source data type was binary. It was mapped to oid for unknown reason. 2) Have there been any recent changes to code that passes through the

Re: pg_dumpall and tablespaces

2021-02-02 Thread Joao Miguel Ferreira
On Tue, Feb 2, 2021 at 4:52 PM Adrian Klaver wrote: > On 2/2/21 8:43 AM, Joao Miguel Ferreira wrote: > > > I got 2 problems concerning tablespaces: > > a) during the restore step I get lots of errors about the necessity to > > have root permissions to re-create the tablespaces and > > b) the ta

Re: pg_dumpall and tablespaces

2021-02-02 Thread Tom Lane
Adrian Klaver writes: > On 2/2/21 8:43 AM, Joao Miguel Ferreira wrote: >> I have a dump file obtained from pg_dumpall on a MAC computer. I need to >> load in onto my Linux laptop running postgres. >> I got 2 problems concerning tablespaces: >> a) during the restore step I get lots of errors about

Re: pg_dumpall and tablespaces

2021-02-02 Thread Joao Miguel Ferreira
On Tue, Feb 2, 2021 at 5:08 PM Tom Lane wrote: > Adrian Klaver writes: > > On 2/2/21 8:43 AM, Joao Miguel Ferreira wrote: > >> I have a dump file obtained from pg_dumpall on a MAC computer. I need > to > >> load in onto my Linux laptop running postgres. > >> I got 2 problems concerning tablespac

Re: permission denied for large object 200936761

2021-02-02 Thread Adrian Klaver
On 2/2/21 9:05 AM, Andrus wrote: Hi! and used psqlodbc to insert this data: create temp table test ( test bytea ) on commit drop; insert into test values ( ?t.t ); This code throws exception type "lo" does not exist but each call adds new row to pg_largeobject_metadata table. Odbc driver

Re: ransomware

2021-02-02 Thread Tim Cross
Marc Millas writes: > Hi, > > I know its quite general. It is as I dont know what approaches may exist. > > Requirement is extremely simple: Is there anyway, from a running postgres > standpoint, to be aware that a ransomware is currently crypting your data ? > > answer can be as simple as: whe

Re: Segmentation fault on startup

2021-02-02 Thread Helmut Bender
(again to the list...) Am 01.02.21 um 21:40 schrieb Tom Lane: Helmut Bender writes: I'm running a nextcloud server in a docker container on an RasPi 4 (only SSD, no SD), which uses PostgreSQL 10 as server. 10.what? We're already up to 15 patch releases for that branch. As I use the docker

Re: count(*) vs count(id)

2021-02-02 Thread Hellmuth Vargas
Hello list My English is not very good, so I pretend that through the examples you understand what I intend to expose -- Recreate the query that is supposedly wrong select calendar.entry, count(*) from generate_series('2020-08-20'::date, '2020-09-15'::date, '1 day'::interval) as calendar(entry)

permission denied to create and drop user

2021-02-02 Thread Joao Miguel Ferreira
Hello all, I have just installed postgresql on Debian stable (from debian apt archives). It seems that the postgres user does not have permissions to DROP/CREATE USER. I was expecting the postgres user to be a superuser but something seems weird. my postgres user does not have the usual superuser

Re: permission denied to create and drop user

2021-02-02 Thread Adrian Klaver
On 2/2/21 1:58 PM, Joao Miguel Ferreira wrote: Hello all, I have just installed postgresql on Debian stable (from debian apt archives). It seems that the postgres user does not have permissions to DROP/CREATE USER. I was expecting the postgres user to be a superuser but something seems weird.

Re: permission denied to create and drop user

2021-02-02 Thread Tom Lane
Joao Miguel Ferreira writes: > I have just installed postgresql on Debian stable (from debian apt > archives). It seems that the postgres user does not have permissions to > DROP/CREATE USER. I was expecting the postgres user to be a superuser but > something seems weird. my postgres user does not

Re: permission denied to create and drop user

2021-02-02 Thread Joao Miguel Ferreira
Hi, On Tue, Feb 2, 2021 at 10:30 PM Tom Lane wrote: > Joao Miguel Ferreira writes: > > I have just installed postgresql on Debian stable (from debian apt > > archives). It seems that the postgres user does not have permissions to > > DROP/CREATE USER. I was expecting the postgres user to be a

Re: permission denied to create and drop user

2021-02-02 Thread Adrian Klaver
On 2/2/21 2:43 PM, Joao Miguel Ferreira wrote: Hi, But... my dump file contains some agressive commands that are actually making a reall mess. here they are: CREATE ROLE pgcon; ALTER ROLE pgcon WITH NOSUPERUSER INHERIT NOCREATEROLE NOCREATEDB LOGIN NOREPLICATION NOBYPASSRLS; CREATE ROLE

Re: count(*) vs count(id)

2021-02-02 Thread Cherio
I just ran a few practical tests on large (~14mil rows) tables that have multiple indexes. SELECT COUNT(id) forces PostgreSQL to use the primary key index. SELECT COUNT(*) allows PostgreSQL to chose an index to use and it seems to be choosing one of smaller size which leads to less IO and hence re