Re: libc to libicu via pg_dump/pg_restore?

2025-02-06 Thread Paul Foerster
Hi Guillaume, > On 6 Feb 2025, at 11:13, Guillaume Lelarge > wrote: > > You probably don't need --disable-triggers. You should fix errors in the > order they appear. The first one is on the drop of the database: > > ERROR: cannot drop the currently open database > > pg_restore can't drop th

libc to libicu via pg_dump/pg_restore?

2025-02-06 Thread Paul Foerster
Hi, I have a problem which I don't understand. I have and do: instance a, libc based, PostgreSQL 15.10: mydb=# \l mydb List of databases Name | Owner | Encoding | Collate |Ctype| ICU Locale | Locale Provider | Access priv

Re: libc to libicu via pg_dump/pg_restore?

2025-02-06 Thread Guillaume Lelarge
Hi, On 06/02/2025 10:04, Paul Foerster wrote: Hi, I have a problem which I don't understand. I have and do: instance a, libc based, PostgreSQL 15.10: mydb=# \l mydb List of databases Name | Owner | Encoding | Collate |Cty

Re: How to perform a long running dry run transaction without blocking

2025-02-06 Thread Robert Leach
> Have you considered a validation app? Have it read the inputs and look in db > for conflicts, rather than attempt the insert. Zero transactions necessary I did consider that about a year or two ago when I first conceived the data validation interface. Doing that now would solve the problem o

Re: libc to libicu via pg_dump/pg_restore?

2025-02-06 Thread Adrian Klaver
On 2/6/25 09:37, Paul Foerster wrote: Hi Guillaume, On 6 Feb 2025, at 15:51, Guillaume Lelarge wrote: You're right. Now I see the "create database" query in your previous email. I should have been more careful, sorry for the noise. No problem. Well, the doc says that --disable-triggers i

Re: Lookup tables

2025-02-06 Thread Peter J. Holzer
On 2025-02-04 22:41:38 +0100, Thiemo Kellner wrote: > 04.02.2025 18:31:09 Michał Kłeczek : > > > > >> On 4 Feb 2025, at 18:27, Thiemo Kellner > >> wrote: > >> > >>  Unless the lookup table is actually a check constraint one > >> can use to populate dropdown boxes in an interface. > > > > Tha

Re: How to perform a long running dry run transaction without blocking

2025-02-06 Thread Robert Leach
>>> The load to the development server does no validation? >>> >>> If so what is the purpose? >>> >>> The background processes are other validation runs? >> It's the same code that executes in both cases (with or without the >> `--validate` flag). All that that flag does is it (effectively) rai

Re: libc to libicu via pg_dump/pg_restore?

2025-02-06 Thread Paul Foerster
Hi Guillaume, > On 6 Feb 2025, at 15:51, Guillaume Lelarge > wrote: > > You're right. Now I see the "create database" query in your previous email. I > should have been more careful, sorry for the noise. No problem. > Well, the doc says that --disable-triggers is only relevant for data-only

Re: libc to libicu via pg_dump/pg_restore?

2025-02-06 Thread Guillaume Lelarge
Hi Paul, On 06/02/2025 12:20, Paul Foerster wrote: Hi Guillaume, On 6 Feb 2025, at 11:13, Guillaume Lelarge wrote: You probably don't need --disable-triggers. You should fix errors in the order they appear. The first one is on the drop of the database: ERROR: cannot drop the currently ope

Re: How to perform a long running dry run transaction without blocking

2025-02-06 Thread Robert Leach
> Please reply to list also. > Ccing list 👍🏻 >> They enter the data in an excel spreadsheet containing about a dozen >> inter-related worksheets, named, for example: Study, Animals, Samples, >> Tissues, Treatments, Sequences > > Long term is there a thought to have them enter directly into dat

Re: How to perform a long running dry run transaction without blocking

2025-02-06 Thread Adrian Klaver
On 2/6/25 09:58, Robert Leach wrote: They enter the data in an excel spreadsheet containing about a dozen inter-related worksheets, named, for example: Study, Animals, Samples, Tissues, Treatments, Sequences Long term is there a thought to have them enter directly into database where validat

Re: How to perform a long running dry run transaction without blocking

2025-02-06 Thread Rob Sargent
> Have you considered a validation app? Have it read the inputs and look in db for conflicts, rather than attempt the insert. Zero transactions necessary

Re: libc to libicu via pg_dump/pg_restore?

2025-02-06 Thread Adrian Klaver
On 2/6/25 01:04, Paul Foerster wrote: Hi, Comments inline. I have a problem which I don't understand. I have and do: instance a, libc based, PostgreSQL 15.10: mydb=# \l mydb List of databases Name | Owner | Encoding | Collat

Re: How to perform a long running dry run transaction without blocking

2025-02-06 Thread Adrian Klaver
On 2/6/25 08:53, Robert Leach wrote: Great questions! Responses inline... Please reply to list also. Ccing list Load to where existing table or temporary table? Existing tables. Note that (and this is a point of contention in our team), we have some fields that must be universally unique

Re: libc to libicu via pg_dump/pg_restore?

2025-02-06 Thread Paul Foerster
Hi Adrian, > On 6 Feb 2025, at 17:31, Adrian Klaver wrote: > > 1) Log into postgres database and do: > > a) DROP DATABASE mydb; > b) CREATE DATABASE mydb ; > > 2) pg_restore -d mydb mydb.dump.gz With create database being "template template0", this is what my script does. But I need the -cC

Re: libc to libicu via pg_dump/pg_restore?

2025-02-06 Thread Paul Foerster
Hi Adrian, > On 6 Feb 2025, at 19:44, Adrian Klaver wrote: > > By ACL do you mean roles? > > If so roles are global to the cluster not the database, so I am not seeing -C > being relevant. > > If not you will need to be more specific about what you are referring to. I did a "pg_dumpall -r >r

Re: How to perform a long running dry run transaction without blocking

2025-02-06 Thread Adrian Klaver
On 2/6/25 12:08, Robert Leach wrote: Alright I am trying to reconcile this with from below, 'The largest studies take just under a minute'. The context of the 'The largest studies take just under a minute' statement is that it's not loading the hefty/time-consuming raw data.  It's only val

Re: How to perform a long running dry run transaction without blocking

2025-02-06 Thread Adrian Klaver
On 2/6/25 11:25, Robert Leach wrote: Have you considered a validation app? Have it read the inputs and look in db for conflicts, rather than attempt the insert. Zero transactions necessary I did consider that about a year or two ago when I first conceived the data validation interface. Doin

Re: Help in vetting Switch from "MD5" to "scram-sha-256" - during DB Upgrade from EC2- PGS - Community Edn ver 13.X to 15.X

2025-02-06 Thread Adrian Klaver
On 2/6/25 16:37, Bharani SV-forum wrote:  Team I am in the process of doing DB Upgrade from EC2- PGS - Community Edn ver 13.X to 15.X including switching from "MD5 " to "scram-sha-256" for password hashing and authentication. We are having tightly integrated appln tech stack having 256+ role

Re: Help in vetting Switch from "MD5" to "scram-sha-256" - during DB Upgrade from EC2- PGS - Community Edn ver 13.X to 15.X

2025-02-06 Thread Adrian Klaver
On 2/6/25 18:03, Bharani SV-forum wrote: Adrian TQ for your valuable input's. *Additional Qsn* Assume  DB ver = 15.X By default encryption = scram-sha-256, Assume pg_hba.conf is quoted the usage as MD5 for the  dbuserid "test_usr_1" *e.g .)* * * hostssl   all test_usr_1 10.20.30.40  md5 i

How to perform a long running dry run transaction without blocking

2025-02-06 Thread Robert Leach
I've been trying to solve this problem in Django and I've finally decided after over a year going down this path that there's no way to solve it in Django (in a database agnostic fashion). So I'm thinking that I need to explore a Postgres-specific solution. FYI, I'm not a database expert. I just

Re: How to perform a long running dry run transaction without blocking

2025-02-06 Thread Adrian Klaver
On 2/6/25 07:40, Robert Leach wrote: Comments inline. Let me try and distill all the back story down to an oversimplified explanation: I created a scientific data submission validation interface that helps researchers compile their data submissions to our database. To do this, I decided to a