How to get a notification

2025-02-05 Thread Igor Korot
Hi, ALL, In my code I'm running following: queries.push_back( L"CREATE FUNCTION __watch_schema_changes() RETURNS event_trigger LANGUAGE plpgsql AS $$ BEGIN NOTIFY tg_tag; END; $$;" ); queries.push_back( L"CREATE EVENT TRIGGER schema_change_notify ON ddl_command_end WHEN TAG

Re: Table copy

2025-02-05 Thread Adrian Klaver
On 2/5/25 14:58, Andy Hartman wrote: Reduce batch size still caused error as reported nothing in WIndows Server log...  The SimplySql is slick because just qry from src and load to dst ... Is there any way to somehow show more of the error in PS i tried the $error but nothing very descriptiv

Re: Table copy

2025-02-05 Thread Andy Hartman
This is going to be a monthly process not just a 1 time exercise. On Wed, Feb 5, 2025 at 5:58 PM Andy Hartman wrote: > Reduce batch size still caused error as reported nothing in WIndows Server > log... > > The SimplySql is slick because just qry from src and load to dst ... Is > there any way

Re: Table copy

2025-02-05 Thread Andy Hartman
Reduce batch size still caused error as reported nothing in WIndows Server log... The SimplySql is slick because just qry from src and load to dst ... Is there any way to somehow show more of the error in PS i tried the $error but nothing very descriptive On Wed, Feb 5, 2025 at 4:23 PM Adrian Kl

Re: Lookup tables

2025-02-05 Thread Thiemo Kellner
El 05-02-25 a las 22:19, Michał Kłeczek escribió: But you can guarantee that if you change the value of the key after the user displays it - the user will get an error on submission (whereas with the surrogate key it would happily proceed without user noticing). As you very rightly say happil

Re: Table copy

2025-02-05 Thread Adrian Klaver
On 2/5/25 13:05, Andy Hartman wrote: nothing in log from mssql side and no  anti-virus How about the Windows Server 2019 system log? -- Adrian Klaver adrian.kla...@aklaver.com

Re: Table copy

2025-02-05 Thread Adrian Klaver
On 2/5/25 13:09, Andy Hartman wrote: I also reduced batch size to 5000 on the last run  .. I like using this SImplySql because it's a script I can launch, so that's why I chose that solution...  I'm using it to load a History Env. You really need to complete your thoughts, remember we have no

Re: Lookup tables

2025-02-05 Thread Michał Kłeczek
> On 5 Feb 2025, at 21:33, Thiemo Kellner wrote: > > > El 05-02-25 a las 13:55, Michał Kłeczek escribió: >>> A) Your release changed the sementics of the record 3. It's meaning >>> changed. I cannot recommend doing that. >> That’s what using natural keys and FK’s restricting their changes gu

Re: Table copy

2025-02-05 Thread Ron Johnson
Could there have been a network hiccup? Or some sort of timeout? If I needed to transfer 360GB of data, I'd probably do something old school like: 1. write a PowerShell script to export a set of rows into a csv file, 7zip compress it, then rsync or scp it to the target. 2. Write a bash script to

Re: Table copy

2025-02-05 Thread Andy Hartman
I also reduced batch size to 5000 on the last run .. I like using this SImplySql because it's a script I can launch, so that's why I chose that solution... I'm using it to load a History Env. THanks again. On Wed, Feb 5, 2025 at 4:05 PM Andy Hartman wrote: > nothing in log from mssql side and

Re: Table copy

2025-02-05 Thread Andy Hartman
nothing in log from mssql side and no anti-virus On Wed, Feb 5, 2025 at 2:06 PM Adrian Klaver wrote: > > > On 2/5/25 9:46 AM, Andy Hartman wrote: > > [6992] ERROR: unexpected EOF on client connection with an open > transaction > > 2025-02-05 12:19:44.919 EST [6992] CONTEXT: COPY sqlt_data_1_2

Re: Lookup tables

2025-02-05 Thread Thiemo Kellner
El 05-02-25 a las 13:55, Michał Kłeczek escribió: A) Your release changed the sementics of the record 3. It's meaning changed. I cannot recommend doing that. That’s what using natural keys and FK’s restricting their changes guarantee: no (accidental) changes to meaning of data. Even with cas

Re: Lookup tables

2025-02-05 Thread Thiemo Kellner
El 05-02-25 a las 19:13, Michał Kłeczek escribió: Only if you do not see the primary key as the main immutable value identifying an object, entity, you name it. Surrogate key cannot identify any (real) object by definition :) What object is identified by PK value 42 in “restaurants” table? Wh

Re: Table copy

2025-02-05 Thread Adrian Klaver
On 2/5/25 9:46 AM, Andy Hartman wrote: [6992] ERROR:  unexpected EOF on client connection with an open transaction 2025-02-05 12:19:44.919 EST [6992] CONTEXT:  COPY sqlt_data_1_2022_03, line 24431524, column dataintegrity 2025-02-05 12:19:44.919 EST [6992] STATEMENT:  COPY sqlt_data_1_2022_03

Re: Lookup tables

2025-02-05 Thread Michał Kłeczek
> On 5 Feb 2025, at 19:07, Thiemo Kellner wrote: > > El 04-02-25 a las 18:08, Michał Kłeczek escribió: >>> Reality tends to become so ambiguous as to not be >>> reflectable (two entirely different restaurants eventually, >>> within the flow of time, carry the very same name). >>> >>> A primar

Re: Lookup tables

2025-02-05 Thread Thiemo Kellner
El 04-02-25 a las 18:08, Michał Kłeczek escribió: Reality tends to become so ambiguous as to not be reflectable (two entirely different restaurants eventually, within the flow of time, carry the very same name). A primary key is very likely not the proper place to reflect arbitrary business logi

Using PgAgent with SQL commands only?

2025-02-05 Thread Csányi Pál
Hello, I am using Debian 12 operating system and have installed on it postgresql 15.10 (Debian 15.10-0+deb12u1) and pgagent -V PostgreSQL Scheduling Agent Version: 4.2.2 booth installed with apt. PgAgent is running as service: systemctl status pgagent ● pgagent.service - pgAgent for PostgreSQ

Re: Table copy

2025-02-05 Thread Andy Hartman
[6992] ERROR: unexpected EOF on client connection with an open transaction 2025-02-05 12:19:44.919 EST [6992] CONTEXT: COPY sqlt_data_1_2022_03, line 24431524, column dataintegrity 2025-02-05 12:19:44.919 EST [6992] STATEMENT: COPY sqlt_data_1_2022_03 (tagid, intvalue, floatvalue, stringvalue, d

Re: Lookup tables

2025-02-05 Thread Thiemo Kellner
04.02.2025 18:12:02 David G. Johnston : > On Tue, Feb 4, 2025 at 9:31 AM Michał Kłeczek wrote: > > Well, we were talking about lookup tables and not entity modelling... I am under the impression that a lookup table IS an entity. You find them in star and snowflake models alike. > >> >> Havi

Re: Table copy

2025-02-05 Thread Adrian Klaver
On 2/5/25 05:40, Andy Hartman wrote: mssql - 2016 PG - 16.1 latest release for Simplysql Latest is a relative term and requires anyone in the future coming across this thread to work out what you are talking about. For that person the current version is 2.1.0. Invoke-SqlBulkCopy -SourceConnec

Re: Commit Latency

2025-02-05 Thread Álvaro Herrera
Hello On 2025-Feb-05, Ramakrishna m wrote: > I have a system handling *300 TPS*, with resource usage *below 10%*. > However, I’m noticing *commit latency of around 200ms* for *1% of > transactions*, occasionally spiking to *1 second*. Since there is no > significant *I/O pressure*, I’m trying to

Re: Table copy

2025-02-05 Thread Andy Hartman
mssql - 2016 PG - 16.1 latest release for Simplysql Invoke-SqlBulkCopy -SourceConnectionName "src" -SourceTable "sqlt_data_1_2022_03" -DestinationConnectionName "dst" -DestinationTable "sqlt_data_1_2022_03" -BatchSize 1 -Notify 2.4 billion records -- I have down other tables of same size no p

Re: Commit Latency

2025-02-05 Thread Laurenz Albe
On Wed, 2025-02-05 at 15:48 +0530, Ramakrishna m wrote: > I’m noticing commit latency of around 200ms for 1% of transactions, > occasionally > spiking to 1 second. Since there is no significant I/O pressure, I’m trying to > identify what else might be causing this in PostgreSQL 16. https://www.cy

Re: Commit Latency

2025-02-05 Thread Tomas Vondra
I think the first thing you should do is verify the performance of the storage, particularly how long it takes to do fsync. You didn't explain what kind of storage or filesystem you're using, so hard to say. It's not clear to me what "resource usage" means exactly, but the fsync may be slow even w

Re: Lookup tables

2025-02-05 Thread Michał Kłeczek
> On 4 Feb 2025, at 22:41, Thiemo Kellner wrote: > > 04.02.2025 18:31:09 Michał Kłeczek : > >> >>> On 4 Feb 2025, at 18:27, Thiemo Kellner wrote: >>> >>>  Unless the lookup table is actually a check constraint one can use to >>> populate dropdown boxes in an interface. >> >> That is eve