Re: IO related waits

On 9/17/24 12:34, veem v wrote: On Tue, 17 Sept 2024 at 21:24, Adrian Klaver > wrote: Which means you need to on Flink end: 1) Use Flink async I/O . 2) Find a client that supports async or fake it by using multiple synchronous clients.

Re: load fom csv

I'll echo vars and see if something looks strange. THanks. On Tue, Sep 17, 2024 at 3:46 PM Rob Sargent wrote: > > > > On Sep 17, 2024, at 12:25 PM, Adrian Klaver > wrote: > > > > On 9/17/24 09:21, Andy Hartman wrote: > >> The command work outside of powershell yes > > > > Then you are going

Re: load fom csv

> On Sep 17, 2024, at 12:25 PM, Adrian Klaver wrote: > > On 9/17/24 09:21, Andy Hartman wrote: >> The command work outside of powershell yes > > Then you are going to need to use whatever debugging tools PowerShell has > available to step through the script to figure out where the problem

Re: IO related waits

On Tue, 17 Sept 2024 at 21:24, Adrian Klaver wrote: > > Which means you need to on Flink end: > > 1) Use Flink async I/O . > > 2) Find a client that supports async or fake it by using multiple > synchronous clients. > > On Postgres end there is this: > > https://www.postgresql.org/docs/current/wa

Re: Failing to allocate memory when I think it shouldn't

Hi, ## Thomas Ziegler (thomas.zieg...@holmsecurity.com): > Except for pgAudit, I don't have any extensions, so it is probably the > JIT. I had no idea there was a JIT, even it should have been obvious. > Thanks for pointing this out! There is - it even has it's own chapter in the documentation:

Re: load fom csv

Still when I try to run from my powershell script it hangs... On Tue, Sep 17, 2024 at 8:31 AM Andy Hartman wrote: > I have bad data in an Int field... > > Thanks for your help. > > On Tue, Sep 17, 2024 at 1:55 AM Muhammad Usman Khan > wrote: > >> Hi, >> Try the following options: >> >> >>-

Re: load fom csv

On 9/17/24 09:21, Andy Hartman wrote: The command work outside of powershell  yes Then you are going to need to use whatever debugging tools PowerShell has available to step through the script to figure out where the problem is. -- Adrian Klaver adrian.kla...@aklaver.com

Re: load fom csv

The command work outside of powershell yes On Tue, Sep 17, 2024 at 11:39 AM Adrian Klaver wrote: > On 9/17/24 08:35, Andy Hartman wrote: > > I don't see anything in LOG > > 1) Are you logging connections/disconnection per?: > > > https://www.postgresql.org/docs/current/runtime-config-logging.ht

Re: IO related waits

On 9/16/24 20:55, veem v wrote: On Tue, 17 Sept 2024 at 03:41, Adrian Klaver > wrote: Are you referring to this?: https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/datastream/operators/asyncio/

Re: IO related waits

On Tue, 17 Sept 2024 at 18:43, Greg Sabino Mullane wrote: > > This is a better place to optimize. Batch many rows per transaction. > Remove unused indexes. > > flushing of the WAL to the disk has to happen anyway(just that it will be >> delayed now), so can this method cause contention in the dat

Re: load fom csv

On 9/17/24 08:35, Andy Hartman wrote: I don't see anything in LOG 1) Are you logging connections/disconnection per?: https://www.postgresql.org/docs/current/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-WHAT If not do so as it will show you if a connection is being attempted. 2) Do the

Re: load fom csv

I don't see anything in LOG On Tue, Sep 17, 2024 at 11:23 AM Adrian Klaver wrote: > On 9/17/24 08:13, Andy Hartman wrote: > > Still when I try to run from my powershell script it hangs... > > > > And the Postgres log shows? > > -- > Adrian Klaver > adrian.kla...@aklaver.com > >

Re: load fom csv

On 9/17/24 08:13, Andy Hartman wrote: Still when I try to run from my powershell script it hangs... And the Postgres log shows? -- Adrian Klaver adrian.kla...@aklaver.com

Re: question on plain pg_dump file usage

On 9/17/24 05:43, Ron Johnson wrote: On Tue, Sep 17, 2024 at 8:22 AM Zwettler Markus (OIZ) mailto:markus.zwett...@zuerich.ch>> wrote: Why must it be a plain text dump instead of a custom or directory dump? Restoring to a new (and differently-named( database is perfectly doable. Because o

Re: question on plain pg_dump file usage

"Zwettler Markus (OIZ)" writes: > I have to do an out-of-place Postgres migration from PG12 to PG16 using: > pg_dump -F p -f dump.sql ... > sed -i "s/old_name/new_name/g" > psql -f dump.sql ... > Both databases are on UTF-8. > I wonder if there could be character set conversion errors here, as t

WG: question on plain pg_dump file usage

Von: Zwettler Markus (OIZ) Gesendet: Dienstag, 17. September 2024 14:22 An: PG-General Mailing List Betreff: question on plain pg_dump file usage I have to do an out-of-place Postgres migration from PG12 to PG16 using: pg_dump -F p -f dump.sql ... sed -i "s/old_name/new_name/g" psql -f dump.sq

Re: question on plain pg_dump file usage

On Tue, Sep 17, 2024 at 8:22 AM Zwettler Markus (OIZ) < markus.zwett...@zuerich.ch> wrote: > pg_dump -F p -f dump.sql … > > sed -i "s/old_name/new_name/g" > > psql -f dump.sql … > Why not rename afterwards? Just "pg_dump mydb | psql -h newhost -f -" and rename things via ALTER. Certainly much saf

WG: [Extern] Re: question on plain pg_dump file usage

Von: Ron Johnson Gesendet: Dienstag, 17. September 2024 14:44 An: PG-General Mailing List Betreff: [Extern] Re: question on plain pg_dump file usage On Tue, Sep 17, 2024 at 8:22 AM Zwettler Markus (OIZ) mailto:markus.zwett...@zuerich.ch>> wrote: I have to do an out-of-place Postgres migration

Re: IO related waits

On Mon, Sep 16, 2024 at 11:56 PM veem v wrote: > So what can be the caveats in this approach, considering transactions > meant to be ACID compliant as financial transactions. > Financial transactions need to be handled with care. Only you know your business requirements, but as Christophe pointe

Re: question on plain pg_dump file usage

On Tue, Sep 17, 2024 at 8:22 AM Zwettler Markus (OIZ) < markus.zwett...@zuerich.ch> wrote: > I have to do an out-of-place Postgres migration from PG12 to PG16 using: > > > > pg_dump -F p -f dump.sql … > > sed -i "s/old_name/new_name/g" > > psql -f dump.sql … > > > > Both databases are on UTF-8. >

Re: load fom csv

I have bad data in an Int field... Thanks for your help. On Tue, Sep 17, 2024 at 1:55 AM Muhammad Usman Khan wrote: > Hi, > Try the following options: > > >- Check if psql is working independently: >psql -h $pgServer -d $pgDatabase -U $pgUser -c "SELECT 1;" >- Check for permission

question on plain pg_dump file usage

I have to do an out-of-place Postgres migration from PG12 to PG16 using: pg_dump -F p -f dump.sql ... sed -i "s/old_name/new_name/g" psql -f dump.sql ... Both databases are on UTF-8. I wonder if there could be character set conversion errors here, as the data is temporarily written to a plain t

Re: update faster way

On 2024-Sep-14, yudhi s wrote: > Hello, > We have to update a column value(from numbers like '123' to codes like > 'abc' by looking into a reference table data) in a partitioned table with > billions of rows in it, with each partition having 100's millions rows. Another option is to not update an

Re: Failing to allocate memory when I think it shouldn't

Hello Christoph, Thanks for your answer and the suggestions, it already helped me out a lot! On 2024-09-14 22:11, Christoph Moench-Tegeder wrote: Hi, ## Thomas Ziegler (thomas.zieg...@holmsecurity.com): There's a lot of information missing here. Let's start from the top. I have had my datab