Re: READ UNCOMMITTED in postgres

2019-12-18 Thread Thomas Kellerer
Matthew Phillips schrieb am 19.12.2019 um 00:12: > Hi, With the current READ UNCOMMITTED discussion happening on > pgsql-hackers [1], It did raise a question/use-case I recently > encountered and could not find a satisfactory solution for. If > someone is attempting to poll for new records on a hig

Re: AccessExclusiveLock with pg_locks.locktype of tuple

2019-12-18 Thread Tom Lane
Erik Jones writes: > I've seen locks with mode of AccessExclusiveLock and locktype of tuple a > few times now but have never been able to reproduce one nor had a chance to > dig into them and now have a couple questions on them: > * When is this kind of heavy lock on a row/tuple taken out? I've d

Re: READ UNCOMMITTED in postgres

2019-12-18 Thread Stephen Frost
Greetings, * Matthew Phillips (mphillip...@gmail.com) wrote: > With the current READ UNCOMMITTED discussion happening on pgsql-hackers > [1], It did raise a question/use-case I recently encountered and could not > find a satisfactory solution for. If someone is attempting to poll for new > records

READ UNCOMMITTED in postgres

2019-12-18 Thread Matthew Phillips
Hi, With the current READ UNCOMMITTED discussion happening on pgsql-hackers [1], It did raise a question/use-case I recently encountered and could not find a satisfactory solution for. If someone is attempting to poll for new records on a high insert volume table that has a monotonically increasing

AccessExclusiveLock with pg_locks.locktype of tuple

2019-12-18 Thread Erik Jones
Greetings, I've seen locks with mode of AccessExclusiveLock and locktype of tuple a few times now but have never been able to reproduce one nor had a chance to dig into them and now have a couple questions on them: * When is this kind of heavy lock on a row/tuple taken out? I've done some code s

Re: How to prevent POSTGRES killing linux system from accepting too much inserts?

2019-12-18 Thread Osahon Oduware
THE TRUTH CANNOT BE HIDDEN **Explosion in my car. https://www.docdroid.net/s11XHOS/the-truth-cannot-be-hidden.pdf On Wed, 18 Dec 2019, 10:54 James(王旭), wrote: > Hello, >> >> I encountered into this kernel message, and I cannot login into the Linux >> system anymore: > > >> >> Dec 17 23:01:50

Re: Postgres AssertFailedException Exception

2019-12-18 Thread Tom Lane
"Patil, Prashant" writes: > We are receiving following AssertFailedException exception while executing > pgrouting query listed below. Anyone have idea about this exception? Is this > known bug? Possible fix? The stack trace shows the exception is well down inside libpgrouting, so you'd need to

Postgres AssertFailedException Exception

2019-12-18 Thread Patil, Prashant
Hi Team, We are receiving following AssertFailedException exception while executing pgrouting query listed below. Anyone have idea about this exception? Is this known bug? Possible fix? Server details - Postgresql 9.3.25 Ubuntu 16.04.6 LTS Pgrouting 2.6.0 2019-12-16 19:31:55 EST Passenger AppPr

Re: How to prevent POSTGRES killing linux system from accepting too much inserts?

2019-12-18 Thread Jeff Janes
On Wed, Dec 18, 2019 at 4:53 AM James(王旭) wrote: > Hello, >> >> I encountered into this kernel message, and I cannot login into the Linux >> system anymore: > > >> >> Dec 17 23:01:50 hq-pg kernel: sh (6563): drop_caches: 1 >> >> Dec 17 23:02:30 hq-pg kernel: INFO: task sync:6573 blocked for more

Re: How to prevent POSTGRES killing linux system from accepting too much inserts?

2019-12-18 Thread Merlin Moncure
On Wed, Dec 18, 2019 at 3:53 AM James(王旭) wrote: > > Hello, >> >> I encountered into this kernel message, and I cannot login into the Linux >> system anymore: >> >> >> >>> Dec 17 23:01:50 hq-pg kernel: sh (6563): drop_caches: 1 >>> >>> Dec 17 23:02:30 hq-pg kernel: INFO: task sync:6573 blocked fo

Re: How to prevent POSTGRES killing linux system from accepting too much inserts?

2019-12-18 Thread Steven Lembark
On Wed, 18 Dec 2019 17:53:26 +0800 "James(王旭)" wrote: > Hello,I encountered into this kernel message, and I cannot login into > the Linux system anymore: > > Dec 17 23:01:50 hq-pg kernel: sh (6563): drop_caches: 1Dec 17 > 23:02:30 hq-pg kernel: INFO: task sync:6573 blocked for more than 120 > se

Re: Tuple concurrency issue in large objects

2019-12-18 Thread Justin
I agree completely, I do not think Postgresql is a good fit for Shalini based on the conversation so far tracking Concurrency is going to be a killer... But i see the temptation to use a DB for this as the updates are ACID less likely to corrupted data for X reason On Wed, Dec 18, 2019 at 12:1

Re: Tuple concurrency issue in large objects

2019-12-18 Thread Tom Lane
Justin writes: > I now see what is causing this specific issue... > The update and row versions is happening on 2kb chunk at a time, That's > going to make tracking what other clients are doing a difficult task. Yeah, it's somewhat unfortunate that the chunkiness of the underlying data storage b

Re: Tuple concurrency issue in large objects

2019-12-18 Thread Justin
I now see what is causing this specific issue... The update and row versions is happening on 2kb chunk at a time, That's going to make tracking what other clients are doing a difficult task. All the clients would have to have some means to notify all the other clients that an update occurred in

Re: Tuple concurrency issue in large objects

2019-12-18 Thread Tom Lane
Justin writes: > I have a question reading through this email chain. Does Large Objects > table using these functions work like normal MVCC where there can be two > versions of a large object in pg_largeobject . Yes, otherwise you could never roll back a transaction that'd modified a large obje

Re: Tuple concurrency issue in large objects

2019-12-18 Thread Justin
I have a question reading through this email chain. Does Large Objects table using these functions work like normal MVCC where there can be two versions of a large object in pg_largeobject . My gut says no as moving/copying potentially 4 TB of data would kill any IO. I can not find any document

Re: Tuple concurrency issue in large objects

2019-12-18 Thread Daniel Verite
Shalini wrote: > Could you also please state the reason why is it happening in case > of large objects? Because concurrent transactions are very well > handled for other data types, but the same is not happening for > lobs. Is it because the fomer are stored in toast table and there is > n

Re: Row locks, SKIP LOCKED, and transactions

2019-12-18 Thread Tom Lane
Steven Winfield writes: >> There are various ways you could deal with this, but I'd probably go for a >> simple scheme where you only have to consult a single row to know if you >> can claim it. You could still put the results into a separate table, but >> use job.state to find work, and set it t

Streaming replication fails after some time with 'incorrect resource manager data checksum'

2019-12-18 Thread Julian Backes
Hello all! I already posted in the slack chat #help channel but got no answer :-( We have a read only / hot standby system and are facing the same problem as described in https://stackoverflow.com/questions/35752389/incorrect-resource-manager-data-checksum-in-record-at-2-xyz-terminating-walrec (t

Re: REINDEX VERBOSE unknown option

2019-12-18 Thread Josef Šimánek
My patch was partially merged at https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=52dcfda48778d16683c64ca4372299a099a15b96 . Thanks to everyone involved in this discussion. pá 13. 12. 2019 v 2:11 odesílatel Josef Šimánek napsal: > I was thinking about this problem and possible

RE: Row locks, SKIP LOCKED, and transactions

2019-12-18 Thread Steven Winfield
> Huh. SERIALIZABLE shouldn't allow two transactions to see no result row > for a given ID and then insert a result row for that ID. One of those > transactions should have to roll back, because otherwise it'd be > incompatible with both serial orderings of the two transactions. Sorry for the mi

RE: Row locks, SKIP LOCKED, and transactions

2019-12-18 Thread Steven Winfield
>> * I observe this even if I crank up the transaction isolation level to >> repeatable read and serializable. >> >> >> I'm wondering if row locks are not obeying the same transactional semantics >> as row data, >Gotta believe it is this: > >https://www.postgresql.org/docs/11/transaction-iso

How to prevent POSTGRES killing linux system from accepting too much inserts?

2019-12-18 Thread James(王旭)
Hello,I encountered into this kernel message, and I cannot login into the Linux system anymore: Dec 17 23:01:50 hq-pg kernel: sh (6563): drop_caches: 1Dec 17 23:02:30 hq-pg kernel: INFO: task sync:6573 blocked for more than 120 seconds.Dec 17 23:02:30 hq-pg kernel: "echo 0 > /proc/sys/kernel/hu

Re: Encoding/collation question

2019-12-18 Thread Karsten Hilbert
On Thu, Dec 12, 2019 at 08:35:53AM -0500, Tom Lane wrote: > C collation basically devolves to strcmp/memcmp, which are as standard > and well-defined as can be. If you're happy with the way it sorts > things then there's no reason not to use it. So that's the collation to use when "technical" so