Re: [HACKERS] Academic help for Postgres

et me > know if it needs more additions/changes. Thanks. Maybe slide 7 (NoSQL Sacrifices) should have a bullet point for "transaction isolation"? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

Re: [HACKERS] WALWriteLock contention

PostgreSQL 9.0 High Performance in chapter 4 (page 79). -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

(range->rangeStart <= range->rangeEnd && (segpage < range->rangeStart || segpage > range->rangeEnd))) SlruDeleteSegment(ctl, filename); -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

eaders (updating, locking or vacuuming). If you have truncated multixacts referenced in your tuples then you have a different form of corruption than the pg_upgrade-tramples-on-oldestMultiXactId case we're trying to handle gracefully here. -- Thomas Munro http://www.enterprisedb.com tolerate-missing-offset-segments-wip.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

think any of the code > that is affected here exists in 9.2, or that any of the fixes involved > were back-patched to 9.2. So that might be something else altogether. Not only that, the pg_multixact/members *directory* is reported missing, which is a different problem entirely. -- Th

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On Sat, May 30, 2015 at 10:48 AM, Andres Freund wrote: > On 2015-05-30 10:41:01 +1200, Thomas Munro wrote: >> On Sat, May 30, 2015 at 10:29 AM, Robert Haas wrote: >> > On Fri, May 29, 2015 at 5:14 PM, Josh Berkus wrote: >> >> Just saw what looks like

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

ltixacts was copied. I haven't managed to get this to work (ie produce a FATAL) and I'm out of time for a little while, but wanted to share this idea in case it helps someone. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Multixid hindsight design

inor > release. What about prepared transactions? They can lock rows FOR SHARE that survive server restarts. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.o

Re: [HACKERS] pg_xlog -> pg_xjournal?

out using your other suggestion "pg_xact" for that? It fits alongside pg_multixact. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

ster). I have also been considering a scenario where multixact ID wraparound occurs during basebackup with some ordering that causes trouble, but I don't yet see why it would break if you replay the WAL from the backup label checkpoint (and I think the repro would take days/weeks to run...) -

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On Wed, Jun 3, 2015 at 3:42 PM, Alvaro Herrera wrote: > Thomas Munro wrote: >> On Tue, Jun 2, 2015 at 9:30 AM, Alvaro Herrera >> wrote: >> > My guess is that the file existed, and perhaps had one or more pages, >> > but the wanted page doesn't exis

[HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

f transaction 131072 DETAIL: Could not open file "pg_multixact/offsets/0002": No such file or directory. But, yeah, this isn't the bug we're looking for. -- Thomas Munro http://www.enterprisedb.com checkpoint-page-boundary.sh Description: Bourne shell script checkpoint-segme

[HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

http://www.postgresql.org/message-id/CAEepm=1_KbHGbmPVmkUGE5qTP+B4efoCJYS0unGo-Mc5NV=u...@mail.gmail.com I see the following during shutdown checkpoint: LOG: could not truncate directory "pg_multixact/offsets": apparent wraparound That message comes from SimpleLruTruncate. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On Fri, Jun 5, 2015 at 11:47 AM, Thomas Munro wrote: > On Fri, Jun 5, 2015 at 9:29 AM, Robert Haas wrote: >> Here's a new version with some more fixes and improvements: >> >> - SetOffsetVacuumLimit was failing to set MultiXactState->oldestOffset >> when the ol

[HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On Fri, Jun 5, 2015 at 1:47 PM, Thomas Munro wrote: > On Fri, Jun 5, 2015 at 11:47 AM, Thomas Munro > wrote: >> On Fri, Jun 5, 2015 at 9:29 AM, Robert Haas wrote: >>> Here's a new version with some more fixes and improvements: >>> [...] >>

Re: [HACKERS] could not truncate directory "pg_subtrans": apparent wraparound

r similar to multixacts, described here: http://www.postgresql.org/message-id/CAEepm=0DqAtnM=23oq44bbnwvn3g6+dxx+s5g4jrbp-vy8g...@mail.gmail.com I will try to repro this. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] could not truncate directory "pg_subtrans": apparent wraparound

On Sat, Jun 6, 2015 at 10:12 AM, Thomas Munro wrote: > On Sat, Jun 6, 2015 at 9:45 AM, Dan Langille wrote: >> I noticed this today on my 9.4.2 server running on FreeBSD 10.1: >> >> Jun 5 18:59:40 slocum postgres[986]: [3957-1] LOG: could not truncate >> direct

Re: [HACKERS] could not truncate directory "pg_subtrans": apparent wraparound

On Sat, Jun 6, 2015 at 1:25 PM, Alvaro Herrera wrote: > Thomas Munro wrote: > >> My idea was that if I could get oldestXact == next XID in >> TruncateSUBSTRANS, then TransactionIdToPage(oldestXact) for a value of >> oldestXact that happens to be immediately after a page

Re: [HACKERS] could not truncate directory "pg_subtrans": apparent wraparound

On Sat, Jun 6, 2015 at 4:51 PM, Thomas Munro wrote: > On Sat, Jun 6, 2015 at 1:25 PM, Alvaro Herrera > wrote: >> Thomas Munro wrote: >> >>> My idea was that if I could get oldestXact == next XID in >>> TruncateSUBSTRANS, then TransactionIdToPage(oldestXact

Re: [HACKERS] could not truncate directory "pg_subtrans": apparent wraparound

On Mon, Jun 8, 2015 at 12:29 PM, Thomas Munro wrote: > Here's a repro script and a suggested patch. Argh... I realised immediately after sending this that subtransaction truncation doesn't even use the oldest XID computed by vacuum, it uses GetOldestXmin (the "oldest transactio

[HACKERS] Checkpoints vs restartpoints

crash recover time for lower write IO? I was wondering about this in the context of the recent multixact work, since such configurations could leave you with different SLRU files on disk which in some versions might change the behaviour in interesting ways. -- Thomas Munro http://www.enterprised

Re: [HACKERS] could not truncate directory "pg_subtrans": apparent wraparound

;t always consume multixacts. [1] http://www.postgresql.org/message-id/CAEepm=0DqAtnM=23oq44bbnwvn3g6+dxx+s5g4jrbp-vy8g...@mail.gmail.com [2] https://commitfest.postgresql.org/5/265/ [3] http://www.postgresql.org/message-id/1274373980-sup-3...@alvh.no-ip.org -- Thomas Munro http://www.enterprisedb.com re

[HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On Wed, Jun 17, 2015 at 6:58 AM, Alvaro Herrera wrote: > Thomas Munro wrote: > >> Thanks. As mentioned elsewhere in the thread, I discovered that the >> same problem exists for page boundaries, with a different error >> message. I've tried the attached repro s

[HACKERS] Inheritance planner CPU and memory usage change since 9.3.2

e for some users after an upgrade. [1] http://www.postgresql.org/message-id/8c9acaa.1f453.14c0da0402f.coremail.chjis...@163.com [2] http://www.postgresql.org/message-id/flat/20141107185824.2513.53...@wrigleys.postgresql.org#20141107185824.2513.53...@wrigleys.postgresql.org -- Thomas Munro

Re: [HACKERS] dblink: add polymorphic functions - review

have to subscribe first so that you can log into it. If you start here and browse to the message in the archive, you'll see a 'Mail this message to ...' link at the bottom: https://lists.postgresql.org/mj/mj_wwwusr/domain=postgresql.org -- Thomas Munro http://www.enterprisedb.c

Re: [HACKERS] Inheritance planner CPU and memory usage change since 9.3.2

141 Q4 30 30 93 1023 1023 992 30751 Q4 500 500 1503 252003 252003 251502 126002501 -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] get_relation_info comment out of sync

Hi The comment for get_relation_info should probably include serverid in the list of rel members that it can update (see attached). -- Thomas Munro http://www.enterprisedb.com get_relation_info-comment.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Solaris testers wanted for strxfrm() behavior

On Mon, Jun 29, 2015 at 7:58 AM, Josh Berkus wrote: > My perspective is that if both SmartOS and OmniOS pass, it's not our > responsibility to support OldSolaris if they won't update libraries. Just by the way, I wonder if this was that bug: https://illumos.org/issues/1594

Re: [HACKERS] Solaris testers wanted for strxfrm() behavior

On Mon, Jun 29, 2015 at 10:57 AM, Tom Lane wrote: > Thomas Munro writes: >> Just by the way, I wonder if this was that bug: >> https://illumos.org/issues/1594 > > Oooh. Might or might not be *same* bug, but it sure looks like it could > have the right symptom. If this

Re: [HACKERS] Rework the way multixact truncations work

5b2f0fe3827fdf3d5a I think saw some extra autovacuum activity that I didn't expect in a simple scenario, but I'm not sure and will continue looking tomorrow. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

abort it. +1 I was thinking recently that it would be very cool to see some estimation of the progress of VACUUM and CLUSTER in a view similar to pg_stat_activity, or the ps title. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

[HACKERS] WAL test/verification tool

Hi I have heard rumours of a tool that could verify or compare the effects of applying WAL records for testing/development purposes, but I've been unable to track it down or find out if it was publicly released. Does anyone know the status of that or what it was called? Thanks, -- T

[HACKERS] Typo in comment in nbtree.h

ray */ + BlockNumber currPage; /* page referenced by items array */ BlockNumber nextPage; /* page's right link when we scanned it */ /* -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Tracking wait event for latches

very useful for extensions to be able to name their wait points. For example, I'd rather see 'postgres_fdw.pgfdw_get_result' or similar than a vague 'Extension' string which appears not only for all wait points in an extension but also for all extensions. I hope we can fig

Re: [HACKERS] kqueue

533ac22, which sprinkled some MAXALIGN macros nearby. I've now done the same thing with the kevent struct because it's cheap, uniform with the other cases and could matter on some platforms for the same reason. It's in the September commitfest here: https://commitfest.postgresql.org/10/

[HACKERS] Reference to UT1

ezone-names.html [2] https://en.wikipedia.org/wiki/Unix_time -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

ession not found in targetlist -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

allel_workers_per_gather parameter. > The result of Q1 is bellow. Is this bug in the Open items on wiki? I don't see it on the Open Issues list. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscri

Re: [HACKERS] Reviewing freeze map code

and that we called record_corrupt_item because VM_ALL_VISIBLE returned true but HeapTupleSatisfiesVacuum on the first tuple returned HEAPTUPLE_DELETE_IN_PROGRESS instead of the expected HEAPTUPLE_LIVE. It did that because HEAP_XMAX_COMMITTED was not set and TransactionIdIsInProgress returned true for

Re: [HACKERS] Reviewing freeze map code

need to protect xmax in the > same way. With this version of the patch, I can no longer get any > TIDs to pop up out of pg_check_visible in my testing. (I haven't run > your test script for lack of the proper Python environment...) I can still reproduce the problem with this

Re: [HACKERS] Reviewing freeze map code

On Wed, Jun 15, 2016 at 11:43 AM, Thomas Munro wrote: > On Wed, Jun 15, 2016 at 12:44 AM, Robert Haas wrote: >> On Tue, Jun 14, 2016 at 8:11 AM, Robert Haas wrote: >>>>> I noticed that the tuples that it reported were always offset 1 in a >>>>> page, and t

Re: [HACKERS] Typo in parallel comment of heap_delete()

l form where I would expect a present subjunctive: "... lest the new path *would* kick ..." /me ducks -- Thomas Munro http://www.enterprisedb.com lest.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

[HACKERS] Assert(LWLockHeldByMeInMode(lock, LW_EXCLUSIVE))

o the next commitfest. -- Thomas Munro http://www.enterprisedb.com bufmgr-assert-lwlock-mode.patch Description: Binary data test-lwlock-mode.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Assert(LWLockHeldByMeInMode(lock, LW_EXCLUSIVE))

On Mon, Jun 20, 2016 at 3:43 PM, Craig Ringer wrote: > On 18 June 2016 at 11:28, Thomas Munro > wrote: >> Several times now when reading, debugging and writing code I've wished >> that LWLockHeldByMe assertions specified the expected mode, especially >> where

Re: [HACKERS] Reviewing freeze map code

commit f2bfe8a2 said that tqual routines would see the HEAP_XMAX_UNLOGGED flag in the event of a crash before logging (though I'm not sure if the tqual routines ever actually did that). -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Reviewing freeze map code

origin page. Then after locking the origin page, if it turns out you need a page but didn't get it earlier, asking for a free page with a higher block number than the origin page. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reviewing freeze map code

a badly timed crash was ever implemented. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [sqlsmith] crashes in RestoreSnapshot on hot standby

thing like the attached. Theory 2: The DSM segment was deleted underneath us. We can see that it was not mapped by the time GDB dumped that (start_address is not accessible). Theory 3: Somehow the xcnt or xsubcnt was wrong or the serialized snapshot was truncated, and we read past the end of

Re: [HACKERS] [sqlsmith] crashes in RestoreSnapshot on hot standby

On Fri, Jul 1, 2016 at 3:25 PM, Amit Kapila wrote: > On Fri, Jul 1, 2016 at 8:48 AM, Thomas Munro > wrote: >> If serialized_snapshot->xcnt == 0, then snapshot->xip never gets >> initialized to a non-NULL value. Then if serialized_snapshot->subxcnt >> > 0, we

Re: [HACKERS] Forthcoming SQL standards about JSON and Multi-Dimensional Arrays (FYI)

eferred back to Working Group". Is that how they say "returned with feedback"? ISO/IEC PDTR 19075-5 (Row Pattern Recognition) has also reached stage 30.60. Does anyone know what that one is about? Maybe something like MATCH_RECOGNIZE in Oracle? [1] http://www.iso.org/iso/home/s

Re: [HACKERS] SELECT DISTINCT never uses an index?

mail.com I hope to return to that and some related ideas eventually as I learn more about the relevant areas of the source code, if someone doesn't beat me to it. https://wiki.postgresql.org/wiki/Loose_indexscan shows a recursive CTE that does the same thing at a higher level. -- Thomas Munro

[HACKERS] LWLocks in DSM memory

sage-id/ca+tgmobjia49ccj0zazbwavv7nkgyt+1zo5cwxkh9aahgn2...@mail.gmail.com -- Thomas Munro http://www.enterprisedb.com lwlocks-in-dsm.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] LWLocks in DSM memory

On Sun, Jul 24, 2016 at 1:10 AM, Thomas Munro wrote: > One solution could be to provide a non-circular variant of the dlist > interface that uses NULL list termination. I've attached a quick > sketch of something like that which seems to work correctly. It is > only lightly

Re: [HACKERS] LWLocks in DSM memory

On Mon, Jul 25, 2016 at 3:02 PM, Thomas Munro wrote: > I measured the following times for unpatched master, on my 4 core laptop: > > 16 workers = 73.067s, 74.869s, 75.338s > 8 workers = 65.846s, 67.622s, 68.039s > 4 workers = 68.763s, 68.980s, 69.035s <-- curiously slowe

Re: [HACKERS] Proposal: revert behavior of IS NULL on row types

ays, but in fact our NOT NULL constraints are equivalent to CHECK (column_name IS DISTINCT FROM NULL). Should we update the documentation with something like the attached? -- Thomas Munro http://www.enterprisedb.com not-null-does-not-mean-check-is-not-null.patch Description: Binary data -- Se

Re: [HACKERS] LWLocks in DSM memory

might require a wider CAS than we have?). I wonder if a proclist with an optional lock-free interface would also be interesting for syncrep queues and others. -- Thomas Munro http://www.enterprisedb.com lwlocks-in-dsm-v2.patch Description: Binary data -- Sent via pgsql-hackers ma

Re: [HACKERS] LWLocks in DSM memory

On Fri, Jul 29, 2016 at 2:12 AM, Robert Haas wrote: > On Thu, Jul 28, 2016 at 12:12 AM, Thomas Munro > wrote: >> On Tue, Jul 26, 2016 at 6:00 AM, Andres Freund wrote: >>> I think the better fix here would acutally be to get rid of a pointer >>> based list here,

[HACKERS] Double invocation of InitPostmasterChild in bgworker with -DEXEC_BACKEND

d is called twice. I can successfully use regular parallel query workers and bgworkers created by extensions if I apply the attached patch. -- Thomas Munro http://www.enterprisedb.com fix.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

[HACKERS] Combining hash values

knuth-derive-a [4] http://community.haskell.org/~simonmar/base/src/Data-HashTable.html -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Combining hash values

On Mon, Aug 1, 2016 at 4:34 AM, Tom Lane wrote: > Thomas Munro writes: >> 2. I suspect that this algorithm for combining hashes is weak, and >> could amplify weaknesses in the hash functions feeding it. > > Very possibly, but ... Concrete example: suppose a clever data

Re: [HACKERS] PostmasterContext survives into parallel workers!?

1_handler. That's likely excessive, but it's what > we've got at the moment. I found this apparently unresolved bug report about glibc fork() inside a signal handler deadlocking: https://sourceware.org/bugzilla/show_bug.cgi?id=4737 I wonder if that could bite postmaster. It's

Re: [HACKERS] Implementing full UTF-8 support (aka supporting 0x00)

https://wiki.postgresql.org/wiki/Todo:ICU ucoll_strcoll takes explicit lengths (though optionally accepts -1 for null terminated mode). http://userguide.icu-project.org/strings#TOC-Using-C-Strings:-NUL-Terminated-vs.-Length-Parameters -- Thomas Munro http://www.enterprisedb.com -- Sent via

Re: [HACKERS] [sqlsmith] FailedAssertion("!(k == indices_count)", File: "tsvector_op.c", Line: 511)

* to the same index. */ - if (k < indices_count && i == indices_to_delete[k]) + while (k < indices_count && i == indices_to_delete[k]) { + drop_lexeme = true; k++; - continue

[HACKERS] Consolidate 'unique array values' logic into a reusable function?

posted a while back that consolidates popcount and ffs/fls implementations. I don't like code duplication :-) -- Thomas Munro http://www.enterprisedb.com array-unique.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes t

[HACKERS] dsm_unpin_segment

to keep the segment alive. This patch needs to close that handle when unpinning. Amazingly, that can be done without any cooperation from the postmaster. I'd be grateful for any feedback or thoughts, and will add this to the commitfest. -- Thomas Munro http://www.enterprisedb.com dsm-unpin-seg

Re: [HACKERS] dsm_unpin_segment

On Tue, Aug 9, 2016 at 12:22 PM, Robert Haas wrote: > On Mon, Aug 8, 2016 at 7:33 PM, Tom Lane wrote: >> Thomas Munro writes: >>> Please find attached a patch to add a corresponding operation >>> 'dsm_unpin_segment'. This gives you a way to ask for the

Re: [HACKERS] dsm_unpin_segment

f the existing cleanup mechanism for the control segment, while making sure that the auxiliary segments get pinned and unpinned exactly once. I'll have more to say about that when I post that patch... -- Thomas Munro http://www.enterprisedb.com dsm-unpin-segment-v2.patch Descr

Re: [HACKERS] dsm_unpin_segment

ot pinned"); Those checks could arguably be assertions rather than errors, but I don't think that pin/unpin operations will ever be high frequency enough for it to be worth avoiding those instructions in production builds. The whole reason for pinning segments is likely to be able to reu

Re: [HACKERS] dsm_unpin_segment

On Wed, Aug 10, 2016 at 2:43 PM, Jim Nasby wrote: > On 8/9/16 6:14 PM, Thomas Munro wrote: >> The can't be static, they need to be in shared memory, because we also >> want to protect against two *different* backends pinning it. > > Right, this would strictly protect

Re: [HACKERS] condition variables

do similar extra work on top for my join points concept, because although I do need waiters to be poked at the time worker aborts or dies, one goodbye prod isn't enough: I'd also need to adjust the join point's set of workers, or put it into error state. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

t looks like if we used fallocate or posix_fallocate in the dsm_impl_posix case we'd get a nice ESPC error, instead of success-but-later-SIGBUS-on-access. Whether there is *also* the possibility of overcommit biting you later I don't know, but I suspect that's an independent pr

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

On Sat, Aug 13, 2016 at 8:26 AM, Thomas Munro wrote: > On Sat, Aug 13, 2016 at 2:08 AM, Tom Lane wrote: >> amul sul writes: >>> When I am calling dsm_create on Linux using the POSIX DSM implementation >>> can succeed, but result in SIGBUS when later try to access the

Re: [HACKERS] condition variables

On Fri, Aug 12, 2016 at 9:47 AM, Robert Haas wrote: > [condition-variable-v1.patch] Don't you need to set proc->cvSleeping = false in ConditionVariableSignal? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

[HACKERS] WIP: Barriers

ostgresql.org/message-id/flat/CAEepm%3D0Vvr9zgwHt67RwuTfwMEby1GiGptBk3xFPDbbgEtZgMg%40mail.gmail.com [3] https://www.postgresql.org/message-id/flat/cab7npqtghfouhag1ejrvskn8-e5fpqvhm7al0tafsdzjqg_...@mail.gmail.com -- Thomas Munro http://www.enterprisedb.com barrier-v1.patch Description: Binary d

Re: [HACKERS] condition variables

On Sun, Aug 14, 2016 at 9:04 AM, Thomas Munro wrote: > On Fri, Aug 12, 2016 at 9:47 AM, Robert Haas wrote: >> [condition-variable-v1.patch] > > Don't you need to set proc->cvSleeping = false in ConditionVariableSignal? I poked at this a bit... OK, a lot... and have some

Re: [HACKERS] condition variables

On Mon, Aug 15, 2016 at 5:58 PM, Thomas Munro wrote: > Also, I have attached a v2->v3 diff ... Ugh. I meant a v1->v2 diff. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript

Re: [HACKERS] WIP: Barriers

. I realise that using signals for this sort of thing is a bit unusual outside the Postgres universe, but won't a semaphore-based implementation require just as many system calls, context switches and scheduling operations? -- Thomas Munro http://www.enterprisedb.com barrier-test.patc

Re: [HACKERS] Anyone want to update our Windows timezone map?

of thing: http://www.unicode.org/cldr/charts/29/supplemental/zone_tzid.html Could that be a better source than dumping stuff from arbitrary Windows versions? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

On Wed, Aug 17, 2016 at 4:50 AM, Robert Haas wrote: > On Fri, Aug 12, 2016 at 9:22 PM, Thomas Munro > wrote: >> On Sat, Aug 13, 2016 at 8:26 AM, Thomas Munro >> wrote: >>> On Sat, Aug 13, 2016 at 2:08 AM, Tom Lane wrote: >>>> amul sul writes: >>&g

Re: [HACKERS] support for NEXT VALUE FOR expression

ttps://www.postgresql.org/message-id/flat/CADLWmXUY2oo4XObQWF3yPUSK%3D5uEiSV%3DeTyLrnu%3DRzteOy%2B3Lg%40mail.gmail.com > I remain of the opinion that using spec-compliant syntax for > non-spec-compliant behavior isn't a great advance. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers

[HACKERS] NOTIFY with tuples

to the source tree, but I thought I'd present this idea and see if it's a duplicate effort, or someone has a much better idea, or it is instantly shot down in flames for technical or other reasons, before investing further in it. Thanks! Thomas Munro [1] http://en.wikipedia.org/wiki/St

Re: [HACKERS] NOTIFY with tuples

On 14 December 2011 04:21, Tom Lane wrote: > Robert Haas writes: >> On Tue, Dec 13, 2011 at 6:30 PM, Thomas Munro wrote: >>> I imagine a very simple system like this, somehow built on top of >>> the existing NOTIFY infrastructure: > >> I'm not sure whet

Re: [HACKERS] NOTIFY with tuples

On 14 December 2011 15:10, Merlin Moncure wrote: > As to the wider point I'm wondering why you can't layer your API on > top of existing facilities (tables, notifications, etc). PGQ (have you > seen that?) does this and it's an absolute marvel.  Meaning, I bet you > could do this with an 'all sql

[HACKERS] WIP -- renaming implicit sequences

s if the user doesn't have appropriate privileges. Useful? Why would anyone not want this behaviour? Have I used inappropriate locking levels? What should I read to understand the rules of locking? Have I failed to handle errors? Have I made memory ownership mistakes? Thanks! Thomas

Re: [HACKERS] WIP -- renaming implicit sequences

On 12 January 2012 00:58, Tom Lane wrote: > Thomas Munro writes: >> Here is an unfinished patch to implement something which appears on >> the TODO list under ALTER: automatic renaming of sequences created >> with serial when the table and column names change.  I've of

Re: [HACKERS] Multithread Query Planner

On 13 January 2012 20:14, Frederico wrote: > I'm trying to develop a multithread planner, and some times is raised a > exception of access memory. I was a bit confused about what you are trying to do -- somehow use concurrency during the planning phase, or during execution (maybe having produced

Re: [HACKERS] WIP -- renaming implicit sequences

On 12 January 2012 00:58, Tom Lane wrote: > Hmm ... this seems a bit inconsistent with the fact that we got rid of > automatic renaming of indexes a year or three back.  Won't renaming of > serials have all the same problems that caused us to give up on renaming > indexes? I was sort of planning

[HACKERS] SKIP LOCKED DATA

ting the new case HeapTupleWouldBlock the same way that HeapTupleSelfUpdated is treated -- that is, goto lnext to fetch the next tuple. 4. Probably some changes to handle table-level locks too. 5. Probably many other things that I'm not aware of right now and won't discover until I dig/ask further and/o

Re: [HACKERS] SKIP LOCKED DATA

On 16 January 2012 08:06, Ilya Kosmodemiansky wrote: > That is quite useful feature to implement smth. like message queues > based on database and so on. > Now there is possibility to jump over luck of such feature in Postgres > using current advisory lock implementation (pg_try_advisory_xact_lock

[HACKERS] Cluster name in ps output

Hi When running more than one cluster I often find myself looking at the output of 'iotop' or other tools wondering which cluster's "wal receiver process" or "checkpointer process" etc I'm seeing. Obviously it's easy enough to find out (for example by looking at a tree view in htop/ps that shows

Re: [HACKERS] Cluster name in ps output

On 5 May 2014 10:10, Andres Freund wrote: > Hi, > > On 2014-05-05 10:00:34 +0000, Thomas Munro wrote: > > When running more than one cluster I often find myself looking at > > the output of 'iotop' or other tools wondering which > > cluster's "wal

Re: [HACKERS] Cluster name in ps output

On 5 May 2014 10:49, Thomas Munro wrote: > On 5 May 2014 10:10, Andres Freund wrote: > >> I guess the question is where this should be available as well. At the >> very least I'd want to reference it in log_line_prefix as well? >> > > Good idea, I will look

Re: [HACKERS] Cluster name in ps output

ht either) * sample conf file updated to show cluster_name and %C in log_line_prefix A shorter version without the log_line_prefix support that Tom didn't like is attached as cluster-name-in-ps-v3-b.patch. I will try to add these to the open commitfest, and see if there is something I can useful

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

a bikeshed colour point of view: * I used SKIP LOCKED DATA like DB2, and Simon used SKIP LOCKED like Oracle, and I guess shorter is sweeter * I used the term wait_policy and an enum, Simon used waitMode and an int * I had noWait and skipLocked travelling separately in some places, Simon had a single parameter, which is much better Best regards, Thomas Munro

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

| 22.34 | 6.77 | I am not experienced at benchmarking and I don't claim that this particular workload or configuration is particularly sensible or representative of anything but it might give some idea of the motivation. Best regards, Thomas Munro

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

DISCARD Of those I think I prefer NOWAIT EXCLUDE (perhaps with NOWAIT ABORT as a long version of the existing NOWAIT behaviour for contrast). Or adding just one new keyword: NOWAIT SKIP SKIP LOCK Regards, Thomas Munro

<    4   5   6   7   8   9