[HACKERS] Re: [COMMITTERS] pgsql: Do all-visible handling in lazy_vacuum_page() outside its critic

2014-06-23 Thread Jeff Davis
_CLEAN action, and it would look a lot more like setting a tuple hint. Also, we might be able to simplify visibilitymap_set if the logging of the heap page for checksums could be done beforehand with this variant of MarkBufferDirtyHint. Regards, Jeff Davis [1] Not that we didn't make

[HACKERS] Re: [COMMITTERS] pgsql: Do all-visible handling in lazy_vacuum_page() outside its critic

2014-06-26 Thread Jeff Davis
g a HEAP2_VISIBLE there is that removing the tuples in the _CLEAN > step increases the chance of it the page now being all visible? It's > separate steps. When setting all-visible was part of the same critical section doing the logging for HEAP2_CLEAN, it seemed to be

Re: [HACKERS] Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

2014-07-06 Thread Jeff Davis
you think I am making things worse with my refactorings. Otherwise I'll keep working on this and hopefully get it committable soon. The attached patch is still a WIP; just posting it here in case you see any red flags. Regards, Jeff Davis *** a/doc/src/sgml/func.sgml --- b/doc/src/sgml

Re: [HACKERS] Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

2014-07-07 Thread Jeff Davis
On Sun, 2014-07-06 at 21:11 -0700, Jeff Davis wrote: > On Wed, 2014-04-16 at 12:50 +0100, Nicholas White wrote: > > Thanks for the detailed feedback, I'm sorry it took so long to > > incorporate it. I've attached the latest version of the patch, fixing > > in par

[HACKERS] Allow multi-byte characters as escape in SIMILAR TO and SUBSTRING

2014-07-10 Thread Jeff Davis
Attached is a small patch to $SUBJECT. In master, only single-byte characters are allowed as an escape. Of course, with the patch it must still be a single character, but it may be multi-byte. Regards, Jeff Davis *** a/src/backend/utils/adt/regexp.c --- b/src/backend/utils/adt/regexp.c

Re: [HACKERS] Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

2014-07-10 Thread Jeff Davis
On Mon, 2014-07-07 at 01:21 -0700, Jeff Davis wrote: > On Sun, 2014-07-06 at 21:11 -0700, Jeff Davis wrote: > > On Wed, 2014-04-16 at 12:50 +0100, Nicholas White wrote: > > > Thanks for the detailed feedback, I'm sorry it took so long to > > > incorporate it. I&#

Re: [HACKERS] Allow multi-byte characters as escape in SIMILAR TO and SUBSTRING

2014-07-11 Thread Jeff Davis
But I think that > it's better to document what character is allowed as escape in LIKE, > SIMILAR TO and SUBSTRING. It should be assumed that multi-byte characters are allowed nearly everywhere, and we should document the places where only single-byte characters are allowed. Regards,

Re: [HACKERS] Allow multi-byte characters as escape in SIMILAR TO and SUBSTRING

2014-07-11 Thread Jeff Davis
On Fri, 2014-07-11 at 11:51 -0400, Tom Lane wrote: > Jeff Davis writes: > > Attached is a small patch to $SUBJECT. > > In master, only single-byte characters are allowed as an escape. Of > > course, with the patch it must still be a single character, but it may > &

Re: [HACKERS] Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

2014-07-21 Thread Jeff Davis
On Thu, 2014-07-10 at 23:43 -0700, Jeff Davis wrote: > On Mon, 2014-07-07 at 01:21 -0700, Jeff Davis wrote: > > On Sun, 2014-07-06 at 21:11 -0700, Jeff Davis wrote: > > > On Wed, 2014-04-16 at 12:50 +0100, Nicholas White wrote: > > > > Thanks for the detailed feedback

[HACKERS] numeric and float comparison oddities

2014-07-31 Thread Jeff Davis
g +/- infinity to numeric so that it can represent every float value? That might make the numeric hierarchy a little cleaner and less surprising. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2014-08-02 Thread Jeff Davis
27;s easier and more accurate. I did some simple pgbench select-only tests, and I didn't see any TPS difference. Regards, Jeff Davis *** a/src/backend/utils/mmgr/aset.c --- b/src/backend/utils/mmgr/aset.c *** *** 242,247 typedef struct AllocChunkData --- 24

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2014-08-08 Thread Jeff Davis
nd 5.3s on both master and memory-accounting. Either way, it's better to be conservative. Attached is a version of the patch with opt-in memory usage tracking. Child contexts inherit the setting from their parent. Regards, Jeff Davis *** a/src/backend/utils/mmgr/aset.c --- b/src/back

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2014-08-10 Thread Jeff Davis
On Fri, 2014-08-08 at 01:16 -0700, Jeff Davis wrote: > Either way, it's better to be conservative. Attached is a version of the > patch with opt-in memory usage tracking. Child contexts inherit the > setting from their parent. There was a problem with the previous patch: the paren

[HACKERS] 9.5: Memory-bounded HashAgg

2014-08-10 Thread Jeff Davis
* performance testing Initial tests indicate that it can be competitive with sort+groupagg when the disk is involved, but more testing is required. Feedback welcome. Regards, Jeff Davis *** a/doc/src/sgml/config.sgml --- b/doc/src/sgml/config.sgml *** *** 2884

Re: [HACKERS] 9.5: Memory-bounded HashAgg

2014-08-11 Thread Jeff Davis
On Mon, 2014-08-11 at 01:29 +0200, Tomas Vondra wrote: > On 10.8.2014 23:26, Jeff Davis wrote: > > This patch is requires the Memory Accounting patch, or something > > similar to track memory usage. > > I think the patch you sent actually includes the accounting patch. Is >

Re: [HACKERS] 9.5: Memory-bounded HashAgg

2014-08-12 Thread Jeff Davis
to be separate hash tables for different aggregates, but it seems like it could work. >(b) bad estimate of required memory -> this is common for aggregates >passing 'internal' state (planner uses some quite high defaults) Maybe some planner hooks? Ideas? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5: Memory-bounded HashAgg

2014-08-14 Thread Jeff Davis
ght? Is there any reason not to just double the number of > batches globally? I didn't quite follow this proposal. > It also seems to me that using HASH_DISK_MAX_PARTITIONS, and then allowing > each work item to create it's own set of additional partitions effectively > renders the HA

Re: [HACKERS] 9.5: Memory-bounded HashAgg

2014-08-14 Thread Jeff Davis
; when you get to that partition. I don't see that there's any special case > here. HashJoin only deals with tuples. With HashAgg, you have to deal with a mix of tuples and partially-computed aggregate state values. Not impossible, but it is a little more awkward than HashJoin

Re: [HACKERS] 9.5: Memory-bounded HashAgg

2014-08-14 Thread Jeff Davis
ointing out that (A) is O(N) and (B) is O(log2(N)). But I suspect that cost (A) is very low. Unfortunately, it would take some effort to test your approach because we'd actually need a way to write out the partially-computed state, and the algorithm itself seems a little more complex.

Re: [HACKERS] 9.5: Memory-bounded HashAgg

2014-08-14 Thread Jeff Davis
e hash table fills up, in which case HashAgg is just as bad as Sort.) That being said, we can hold out for an array_agg fix if desired. As I pointed out in another email, my proposal is compatible with the idea of dumping groups out of the hash table, and does take some steps in that directio

Re: [HACKERS] 9.5: Memory-bounded HashAgg

2014-08-19 Thread Jeff Davis
ch is (potentially) commit-worthy, and your statement that it (potentially) solves a real problem is a big help. Regards, Jeff Davis [1] http://blogs.msdn.com/b/craigfr/archive/2008/01/18/partial-aggregation.aspx -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2014-08-19 Thread Jeff Davis
s like it might be an important threshold. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2014-08-19 Thread Jeff Davis
with the idea of tracking space for an entire hierarchy. Also, as I pointed out in my reply to Robert, adding too many fields to MemoryContextData may be the cause of the regression. Your idea requires only one field, which doesn't show the same regression in my tests. Regards, Jeff

[HACKERS] New expression evaluator and indirect jumps

2017-04-01 Thread Jeff Davis
d the assembly we wanted rather than how much it actually improved performance. Can someone please point me to the numbers? Do they refute the conclusions in the paper, or are we concerned about a wider range of processors? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-ha

[HACKERS] Range Merge Join v1

2017-04-06 Thread Jeff Davis
ow (though more investigation might be useful here). Also, it doesn't provide any alternative to the nestloop-with-inner-index we already offer at the leaf level today. Regards, Jeff Davis diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c index a18ab43..1110c1e

Re: [HACKERS] Range Merge Join v1

2017-04-11 Thread Jeff Davis
en if the input relations are subqueries. Regards, Jeff Davis On Thu, Apr 6, 2017 at 1:43 AM, Jeff Davis wrote: > > Example: > > > Find different people using the same website at the same time: > > create table session(sessionid text, username text, duri

Re: [HACKERS] Range Merge Join v1

2017-04-11 Thread Jeff Davis
On Tue, Apr 11, 2017 at 12:17 AM, Jeff Davis wrote: > Version 2 attached. Fixed a few issues, expanded tests, added docs. It looks like the CF app only listed my perf test script. Re-attaching rangejoin-v2.patch so that it appears in the CF app. Identical to other rangejoin-v2.patch. Rega

Re: [HACKERS] Merge join for GiST

2017-04-11 Thread Jeff Davis
ome experimental evaluation here: > http://www.adass2016.inaf.it/images/presentations/10_Korotkov.pdf Do you have a sense of how this might compare with range merge join? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to y

Re: [HACKERS] Merge join for GiST

2017-04-12 Thread Jeff Davis
On Tue, Apr 11, 2017 at 8:35 AM, Alexander Korotkov wrote: > On Tue, Apr 11, 2017 at 5:46 PM, Jeff Davis wrote: >> Do you have a sense of how this might compare with range merge join? > > > If you have GiST indexes over ranges for both sides of join, then this > method co

Re: [HACKERS] Merge join for GiST

2017-04-12 Thread Jeff Davis
rees are faster. I don't quite follow. I don't think any of these proposals uses btree, right? Range merge join doesn't need any index, your proposal uses gist, and PgSphere's crossmatch uses gist. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range Merge Join v1

2017-04-20 Thread Jeff Davis
use in this > function) >&op_strategy, > Looks like filterdiff destroyed my patch from git. Attaching unified version against master 3820c63d. Thanks! Jeff Davis diff --git a/doc/src/sgml/rangetypes.sgml b/doc/src/sgml/rangetypes.sgml index 9557c16..84578a7 100644 --- a/doc

Re: [HACKERS] [POC] hash partitioning

2017-05-02 Thread Jeff Davis
th better push-downs, which will be good for parallel query. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] hash partitioning

2017-05-03 Thread Jeff Davis
On Tue, May 2, 2017 at 7:01 PM, Robert Haas wrote: > On Tue, May 2, 2017 at 9:01 PM, Jeff Davis wrote: >> 1. Consider a partition-wise join of two hash-partitioned tables. If >> that's a hash join, and we just use the hash opclass, we immediately >> lose some useful

[HACKERS] Hash Functions

2017-05-11 Thread Jeff Davis
very important. If we run out of bits, we can just salt the hash function differently and get more hash bits. This is not urgent and I believe we should just implement salts when and if some algorithm needs them. Regards, Jeff Davis [1] You can a kind of mirroring in the hash outputs indicati

Re: [HACKERS] Hash Functions

2017-05-13 Thread Jeff Davis
ard currently. But hash partitioning is too valuable to give up on entirely. I think we should consider supporting a limited subset of types for now with something not based on the hash am. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Hash Functions

2017-05-13 Thread Jeff Davis
ot always desirable for parallelism. Hash partitioning doesn't have these issues and goes very nicely with parallel query. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Functions

2017-05-13 Thread Jeff Davis
/message-id/CAMp0ubfNMSGRvZh7N7TRzHHN5tz0ZeFP13Aq3sv6b0H37fdcPg%40mail.gmail.com Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Functions

2017-05-15 Thread Jeff Davis
ybe hash partitions are really a "semi-logical" partitioning that the optimizer understands, but where things like per-partition check constraints don't make sense. Regards, Jeff Davis Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] Hash Functions

2017-05-15 Thread Jeff Davis
oot as Andres and others suggested, and disable a lot of logical partitioning capabilities. I'd be a little worried about what we do with attaching/detaching, though. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Functions

2017-05-16 Thread Jeff Davis
7;t surprised, I think users will understand why these aren't quite the same concepts. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Functions

2017-05-16 Thread Jeff Davis
ould take care of the naming problem. > If Java has portable hash functions, why can't we? Java standardizes on a particular unicode encoding (utf-16). Are you suggesting that we do the same? Or is there another solution that I am missing? Regards, Jeff Davis

Re: [HACKERS] Hash Functions

2017-05-17 Thread Jeff Davis
x27;t mean that we should necessarily forbid them, but it should make us question whether combining range and hash partitions is really the right design. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Functions

2017-05-17 Thread Jeff Davis
. partitioning on date) * both offer some maintenance benefits (e.g. reindex one partition at a time), though range partitioning seems like it offers better flexibility here in some cases I lean toward separating the concepts, but Robert is making some reasonable arguments and I could be convinced.

[HACKERS] Hash Functions

2017-05-18 Thread Jeff Davis
releases later we make the typical cases work out of the box. I'm fine with it as long as we don't paint ourselves into a corner. Of course we still have work to do on the hash functions. We should solve at least the most glaring portability problems, and try to harmonize the hash opfamilies. If you agree, I can put together a patch or two. Regards, Jeff Davis

Re: [HACKERS] Hash Functions

2017-06-01 Thread Jeff Davis
fine with either option. > 2. Add an option like --dump-partition-data-with-parent. I'm not sure > who originally proposed this, but it seems that everybody likes it. > What we disagree about is the degree to which it's sufficient. Jeff > Davis thinks it doesn't go

Re: [HACKERS] Hash Functions

2017-06-01 Thread Jeff Davis
o the latter. Once we support more pushdowns to partitions, the only question is: what are your join keys and what are your grouping keys? Text is absolutely a normal join key or group key. Consider joins on a user ID or grouping by a model number. Regards, Jeff Davis -- Sent via pgsq

Re: [HACKERS] Range Merge Join v1

2017-06-02 Thread Jeff Davis
). * Better integration with the catalog so that users could add their own types that support range merge join. Thank you for the review. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range Merge Join v1

2017-08-31 Thread Jeff Davis
m as suggested, but it has 4 possible values rather than 3. * Added support for joining on contains or contained by (@> or <@) and updated tests. Regards, Jeff Davis diff --git a/doc/src/sgml/rangetypes.sgml b/doc/src/sgml/rangetypes.sgml index 9557c16..84578a7 100644 *** a/doc/src/sgml/r

Re: [HACKERS] Range Merge Join v1

2017-09-17 Thread Jeff Davis
On Thu, Aug 31, 2017 at 1:52 AM, Jeff Davis wrote: > Updated patch attached. Changelog: > > * Rebased > * Changed MJCompare to return an enum as suggested, but it has 4 > possible values rather than 3. > * Added support for joining on contains or contained by (@> or <@) a

Re: [HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-09-07 Thread Jeff Davis
ugh nothing is free, the cost seems very low, and at least three people have expressed interest in this patch. What tips the balance is that we expose the unix epoch in the pgbench logs, as Tomas points out. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-09-07 Thread Jeff Davis
On Mon, 2015-09-07 at 17:47 -0300, Alvaro Herrera wrote: > Jeff Davis wrote: > > On Sun, 2015-03-22 at 19:47 +0100, Andres Freund wrote: > > > On 2015-03-22 00:47:12 +0100, Tomas Vondra wrote: > > > > from time to time I need to correlate PostgreSQL logs to other lo

Re: [HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-09-07 Thread Jeff Davis
;m' format) and once for the regular log ('m', 'n', or 't'). If the regular log uses 'm', that would be some wasted cycles formatting it the same way twice. Is it worth a little extra ugliness to cache both the timeval and the formatted string? Regards,

Re: [HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-09-07 Thread Jeff Davis
hed. Please take a quick look. Regards, Jeff Davis *** a/src/backend/utils/error/elog.c --- b/src/backend/utils/error/elog.c *** *** 143,152 static int errordata_stack_depth = -1; /* index of topmost active frame */ static int recursion_depth = 0; /* to detect

Re: [HACKERS] PQputCopyData dont signal error

2014-02-04 Thread Jeff Davis
d, because then the client needs to figure out the line breaks, which is a burden in many cases. Certainly we don't want to *guarantee* that the backend will issue an error at any particular point, because of the buffering on the server side. But from a practical standpoint, the server will let the cl

Re: [HACKERS] Freezing without write I/O

2013-09-02 Thread Jeff Davis
else, that might help debug this patch, if necessary. There might also be cases where we can freeze more eagerly to avoid the case where very old (but unfrozen) and very new tuples mix on the same page. Perhaps Robert has some thoughts here, as well. Regards, Jeff Davis -- Sent via

Re: [HACKERS] ENABLE/DISABLE CONSTRAINT NAME

2013-09-02 Thread Jeff Davis
n inserting data must follow a certain order. >And people usually like to insert the data but not affected by > foreign keys or check. Is there any semantic difference between marking a constraint as DISABLED and simply dropping it? Or does it just make it easier to re-add it later?

[HACKERS] [9.4] Make full_page_writes only settable on server start?

2013-09-02 Thread Jeff Davis
PGC_POSTMASTER. Any reason to keep the setting as a PGC_SIGHUP? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [9.4] Make full_page_writes only settable on server start?

2013-09-04 Thread Jeff Davis
. That means, if checksums are enabled and full_page_writes is off, we'll still write some full page images for checksums. I'd like to remedy that. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.4 regression

2013-09-04 Thread Jeff Davis
;d better do more before deciding what to do. Greg Smith, Do you have some thoughts about this? More tests we should run? We may need to revert the patch. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [9.4] Make full_page_writes only settable on server start?

2013-09-04 Thread Jeff Davis
On Wed, 2013-09-04 at 11:32 -0400, Tom Lane wrote: > Jeff Davis writes: > > I think code complexity matters quite a lot. If we can eliminate some > > complex code in a complex area, and all we give up is a feature with > > essentially no use case, that sounds like we&#

Re: [HACKERS] [9.4] Make full_page_writes only settable on server start?

2013-09-04 Thread Jeff Davis
On Wed, 2013-09-04 at 17:00 +0200, Andres Freund wrote: > On 2013-09-04 07:57:15 -0700, Jeff Davis wrote: > > XLogSaveBufferForHint() calls XLogCheckBuffer() but doesn't also look at > > the full page writes setting (like in XLogInsert()). That means, if > >

[HACKERS] initdb and fsync

2012-01-27 Thread Jeff Davis
a platform-independent fsync be available at initdb time? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] initdb and fsync

2012-01-28 Thread Jeff Davis
olved the problem, and I think it was only observed once (though I could probably reproduce it if I tried). The symptom was a log message indicating that PG_VERSION was missing or corrupt on a system that was previously started and online (albeit briefly for a test). Regards, Jeff Davis

Re: [HACKERS] initdb and fsync

2012-01-28 Thread Jeff Davis
ink there would be a surprise factor if sometimes initdb had a long pause at the end and caused 10GB of data to be written out. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: GiST for range types (was Re: [HACKERS] Range Types - typo + NULL string constructor)

2012-01-29 Thread Jeff Davis
Marking "ready for committer", but please apply my comment fixes at your discretion. Regards, Jeff Davis PS: the test was run on my workstation (Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz) with work_mem=512MB, shared_buffers=512MB, and checkpoint_segments=32. The rest of the setti

Re: [HACKERS] initdb and fsync

2012-02-04 Thread Jeff Davis
se with fsync > calls. It looks like there are only a few places, so I don't think clutter is really the problem with the simple patch at this point (unless there is a portability problem with just calling fsync). Regards, Jeff Davis initdb-fsync.patch.gz Description: GNU Zip com

Re: [HACKERS] initdb and fsync

2012-02-05 Thread Jeff Davis
evelopers can set. Or maybe developers don't care about 0.3s? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SSI rw-conflicts and 2PC

2012-02-22 Thread Jeff Davis
ransaction due to > serialization failure involving a prepared transaction, we might want > to include the prepared transaction's gid in the errdetail. I like this idea. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] initdb and fsync

2012-03-12 Thread Jeff Davis
#x27;s pretty inefficient considering that initdb -D data --nosync && sync only takes a couple seconds. Clearly batching the operation is a big help. Maybe there's some more efficient way to fsync a lot of files/directories? Or maybe I can mitigate it by avoiding files that don&#

Re: [HACKERS] initdb and fsync

2012-03-13 Thread Jeff Davis
se+sync: ~ 1.3s Patch attached. Now I feel much better about it. Most people will either have fadvise, a write cache (rightly or wrongly), or actually need the sync. Those that have none of those can use -N. Regards, Jeff Davis initdb-fsync-20120313.patch.gz Description: GNU Zip

Re: [HACKERS] initdb and fsync

2012-03-14 Thread Jeff Davis
On Wed, 2012-03-14 at 10:26 +0100, Andres Freund wrote: > On Wednesday, March 14, 2012 05:23:03 AM Jeff Davis wrote: > > On Tue, 2012-03-13 at 09:42 +0100, Andres Freund wrote: > > > for recursively everything in dir: > > >posix_fadvise(fd, POSIX_FADV_DONTNEED); &

Re: [HACKERS] Storage Manager crash at mdwrite()

2012-03-15 Thread Jeff Davis
here > and it executed without any problems (I wrote/deleted 100 files). > That's why I'm doubting that it's something related to postgreSQL. > > > Any ideas on what's going wrong? What code, exactly, did you change in md.c, and anywhere else in postgres? Are

Re: [HACKERS] Storage Manager crash at mdwrite()

2012-03-15 Thread Jeff Davis
son you think that these would all work together nicely? I don't know the specifics, but I understand there are numerous perils to linking complex C++ code into complex C code, particularly around exception handling. Look in the archives for previous discussions around C++. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] initdb and fsync

2012-03-16 Thread Jeff Davis
gain, maybe casual users don't care much about ~10s for initdb anyway? It's a fairly rare operation for everyone except developers. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-16 Thread Jeff Davis
decreases so does the safety of the assumption that > rows are uniformly distributed. Just trying to follow along. You mean "as the selectivity _increases_ the safety of the assumption that the rows are uniformly distributed decreases", right? Regards, Jeff Davis -- Sent

Re: [HACKERS] Storage Manager crash at mdwrite()

2012-03-16 Thread Jeff Davis
I think the next step would be for you to compile in debug mode with -O0, and attach to the bgwriter process with gdb, and step through it. Then, you can see the exact path which causes the bgwriter to exit, and that will give you a better idea where the problem is. Regards, Jeff Davis

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-19 Thread Jeff Davis
hat should be modeled or implemented though. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] initdb and fsync

2012-03-25 Thread Jeff Davis
posix_fadvise() is a fallback. Regards, Jeff Davis initdb-fsync-20120325.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-26 Thread Jeff Davis
high-availability.sgml makes reference to pg_cancel_backend(), and it might be worthwhile to add an "...and pg_terminate_backend()" there. Other than that, it looks good to me. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade incorrectly equates pg_default and database tablespace

2012-03-30 Thread Jeff Davis
tblspace = dbinfo->db_tblspace; > > Patch to fix this is attached. I confirmed this bug upgrading 9.1 to master, and that this patch fixes it. Thank you for the report! Patch looks good to me as well, with one very minor nitpick: the added comment is missing an apostrophe. Bruce, ca

Re: [HACKERS] pg_upgrade incorrectly equates pg_default and database tablespace

2012-04-06 Thread Jeff Davis
On Fri, 2012-03-30 at 13:11 -0700, Jeff Davis wrote: > I confirmed this bug upgrading 9.1 to master, and that this patch fixes > it. Thank you for the report! > > Patch looks good to me as well, with one very minor nitpick: the added > comment is missing an apostrophe. > >

Re: [HACKERS] bug in fast-path locking

2012-04-09 Thread Jeff Davis
must indicate whether it's in this code path or not; meaning that it needs to do something on the error path (in this case, decrement the count). That was the source of this bug. There may be a way around this problem, but nothing occurs to me right now. Regards,

Re: [HACKERS] bug in fast-path locking

2012-04-09 Thread Jeff Davis
way until you decremented it (which is normally at the time the lock is released); which is a cure worse than the disease. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] bug in fast-path locking

2012-04-09 Thread Jeff Davis
ckAcquire(); I don't think it's necessary outside of LockErrorCleanup(), right? I think there could be some more asserts. There are three sites that decrement FastPathStrongRelationLocks, but in two of them StrongLockInProgress should always be NULL. Other than that, it looks good to me.

Re: [HACKERS] bug in fast-path locking

2012-04-09 Thread Jeff Davis
On Mon, 2012-04-09 at 22:47 -0700, Jeff Davis wrote: > but other similar paths do: > > if (!proclock) > { > AbortStrongLockAcquire(); > > I don't think it's necessary outside of LockErrorCleanup(), right? I take that back, it's necessary

Re: [HACKERS] Last gasp

2012-04-10 Thread Jeff Davis
t. I agree that we should continue to have a mechanism to review patches that aren't ready for commit, though I'm fine if we change it. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-25 Thread Jeff Davis
we think we need. That being said, I agree with you that planning in advance is important here, so that everyone knows when they need to get format-breaking changes in by. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] visibilitymap_set and checksums

2013-05-29 Thread Jeff Davis
I just can't see a better way. Agreed on all counts. Comment patches are welcome, of course. I'll add that if we remove PD_ALL_VISIBLE, this complexity disappears. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] getting rid of freezing

2013-05-29 Thread Jeff Davis
of where we are today, > although perhaps I am a pessimist. If we removed PD_ALL_VISIBLE, then this would be very simple, right? We would just follow normal logging rules for setting the visible or frozen bit. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-

Re: [HACKERS] getting rid of freezing

2013-05-29 Thread Jeff Davis
r in detail, but it seems like it may be possible -- it may even simplify things. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-29 Thread Jeff Davis
al to consider all-visible pages as frozen would be a > huge win, because you'd only need to emit XLOG_HEAP_VISIBLE for every > page in the heap, rather than XLOG_HEAP_FREEZE. Agreed. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-05-30 Thread Jeff Davis
age is already dirty. I don't think we're in a hurry to reuse that bit for something else, so maybe it's best to just ignore it entirely. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Placing hints in line pointers

2013-06-09 Thread Jeff Davis
the list of "waiting for a format change"? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

2013-06-09 Thread Jeff Davis
ums are enabled, because we removed page-level bits indicating the presence of a checksum. The patch is a bit ugly: I had to copy some code, and copy the entire checksum.c file (minus some Asserts, which don't work in an external program). Suggestions welcome. Regards, Jeff Davis

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

2013-06-09 Thread Jeff Davis
K_MASK) == HEAP_XMAX_EXCL_LOCK) > #define HEAP_XMAX_IS_KEYSHR_LOCKED(infomask) \ > (((infomask) & HEAP_LOCK_MASK) == HEAP_XMAX_KEYSHR_LOCK) > > Presumably it'd be better to do something similar. I was hesitant to do too much interpretation of the bits. Do you think it would be better to

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

2013-06-10 Thread Jeff Davis
ksum code similarly to what we did for the CRC stuff recently, > so that you only need access to a .h file for it? The CRC implementation is entirely in header files. Do you think we need to go that far, or is it fine to just put it in libpgport and link that to pg_filedump? Regards, Je

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

2013-06-14 Thread Jeff Davis
the cut-point between checksum.c and bufpage.c at the > wrong place. IMO we should move PageCalcChecksum16 in toto into > checksum.c (or really now into checksum_impl.h), because that and not > just checksum_block() is the functionality that is wanted. Agreed. Regards, Jeff Davis -

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Jeff Davis
On Fri, 2013-06-14 at 16:10 +0200, Andres Freund wrote: > Jeff Davis has a patch pending > (1365493015.7580.3240.camel@sussancws0025) that passes the buffer_std > flag down to MarkBufferDirtyHint() for exactly that reason. I thought we > were on track committing that, but rereading t

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-14 Thread Jeff Davis
icrobenchmarks appear to show a benefit, and (thanks to Noah's comment) it seems like the change is safe. Are there any remaining questions or objections? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-06-14 Thread Jeff Davis
recent extension, which is a weakness in the current checksums code. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

  1   2   3   4   5   6   7   8   9   10   >