Re: [HACKERS] Misleading error message in logical decoding for binary plugins

On Fri, Aug 29, 2014 at 11:15 PM, Andres Freund wrote: > > On top of that, a > > logical receiver receives data in textual form even if the decoder is > > marked as binary when getting a message with PQgetCopyData. > > I have no idea what you mean by that. The data is sent in the format the > out

[HACKERS] COPY and heap_sync

If you insert tuples with COPY into a table created or truncated in the same transaction, at the end of the COPY it calls heap_sync. But there cases were people use COPY in a loop with a small amount of data in each statement. Now it is calling heap_sync many times, and if NBuffers is large doing

Re: [HACKERS] ALTER SYSTEM RESET?

On Wed, Aug 27, 2014 at 7:16 PM, Fujii Masao wrote: > The patch looks good to me. One minor comment is; probably you need to > update the tab-completion code. Thanks for the review. I have updated the patch to support tab-completion. As this is a relatively minor change, I will mark it as "Ready

Re: [HACKERS] On partitioning

On Sat, Aug 30, 2014 at 12:56 AM, Alvaro Herrera wrote: > Prompted by a comment in the UPDATE/LIMIT thread, I saw Marko Tiikkaja > reference Tom's post > http://www.postgresql.org/message-id/1598.1399826...@sss.pgh.pa.us > which mentions the possibility of a different partitioning > implementation

Re: [HACKERS] Inverse of pg_get_serial_sequence?

Andres Freund-3 wrote > On 2014-08-29 17:55:38 -0700, David G Johnston wrote: >> Andres Freund-3 wrote > >> >> pg_get_sequence_ownedby(...) > > My problem is that that possibly be confused with the user owning the > sequence :/ Though as soon as that person reads the output their misunderstandi

Re: [HACKERS] Inverse of pg_get_serial_sequence?

On 2014-08-29 17:55:38 -0700, David G Johnston wrote: > Andres Freund-3 wrote > > Hi, > > > > We have pg_get_serial_sequence() mapping (relation, colum) to the > > sequence. What I'm missing right now is the inverse. I.e. given a > > sequence tell me the owner. > > describe.c has a query for that,

Re: [HACKERS] Inverse of pg_get_serial_sequence?

Andres Freund-3 wrote > Hi, > > We have pg_get_serial_sequence() mapping (relation, colum) to the > sequence. What I'm missing right now is the inverse. I.e. given a > sequence tell me the owner. > describe.c has a query for that, and it's not too hard to write, but it > still seems 'unfriendly' n

Re: [HACKERS] [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins

On 2014-08-26 22:04:03 -0400, Robert Haas wrote: > That's all I see on a first-read through. I think I fixed all of them. Thanks. > There might be other > issues, and I haven't checked through it in great detail for mundane > bugs, but generally, I favor pressing on relatively rapidly toward a >

Re: [HACKERS] Why data of timestamptz does not store value of timezone passed to it?

Hi Craig, On Fri, Aug 29, 2014 at 10:17:17AM +0800, Craig Ringer wrote: > (...) It should also discuss the approach of storing a (instant > timestamptz, timezone text) or (instant timestampts, tzoffset > smallint) tuple for when unambiguous representation is required. > > (I guess I just voluntee

Re: [HACKERS] delta relations in AFTER triggers

Heikki Linnakangas wrote: > On 08/28/2014 12:03 AM, Kevin Grittner wrote: >> Heikki Linnakangas wrote: >>> I suggest adding a new hook to the ParseState struct, (p_rangevar_hook >>> ?). The planner calls it whenever it sees a reference to a table, and >>> the hook function returns back some sort

Re: [HACKERS] Re: Why data of timestamptz does not store value of timezone passed to it?

On 8/29/14 11:27 AM, Greg Stark wrote: > You know... I wonder if we have enough leverage in the standards > committee these days that we could usefully push that direction > instead of being pushed around. The standard timestamp with time zone > is not very useful and I'm sure the standards committ

Re: [HACKERS] Switch pg_basebackup to use -X stream instead of -X fetch by default?

On Fri, Aug 29, 2014 at 10:34 PM, Peter Eisentraut wrote: > On 8/27/14 2:55 AM, Magnus Hagander wrote: >> I think the easy way of doing that is to just create an xlog.tar file. >> Since we already create "base.tar" and possibly n*, >> adding one more file shouldn't be a big problem, and would make

Re: [HACKERS] Switch pg_basebackup to use -X stream instead of -X fetch by default?

On 8/27/14 2:55 AM, Magnus Hagander wrote: > I think the easy way of doing that is to just create an xlog.tar file. > Since we already create "base.tar" and possibly n*, > adding one more file shouldn't be a big problem, and would make such > an implementation much easier. Would be trivial to do .t

Re: [HACKERS] alter user set local_preload_libraries.

On 8/28/14 9:01 AM, Kyotaro HORIGUCHI wrote: > I found this issue when trying per-pg_user (role) loading of > auto_analyze and some tweaking tool. It is not necessarily set by > the user by own, but the function to decide whether to load some > module by the session-user would be usable, at least,

Re: [HACKERS] On partitioning

On Fri, Aug 29, 2014 at 11:56 AM, Alvaro Herrera wrote: > In this design, partitions are first-class objects, not normal tables in > inheritance hierarchies. There are no pg_inherits entries involved at all. Whoa. I always assumed that table inheritance was a stepping-stone to real partitioning

Re: [HACKERS] On partitioning

Hannu Krosing wrote: > Cross-partition indexes are especially needed if we want to allow putting > UNIQUE constraints on non-partition-key columns. I'm not going to implement cross-partition indexes in the first patch. They are a huge can of worms. -- Álvaro Herrerahttp://www.2n

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Le 28/08/2014 05:58, Michael Paquier a écrit : > On Thu, Aug 28, 2014 at 7:37 AM, Julien Rouhaud > wrote: >> >> Attached v2 patch implements this approach. All the work is still >> done in pg_stat_get_archiver, as I don't think that having a >> spec

Re: [HACKERS] On partitioning

On 08/29/2014 05:56 PM, Alvaro Herrera wrote: > Prompted by a comment in the UPDATE/LIMIT thread, I saw Marko Tiikkaja > reference Tom's post > http://www.postgresql.org/message-id/1598.1399826...@sss.pgh.pa.us > which mentions the possibility of a different partitioning > implementation than what

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

On 29.8.2014 16:12, Tom Lane wrote: > Jeff Davis writes: >> I have a new approach to the patch which is to call a callback at each >> block allocation and child contexts inherit the callback from their >> parents. The callback could be defined to simply dereference and >> increment its argument, w

Re: [HACKERS] Built-in binning functions

Hi I am looking to your last patch and I have a few questions, notes 1. I am thinking so reduction to only numeric types is not necessary - although we can live without it - but there are lot of non numeric categories: chars, date, ... 2. Still I strongly afraid about used searching method - the

Re: [HACKERS] On partitioning

On 08/29/2014 07:15 PM, Tom Lane wrote: > Alvaro Herrera writes: >> Tom Lane wrote: >>> One other interesting thought that occurs to me: are we going to support >>> UPDATEs that cause a row to belong to a different partition? If so, how >>> are we going to handle the update chain links? >> Bah, I

Re: [HACKERS] clang warning on master

Pavel Stehule writes: > I see a lot of warnings > [pavel@localhost postgresql]$ make all | grep warning > exec.c:280:2: warning: implicitly declaring library function 'strlcpy' with > type 'unsigned long (char *, const char *, unsigned long)' > strlcpy(link_buf, fname, sizeof(link_buf));

Re: [HACKERS] On partitioning

Andres Freund writes: > On 2014-08-29 13:29:19 -0400, Tom Lane wrote: >> An actual fix would presumably involve adding a partition number to the >> ctid chain field in tuples in partitioned tables. The reason I bring it >> up now is that we'd have to commit to doing that (or at least leaving room

[HACKERS] clang warning on master

Hi I see a lot of warnings [pavel@localhost postgresql]$ make all | grep warning exec.c:280:2: warning: implicitly declaring library function 'strlcpy' with type 'unsigned long (char *, const char *, unsigned long)' strlcpy(link_buf, fname, sizeof(link_buf)); ^ exec.c:280:2: note:

Re: [HACKERS] pgbench throttling latency limit

Hello Heikki, [...] I would be fine with both. After giving it some thought, ISTM better to choose consistency over intuition, and have latency under throttling always defined wrt the scheduled start time and not the actual start time, even if having a latency of 1 ms for an OLTP load

Re: [HACKERS] On partitioning

On 2014-08-29 13:29:19 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-08-29 13:15:16 -0400, Tom Lane wrote: > >> Hm. I certainly agree that it's a case that could be disallowed for a > >> first cut, but it'd be nice to have some clue about how we might allow it > >> eventually. > > >

Re: [HACKERS] On partitioning

Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> One other interesting thought that occurs to me: are we going to support > >> UPDATEs that cause a row to belong to a different partition? If so, how > >> are we going to handle the update chain links? > > > Bah, I didn't mention

Re: [HACKERS] [v9.5] Custom Plan API

On Wed, Aug 27, 2014 at 6:51 PM, Kouhei Kaigai wrote: >> > I'd like to follow this direction, and start stripping the DDL support. >> >> ...please make it so. >> > The attached patch eliminates DDL support. > > Instead of the new CREATE CUSTOM PLAN PROVIDER statement, > it adds an internal functio

Re: [HACKERS] On partitioning

Andres Freund writes: > On 2014-08-29 13:15:16 -0400, Tom Lane wrote: >> Hm. I certainly agree that it's a case that could be disallowed for a >> first cut, but it'd be nice to have some clue about how we might allow it >> eventually. > Not pretty, but we could set t_ctid to some 'magic' value w

[HACKERS] Inverse of pg_get_serial_sequence?

Hi, We have pg_get_serial_sequence() mapping (relation, colum) to the sequence. What I'm missing right now is the inverse. I.e. given a sequence tell me the owner. describe.c has a query for that, and it's not too hard to write, but it still seems 'unfriendly' not to provide it. Does anybody disl

Re: [HACKERS] On partitioning

On 2014-08-29 13:15:16 -0400, Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> One other interesting thought that occurs to me: are we going to support > >> UPDATEs that cause a row to belong to a different partition? If so, how > >> are we going to handle the update chain links?

Re: [HACKERS] On partitioning

Alvaro Herrera writes: > Tom Lane wrote: >> One other interesting thought that occurs to me: are we going to support >> UPDATEs that cause a row to belong to a different partition? If so, how >> are we going to handle the update chain links? > Bah, I didn't mention it? My current thinking is th

Re: [HACKERS] On partitioning

Tom Lane wrote: > Greg Stark writes: > > And I think there are still unanswered questions about indexes. > > One other interesting thought that occurs to me: are we going to support > UPDATEs that cause a row to belong to a different partition? If so, how > are we going to handle the update chai

Re: [HACKERS] On partitioning

Greg Stark writes: > And I think there are still unanswered questions about indexes. One other interesting thought that occurs to me: are we going to support UPDATEs that cause a row to belong to a different partition? If so, how are we going to handle the update chain links?

Re: [HACKERS] On partitioning

2014-08-29 18:35 GMT+02:00 Tom Lane : > Alvaro Herrera writes: > > [ partition sketch ] > > > In this design, partitions are first-class objects, not normal tables in > > inheritance hierarchies. There are no pg_inherits entries involved at > all. > > Hm, actually I'd say they are *not* first cl

Re: [HACKERS] On partitioning

On Fri, Aug 29, 2014 at 4:56 PM, Alvaro Herrera wrote: > For scan plans, we need to prepare Append lists which are used to scan > for tuples in a partitioned relation. We can setup fake constraint > expressions based on the partitioning expressions, which let the planner > discard unnecessary par

Re: [HACKERS] On partitioning

Alvaro Herrera writes: > [ partition sketch ] > In this design, partitions are first-class objects, not normal tables in > inheritance hierarchies. There are no pg_inherits entries involved at all. Hm, actually I'd say they are *not* first class objects; the problem with the existing design is

[HACKERS] On partitioning

Prompted by a comment in the UPDATE/LIMIT thread, I saw Marko Tiikkaja reference Tom's post http://www.postgresql.org/message-id/1598.1399826...@sss.pgh.pa.us which mentions the possibility of a different partitioning implementation than what we have so far. As it turns out, I've been thinking abo

[HACKERS] Re: Why data of timestamptz does not store value of timezone passed to it?

On Fri, Aug 29, 2014 at 4:19 PM, David G Johnston wrote: > And forcing people to change their data types to migrate to PostgreSQL is > undesirable IF our type is usefully equivalent to others in the majority of > situations - though I don't know if that is actually the case. You know... I wonder

Re: [HACKERS] Why data of timestamptz does not store value of timezone passed to it?

Kevin Grittner writes: > It was actually rather disappointing to hear that we had a > conforming implementation and changed away from it circa the 7.2 > release; That is not the case. The existing implementation is work that Tom Lockhart did around 6.3 or so. It was called timestamp at the time

[HACKERS] Re: Why data of timestamptz does not store value of timezone passed to it?

On Fri, Aug 29, 2014 at 11:12 AM, Greg Stark [via PostgreSQL] < ml-node+s1045698n5816903...@n5.nabble.com> wrote: > On Fri, Aug 29, 2014 at 4:03 PM, Kevin Grittner <[hidden email] > > wrote: > > It was actually rather disappointing to hear that

Re: [HACKERS] Misleading error message in logical decoding for binary plugins

On Fri, Aug 29, 2014 at 11:39 PM, Andres Freund wrote: > Yes. The output plugin declares whether it requires the *output method* > to support binary data. pg_logical_slot_peek_changes *can not* support > binary data because it outputs data as > text. pg_logical_slot_peek_binary_changes *can* supp

[HACKERS] Re: Why data of timestamptz does not store value of timezone passed to it?

On Fri, Aug 29, 2014 at 4:03 PM, Kevin Grittner wrote: > It was actually rather disappointing to hear that we had a > conforming implementation and changed away from it circa the 7.2 > release; and even more disturbing to hear that decision is still > being defended on the grounds that there's no

Re: [HACKERS] Why data of timestamptz does not store value of timezone passed to it?

Tom Lane wrote: > Kevin Grittner writes: > >> But the standard doesn't say anything about storing a time zone >> *name* or *abbreviation* -- it requires that it be stored as UTC >> with the *offset* (in hours and minutes).  That makes it pretty >> close to what we have -- it's all about a differe

Re: [HACKERS] Removing dependency to wsock32.lib when compiling code on WIndows

On 08/29/2014 10:15 AM, Noah Misch wrote: On Thu, Aug 21, 2014 at 11:29:31PM -0400, Noah Misch wrote: On Thu, Aug 21, 2014 at 09:18:22AM -0400, Andrew Dunstan wrote: What's happening about this? Buildfarm animal jacana is consistently red because of this. If nobody plans to do the aforementio

Re: [HACKERS] Misleading error message in logical decoding for binary plugins

On 2014-08-29 23:31:49 +0900, Michael Paquier wrote: > On Fri, Aug 29, 2014 at 11:15 PM, Andres Freund > wrote: > > > No, a textual output plugin is *NOT* allowed to produce binary > > output. That'd violate e.g. pg_logical_slot_peek_changes's return type > > because it's only declared to return

Re: [HACKERS] Question about coding of free space map

Heikki Linnakangas writes: > On 08/26/2014 05:13 AM, Tatsuo Ishii wrote: >> While looking into backend/storage/freespace/freespace.c, I noticed >> that struct FSMAddress is passed to functions by value, rather than >> reference. > There isn't really any strict coding rule on that. We pass RelFile

Re: [HACKERS] LIMIT for UPDATE and DELETE

Marko Tiikkaja writes: > The LIMIT part *has* to happen after the rows have been locked or it > will work very surprisingly under concurrency (sort of like how FOR > SHARE / FOR UPDATE worked before 9.0). Good point. > So either it has to be inside > ModifyTable or the ModifyTable has to some

Re: [HACKERS] Misleading error message in logical decoding for binary plugins

On Fri, Aug 29, 2014 at 11:15 PM, Andres Freund wrote: > No, a textual output plugin is *NOT* allowed to produce binary > output. That'd violate e.g. pg_logical_slot_peek_changes's return type > because it's only declared to return text. > A textual output plugin can call pg_logical_slot_peek_bi

Re: [HACKERS] Misleading error message in logical decoding for binary plugins

On 2014-08-29 23:07:44 +0900, Michael Paquier wrote: > On Fri, Aug 29, 2014 at 10:48 PM, Andres Freund > wrote: > > > On 2014-08-29 22:42:46 +0900, Michael Paquier wrote: > > > Hi all, > > > > > > Using a plugin producing binary output, I came across this error: > > > =# select data from pg_logic

Re: [HACKERS] Removing dependency to wsock32.lib when compiling code on WIndows

On Thu, Aug 21, 2014 at 11:29:31PM -0400, Noah Misch wrote: > On Thu, Aug 21, 2014 at 09:18:22AM -0400, Andrew Dunstan wrote: > > What's happening about this? Buildfarm animal jacana is consistently > > red because of this. > > If nobody plans to do the aforementioned analysis in the next 4-7 days

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

Jeff Davis writes: > I have a new approach to the patch which is to call a callback at each > block allocation and child contexts inherit the callback from their > parents. The callback could be defined to simply dereference and > increment its argument, which would mean block allocations anywhere

Re: [HACKERS] Misleading error message in logical decoding for binary plugins

On Fri, Aug 29, 2014 at 10:48 PM, Andres Freund wrote: > On 2014-08-29 22:42:46 +0900, Michael Paquier wrote: > > Hi all, > > > > Using a plugin producing binary output, I came across this error: > > =# select data from pg_logical_slot_peek_changes('foo', NULL, NULL); > > ERROR: 0A000: output pl

Re: [HACKERS] Misleading error message in logical decoding for binary plugins

On 2014-08-29 22:42:46 +0900, Michael Paquier wrote: > Hi all, > > Using a plugin producing binary output, I came across this error: > =# select data from pg_logical_slot_peek_changes('foo', NULL, NULL); > ERROR: 0A000: output plugin cannot produce binary output > LOCATION: pg_logical_slot_get_c

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

Mark Kirkwood wrote: > On 29/08/14 08:56, Alvaro Herrera wrote: > >Robert Haas wrote: > > > >>I agree that you might not like that. But you might not like having > >>the table vacuumed slower than the configured rate, either. My > >>impression is that the time between vacuums isn't really all tha

[HACKERS] Misleading error message in logical decoding for binary plugins

Hi all, Using a plugin producing binary output, I came across this error: =# select data from pg_logical_slot_peek_changes('foo', NULL, NULL); ERROR: 0A000: output plugin cannot produce binary output LOCATION: pg_logical_slot_get_changes_guts, logicalfuncs.c:404 Shouldn't the error message be h

Re: [HACKERS] [REVIEW] Re: Fix xpath() to return namespace definitions

Greetings, Because of the memory bug in xmlCopyNode, this is a new patch with different method. In this patch, instead of using xmlCopyNode to bring the namespace back, we added the required namespaces to the node before dumping the node to string, and cleaning it up afterwards (because the same

Re: [HACKERS] Question about coding of free space map

On 08/26/2014 05:13 AM, Tatsuo Ishii wrote: While looking into backend/storage/freespace/freespace.c, I noticed that struct FSMAddress is passed to functions by value, rather than reference. I thought our code practice is defining pointer to a struct data and using the pointer for parameter passi

Re: [HACKERS] LIMIT for UPDATE and DELETE

On 8/29/14 1:53 PM, I wrote: This is just my personal opinion, but what I think should happen is: 1) We put the LIMIT inside ModifyTable like this patch does. This doesn't prevent us from doing ORDER BY in the future, but helps numerous people who today have to Oops, looks like I didn't f

Re: [HACKERS] LIMIT for UPDATE and DELETE

On 8/29/14 12:20 PM, Etsuro Fujita wrote: The patch places limit-counting inside ModifyTable, and works well for inheritance trees, but I'm not sure that that is the right way to go. I think that this feature should be implemented in the way that we can naturally extend it to the ORDER-BY-LIMIT

Re: [HACKERS] pgbench throttling latency limit

Hello Heikki, This now begs the question: In --rate mode, shouldn't the reported transaction latency also be calculated from the *scheduled* start time, not the time the transaction actually started? Otherwise we're using two different definitions of "latency", one for the purpose of the li

Re: [HACKERS] LIMIT for UPDATE and DELETE

(2014/08/25 15:48), Etsuro Fujita wrote: (2014/08/15 6:18), Rukh Meski wrote: Based on the feedback on my previous patch, I've separated only the LIMIT part into its own feature. This version plays nicely with inheritance. The intended use is splitting up big UPDATEs and DELETEs into batches m

Re: [HACKERS] pgbench throttling latency limit

On 08/27/2014 08:05 PM, Fabien COELHO wrote: [...] Yeah, something like that. I don't think it would be necessary to set statement_timeout, you can inject that in your script or postgresql.conf if you want. I don't think aborting a transaction that's already started is necessary either. You c

Re: [HACKERS] v4 protocol TODO item - Lazy fetch/stream of TOASTed values?

On 08/28/2014 02:14 PM, Craig Ringer wrote: Hi all Per the protocol todo: https://wiki.postgresql.org/wiki/Todo#Wire_Protocol_Changes do you think it's reasonable to allow for delayed sending of big varlena values and arrays in the protocol? Yes. - Heikki -- Sent via pgsql-hackers maili

Re: [HACKERS] psql \watch versus \timing

On 08/28/2014 02:46 PM, Fujii Masao wrote: On Tue, Aug 26, 2014 at 4:55 AM, Heikki Linnakangas wrote: On 08/25/2014 10:48 PM, Heikki Linnakangas wrote: Actually, perhaps it would be better to just copy-paste PSQLexec, and modify the copy to suite \watch's needs. (PSQLexecWatch? SendWatchQuery?

Re: [HACKERS] implement subject alternative names support for SSL connections

On 08/28/2014 07:28 PM, Alexey Klyukin wrote: On Mon, Aug 25, 2014 at 12:02 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: On 08/24/2014 03:11 PM, Alexey Klyukin wrote: On Wed, Aug 20, 2014 at 11:53 AM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: The patch doesn't seem

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

On Fri, 2014-08-22 at 12:34 -0400, Robert Haas wrote: > Still doesn't look good here. On the same PPC64 machine I've been > using for testing: I have a new approach to the patch which is to call a callback at each block allocation and child contexts inherit the callback from their parents. The ca

Re: [HACKERS] Why data of timestamptz does not store value of timezone passed to it?

On 08/29/2014 05:28 AM, Tom Lane wrote: "k...@rice.edu" writes: On Thu, Aug 28, 2014 at 03:33:56PM -0400, Bruce Momjian wrote: So the standard requires storing of original timezone in the data type? I was not aware of that. I do not have a copy of the SQL 92 spec, but several references to t

Re: [HACKERS] Multithreaded SIGPIPE race in libpq on Solaris

On 29 August 2014 01:04, Thomas Munro wrote: > On 28 August 2014 23:45, Tom Lane wrote: >> I don't claim to be an expert on this stuff, but I had the idea that >> multithreaded environments were supposed to track signal state per-thread >> not just per-process, precisely because of issues like th