Re: [HACKERS] extending relations more efficiently

2012-05-02 Thread Jeroen Vermeulen
On 2012-05-01 22:06, Robert Haas wrote: It might also be interesting to provide a mechanism to pre-extend a relation to a certain number of blocks, though if we did that we'd have to make sure that autovac got the memo not to truncate those pages away again. Good point. And just to check befo

Re: [HACKERS] foreign key locks, 2nd attempt

2012-02-24 Thread Jeroen Vermeulen
On 2012-02-23 22:12, Noah Misch wrote: That alone would not simplify the patch much. INSERT/UPDATE/DELETE on the foreign side would still need to take some kind of tuple lock on the primary side to prevent primary-side DELETE. You then still face the complicated case of a tuple that's both loc

Re: [HACKERS] foreign key locks, 2nd attempt

2012-02-23 Thread Jeroen Vermeulen
On 2012-02-23 10:18, Simon Riggs wrote: However, review of such a large patch should not be simply pass or fail. We should be looking back at the original problem and ask ourselves whether some subset of the patch could solve a useful subset of the problem. For me, that seems quite likely and th

Re: [HACKERS] VACUUM ANALYZE is faster than ANALYZE?

2012-02-22 Thread Jeroen Vermeulen
On 2012-02-22 16:29, Tom Lane wrote: (Snip context) VACUUM ANALYZE consists of two separate passes, VACUUM and then ANALYZE, and the second pass is going to be "random" I/O by your definition no matter what. I don't suppose there's a cas

Re: [HACKERS] foreign key locks, 2nd attempt

2011-11-11 Thread Jeroen Vermeulen
On 2011-11-12 00:30, David Kerr wrote: Is this being suggested in lieu of Alvaro's patch? because it seems to be adding complexity to the system (multiple types of primary key definitions) instead of just fixing an obvious problem (over-aggressive locking done on FK checks). It wouldn't be a n

Re: [HACKERS] Disable OpenSSL compression

2011-11-08 Thread Jeroen Vermeulen
On 2011-11-08 22:59, Albe Laurenz wrote: In addition to the oprofile data I collected three times: - the duration as shown in the server log - the duration as shown by \timing - the duration of the psql command as measured by "time" [...] I think this makes a good case for disabling compress

Re: [HACKERS] foreign key locks, 2nd attempt

2011-11-06 Thread Jeroen Vermeulen
On 2011-11-04 01:12, Alvaro Herrera wrote: I would like some opinions on the ideas on this patch, and on the patch itself. If someone wants more discussion on implementation details of each part of the patch, I'm happy to provide a textual description -- please just ask. Jumping in a bit late

Re: [HACKERS] Multiple queries in transit

2011-11-06 Thread Jeroen Vermeulen
On 2011-11-03 17:26, Marko Kreen wrote: On Mon, Oct 31, 2011 at 7:09 PM, Tom Lane wrote: Can't you do that today with a multi-command string submitted to PQsendQuery, followed by multiple calls to PQgetResult? It's more annoying to to error handling on that, plus it still keeps the blocking b

Re: [HACKERS] IDLE in transaction introspection

2011-11-01 Thread Jeroen Vermeulen
On 2011-11-01 21:13, Andrew Dunstan wrote: Rename it please. "current_query" will just be wrong. I'd be inclined just to call it "query" or "query_string" and leave it to the docs to define the exact semantics. I think "query" for a query that isn't ongoing would be just as wrong. How about "

Re: [HACKERS] Multiple queries in transit

2011-10-31 Thread Jeroen Vermeulen
On 2011-11-01 00:53, Merlin Moncure wrote: On Mon, Oct 31, 2011 at 12:49 PM, Mark Hills wrote: Furthermore, in most apps it'd be a serious PITA to keep track of which reply is for which query, so I doubt that such a feature is of general usefulness. In our UI case, we already have a queue.

Re: [HACKERS] News on Clang

2011-06-24 Thread Jeroen Vermeulen
On 2011-06-25 00:02, Peter Geoghegan wrote: At a large presentation that I and other PG community members were present at during FOSDEM, Postgres was specifically cited as an example of a medium-sized C program that had considerably improved compile times on Clang. While I was obviously unable t

Re: [HACKERS] [BUG] Denormal float values break backup/restore

2011-06-20 Thread Jeroen Vermeulen
On 2011-06-20 19:22, Marti Raudsepp wrote: AIUI that is defined to be a little vague, but includes denormalized numbers that would undergo any rounding at all. It says that on overflow the conversion should return the appropriate HUGE_VAL variant, and set ERANGE. On underflow it returns a rea

Re: [HACKERS] [BUG] Denormal float values break backup/restore

2011-06-11 Thread Jeroen Vermeulen
On 2011-06-11 01:57, Tom Lane wrote: (5) Lobby your libc providers to make strtod accept denormals without throwing ERANGE. There is no obvious reason why it shouldn't. (On at least one of my boxes, it does.) The standard either explicitly allows or requires this behaviour (depending on whi

Re: [HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-04 Thread Jeroen Vermeulen
On 2010-12-03 20:49, Ana Carolina Brito de Almeida wrote: We add a simple case study (sourceforge page): http://sourceforge.net/projects/hypotheticalind/files/TUTORIAL_8_4.pdf/download Great, thanks! I'll try to write a bit more about it later: http://pqxx.org/development/libpqxx/wiki/Hypoth

Re: [HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-03 Thread Jeroen Vermeulen
On 2010-12-03 19:44, Sergio Lifschitz wrote: Indeed, hypothetical indexes are good to check potentially good configurations without harming the whole system with actual index creation. Please observer that we've added an "explain hypothetical" command, that will include plans considering hypothet

Re: [HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-03 Thread Jeroen Vermeulen
On 2010-12-02 00:48, Ana Carolina Brito de Almeida wrote: We would like to inform you all that our extension to PostgreSQL, that includes hypothetical indexes (and soon index self-tuning), is available through a sourgeforge project. This was suggested at PgCon 2010 and we hope some of you may

Re: [HACKERS] Indent authentication overloading

2010-11-17 Thread Jeroen Vermeulen
On 2010-11-18 00:14, Magnus Hagander wrote: If it was a matter of changing it for those who use ident over tcp, I really wouldn't hesitate - they're few :-) But the problem is that it's the ident-over-tcp that's correctly named, not the other one... True. By the way ISTR we don't fall back to

Re: [HACKERS] Indent authentication overloading

2010-11-17 Thread Jeroen Vermeulen
On 2010-11-17 22:43, Magnus Hagander wrote: at the advantage of not confusing new users. We could of course also just drop ident-over-tcp completely, but there might be some poor guy out there who actually *uses* it :-) As far as I know, companies do use it in their internal networks where th

Re: [HACKERS] Amazon now supporting GPU focused EC2 instances

2010-11-15 Thread Jeroen Vermeulen
On 2010-11-15 18:49, Greg Stark wrote: I've seen papers on doing relational joins using GPUs and I'm sure there are other algorithms we wonderful stuff we could do. But if it comes at the cost of being able to handle arbitrary join clauses it'll be a tough sacrifice to make. Perhaps the cooles

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-26 Thread Jeroen Vermeulen
Mark Mielke wrote: Re-planning a generic plan with another generic plan may generate zero benefit, with a measurable cost. More on this after... Nobody's talking about doing that any more. I proposed it initially because I didn't know about changes that made it unnecessary. All the points

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-25 Thread Jeroen Vermeulen
Robert Haas wrote: On Wed, Feb 17, 2010 at 5:52 PM, Jeroen Vermeulen wrote: I may have cut this out of my original email for brevity... my impression is that the planner's estimate is likely to err on the side of scalability, not best-case response time; and that this is more likely to h

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-21 Thread Jeroen Vermeulen
Greg Stark wrote: So in principle I agree with this idea. I think a conservative value for the constant would be more like 100x though. If I told you we had an easy way to speed all your queries up by 10% by caching queries but were just choosing not to then I think you would be unhappy. Whereas

Re: [HACKERS] libpq PGresult object and encoding

2010-02-21 Thread Jeroen Vermeulen
Jeff Davis wrote: libpq has a PQclientEncoding() function that takes a connection object. However, the client encoding is, in some cases, a property of the result object. For instance, if your client_encoding changes, but you keep the result object around, you have no way to determine later wha

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-15 Thread Jeroen Vermeulen
Tom Lane wrote: Well, no, consider the situation where planning takes 50 ms, the generic plan costs 100ms to execute, but a parameter-specific plan would take 1ms to execute. Planning is very expensive compared to execution but it's still a win to do it. I think that's a fun and worthwhile pr

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-09 Thread Jeroen Vermeulen
Andres Freund wrote: = Actual-cost threshold = Also stop using the generic plan if the statement takes a long time to run in practice. Statistics may have gone bad. It could also be a one-off due to a load peak or something, but that's handled by: That is not that easy. It means that you ha

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-09 Thread Jeroen Vermeulen
Richard Huxton wrote: = Actual-cost threshold = Also stop using the generic plan if the statement takes a long time to run in practice. Do you mean: 1. Rollback the current query and start again 2. Mark the plan as a bad one and plan again next execute If you can figure out how to do #1 then

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-09 Thread Jeroen Vermeulen
Yeb Havinga wrote: I've been discussing this with Josh, Heikki, and Peter E. over the past few weeks. Is this searchable in the archives? I'm interested in ideas discussed. No, sorry. These were face-to-face discussions at linux.conf.au and FOSDEM. If a prepared statement takes parameters,

[HACKERS] Avoiding bad prepared-statement plans.

2010-02-09 Thread Jeroen Vermeulen
I've been discussing this with Josh, Heikki, and Peter E. over the past few weeks. As Peter observed years ago, prepared statements can perform badly because their plans are overly generic. Also, statistics change and sometimes plans should change with them. It would be nice if we could avo

Re: [HACKERS] libpq WSACleanup is not needed

2009-01-20 Thread Jeroen Vermeulen
Merlin Moncure wrote: I think init/uninit is the answer. While writing libpqtypes, we noted that libpq is just plain awkward in a few different ways and probably deserves a rewrite at some point. not today though Would there be any serious harm in: 1. doing the WSAStartup() when the fir

Re: [HACKERS] Fixes for compiler warnings

2009-01-20 Thread Jeroen Vermeulen
Peter Eisentraut wrote: -Wformat-security warns about printf(var); but not about printf(var, a); I don't understand that; the crash or exploit potential is pretty much the same in both cases. Not sure this is the reason, but in the first case any risk is trivially avoided by usin

Re: [HACKERS] binary representation of datatypes

2008-10-21 Thread Jeroen Vermeulen
Matthieu Imbert wrote: scenario 1 - parse the textual representation of all results of requests to the database and convert textual timestamps to a binary format that i choose among those ones (number of microseconds since 2000-01-01, or a structure similar to pg_tm (but with microsecond preci

Re: [HACKERS] Concurrent VACUUM and ANALYZE

2008-07-24 Thread Jeroen Vermeulen
Jonah H. Harris wrote: On Mon, Jul 21, 2008 at 10:19 PM, Tom Lane <[EMAIL PROTECTED]> wrote: "Jonah H. Harris" <[EMAIL PROTECTED]> writes: The case I'm looking at is a large table which requires a lazy vacuum, and a zero vacuum cost delay would cause too much I/O. Yet, this table has enough in