Re: [HACKERS] Page Checksums

2012-01-23 Thread Florian Weimer
* Robert Treat: > Would it be unfair to assert that people who want checksums but aren't > willing to pay the cost of running a filesystem that provides > checksums aren't going to be willing to make the cost/benefit trade > off that will be asked for? Yes, it is unfair of course, but it's > inter

Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-01-23 Thread Peter Geoghegan
On 24 January 2012 06:26, Tatsuo Ishii wrote: >> ** pgbench, permanent tables, scale factor 100, 300 s ** >> 1 group-commit-2012-01-21 614.425851 -10.4% >> 8 group-commit-2012-01-21 4705.129896 +6.3% >> 16 group-commit-2012-01-21 7962.131701 +2.0% >> 24 group-commit-2012-01-21 13074.939290 -1.5% >

Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-01-23 Thread Tatsuo Ishii
> ** pgbench, permanent tables, scale factor 100, 300 s ** > 1 group-commit-2012-01-21 614.425851 -10.4% > 8 group-commit-2012-01-21 4705.129896 +6.3% > 16 group-commit-2012-01-21 7962.131701 +2.0% > 24 group-commit-2012-01-21 13074.939290 -1.5% > 32 group-commit-2012-01-21 12458.962510 +4.5% > 80

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-23 Thread Robert Treat
On Mon, Jan 23, 2012 at 3:21 AM, Benedikt Grundmann wrote: > On 19/01/12 17:39, Greg Smith wrote: >> On 1/19/12 1:10 PM, Robert Haas wrote: >> >I have to say that I find that intensely counterintuitive.  The >> >current settings are not entirely easy to tune correctly, but at least >> >they're eas

Re: [HACKERS] Page Checksums

2012-01-23 Thread Robert Treat
On Sat, Jan 21, 2012 at 6:12 PM, Jim Nasby wrote: > On Jan 10, 2012, at 3:07 AM, Simon Riggs wrote: >> I think we could add an option to check the checksum immediately after >> we pin a block for the first time but it would be very expensive and >> sounds like we're re-inventing hardware or OS fea

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-23 Thread Greg Smith
Benedikt Grundmann wrote: What I think is missing is a clear way to know if you are vacuuming (and analyzing) enough, and how much you are paying for that. Any good way to measure if you're vacuuming a particular table enough needs to note how much free space is in that table and its inde

Re: [HACKERS] Publish checkpoint timing and sync files summary data to pg_stat_bgwriter

2012-01-23 Thread Greg Smith
Robert Haas wrote: On Sat, Jan 21, 2012 at 6:32 PM, Jeff Janes wrote: I'm finding the backend_writes column pretty unfortunate. The only use I know of for it is to determine if the bgwriter is lagging behind. Yet it doesn't serve even this purpose because it lumps together the backend writ

Re: [HACKERS] Next steps on pg_stat_statements normalisation

2012-01-23 Thread Peter Geoghegan
On 22 January 2012 05:30, Peter Geoghegan wrote: > The syntax for constants is sufficiently simple that I think that a > good set of regression tests should make this entirely practicable, > covering all permutations of relevant factors affecting how the > implementation should act, including for

Re: [HACKERS] Patch: Allow SQL-language functions to reference parameters by parameter name

2012-01-23 Thread Matthew Draper
On 19/01/12 20:28, Hitoshi Harada wrote: >> (Now it occurred to me that forgetting the #include parse_func.h might >> hit this breakage..., so I'll fix it here and continue to test, but if >> you'll fix it yourself, let me know) > > I fixed it here and it now works with my environment. Well spott

Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 7:52 PM, Simon Riggs wrote: > On Mon, Jan 23, 2012 at 3:49 PM, Robert Haas wrote: > >>> The other patches have clearer and specific roles without heuristics >>> (mostly), so are at least viable for 9.2, though still requiring >>> agreement. >> >> I think we must also drop

Re: [HACKERS] Inline Extension

2012-01-23 Thread Daniel Farina
On Mon, Jan 23, 2012 at 8:17 AM, Dimitri Fontaine wrote: > Robert Haas writes: >>> Ok, but then, what about .so files?  Wouldn't it make sense to be able >>> to ship also the executable modules needed, and if not, why not? > > Now you can dump/restore any extension fully, and we can even ship any

Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-01-23 Thread Simon Riggs
On Mon, Jan 23, 2012 at 3:49 PM, Robert Haas wrote: >> The other patches have clearer and specific roles without heuristics >> (mostly), so are at least viable for 9.2, though still requiring >> agreement. > > I think we must also drop removebufmgrfreelist-v1 from consideration, ... I think you

Re: [HACKERS] Measuring relation free space

2012-01-23 Thread Noah Misch
On Mon, Jan 23, 2012 at 04:56:24PM -0300, Alvaro Herrera wrote: > Excerpts from Noah Misch's message of vie ene 20 22:33:30 -0300 2012: > > pgstattuple() figures the free_percent by adding up all space available to > > hold tuples and dividing that by the simple size of the relation. Non-leaf > >

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread Merlin Moncure
On Mon, Jan 23, 2012 at 4:12 PM, A.M. wrote: > On Jan 23, 2012, at 4:45 PM, Merlin Moncure wrote: >> Prefer the version.  But why send this over and over with each bind? >> Wouldn't you negotiate that when connecting? Most likely, optionally, >> doing as much as you can from the server version?  P

Re: [HACKERS] JSON for PG 9.2

2012-01-23 Thread Pavel Stehule
2012/1/23 Merlin Moncure : > On Sun, Jan 15, 2012 at 10:08 AM, Andrew Dunstan wrote: >> Here's an update that adds row_to_json, plus a bit more cleanup. > > why not call all these functions 'to_json' and overload them? -1 older proposal is more consistent with xml functions Pavel > > merlin -

Re: [HACKERS] JSON for PG 9.2

2012-01-23 Thread Andrew Dunstan
On 01/23/2012 05:21 PM, Merlin Moncure wrote: On Sun, Jan 15, 2012 at 10:08 AM, Andrew Dunstan wrote: Here's an update that adds row_to_json, plus a bit more cleanup. why not call all these functions 'to_json' and overload them? I don't honestly feel that advances clarity much. And we mi

Re: [HACKERS] JSON for PG 9.2

2012-01-23 Thread Merlin Moncure
On Sun, Jan 15, 2012 at 10:08 AM, Andrew Dunstan wrote: > Here's an update that adds row_to_json, plus a bit more cleanup. why not call all these functions 'to_json' and overload them? merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscrip

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread A.M.
On Jan 23, 2012, at 4:45 PM, Merlin Moncure wrote: > On Mon, Jan 23, 2012 at 2:00 PM, A.M. wrote: >> One simple way clients could detect the binary encoding at startup would be >> to pass known test parameters and match against the returned values. If the >> client cannot match the response, t

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Marko Kreen
On Mon, Jan 23, 2012 at 11:06 PM, Robert Haas wrote: > On Mon, Jan 23, 2012 at 11:15 AM, Noah Misch wrote: >> As I said upthread, and you appeared to agree, the protocol is independent of >> individual data type send/recv formats.  Even if we were already adding >> protocol v4 to PostgreSQL 9.2,

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Merlin Moncure
On Mon, Jan 23, 2012 at 3:06 PM, Robert Haas wrote: > Not being responsible from the maintenance of any PostgreSQL drivers > whatsoever, I don't have a strong feeling about which of these is the > case, and I'd like us to hear from the people who do. I'm just gonna come right out and say that GUC

Re: [HACKERS] PL/pgSQL return value in after triggers

2012-01-23 Thread Pavel Stehule
Hello Peter I checked code, and I don't think so this is good. A design of optional NULL is going to inconsistent syntax. RETURN (OLD, NEW, NULL, /* nothing */) is not consistent But my main argument is not intuitive behave of BEFORE triggers after this change. When somebody write BEFORE trig

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread Merlin Moncure
On Mon, Jan 23, 2012 at 2:00 PM, A.M. wrote: > One simple way clients could detect the binary encoding at startup would be > to pass known test parameters and match against the returned values. If the > client cannot match the response, then it should choose the text > representation. > > Alter

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 11:15 AM, Noah Misch wrote: > As I said upthread, and you appeared to agree, the protocol is independent of > individual data type send/recv formats.  Even if we were already adding > protocol v4 to PostgreSQL 9.2, having array_send() change its behavior in > response to th

Re: [HACKERS] Client Messages

2012-01-23 Thread Jim Mlodgenski
On Wed, Jan 18, 2012 at 9:19 AM, Jim Mlodgenski wrote: > On Wed, Jan 18, 2012 at 3:08 AM, Heikki Linnakangas > wrote: >> On 18.01.2012 07:49, Fujii Masao wrote: >>> >>> On Fri, Jan 6, 2012 at 1:38 AM, Jim Mlodgenski  wrote: I have a need to send banner messages to a psql client that I c

Re: [HACKERS] JSON for PG 9.2

2012-01-23 Thread Peter Eisentraut
On sön, 2012-01-22 at 11:43 -0500, Andrew Dunstan wrote: > Actually, given recent discussion I think that test should just be > removed from json.c. We don't actually have any test that the code > point is valid (e.g. that it doesn't refer to an unallocated code > point). We don't do that elsewher

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread A.M.
On Jan 23, 2012, at 2:49 PM, Tom Lane wrote: > Marko Kreen writes: >> [ bytea_output doesn't need to be GUC_REPORT because format is >> autodetectable ] > > Fair enough. Anyway we're really about two years too late to revisit that. > >> Btw, it does not seems that per-request metainfo change

Re: [HACKERS] Measuring relation free space

2012-01-23 Thread Alvaro Herrera
Excerpts from Noah Misch's message of vie ene 20 22:33:30 -0300 2012: > On Fri, Jan 20, 2012 at 07:03:22PM -0500, Jaime Casanova wrote: > > On Wed, Jan 18, 2012 at 7:01 PM, Noah Misch wrote: > > > On Wed, Jan 18, 2012 at 09:46:20AM -0500, Jaime Casanova wrote: > > >> > > >> ignoring all non-leaf

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread Tom Lane
Marko Kreen writes: > [ bytea_output doesn't need to be GUC_REPORT because format is autodetectable > ] Fair enough. Anyway we're really about two years too late to revisit that. > Btw, it does not seems that per-request metainfo change requires > "major version". It just client can send extr

Re: [HACKERS] Multithread Query Planner

2012-01-23 Thread Merlin Moncure
On Fri, Jan 13, 2012 at 2:29 PM, Christopher Browne wrote: > On Fri, Jan 13, 2012 at 3:14 PM, Frederico wrote: >> Hi folks. >> >> Is there any restriction in create and start threads inside Postgres? >> >> I'm trying to develop a multithread planner, and some times is raised a >> exception of ac

Re: [HACKERS] Collect frequency statistics for arrays

2012-01-23 Thread Alexander Korotkov
On Mon, Jan 23, 2012 at 7:58 PM, Noah Misch wrote: > > + /* Take care about events with low probabilities. */ > > + if (rest > DEFAULT_CONTAIN_SEL) > > + { > > Why the change from "rest > 0" to this in the latest version? > Ealier addition of "rest" distribution require O(m) time. Now

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread Marko Kreen
On Mon, Jan 23, 2012 at 11:20:52AM -0500, Tom Lane wrote: > Robert Haas writes: > > On Mon, Jan 23, 2012 at 9:59 AM, Marko Kreen wrote: > >> Now that I think about it, same applies to bytea_output? > > > Probably so. But I think we need not introduce quite so many new > > threads on this patch.

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Marko Kreen
On Mon, Jan 23, 2012 at 5:34 PM, Robert Haas wrote: > On Mon, Jan 23, 2012 at 9:36 AM, Florian Weimer wrote: >> * Robert Haas: In this particular case, I knew that the change was coming and could push updated Java and Perl client libraries well before the server-side change hit our

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread Tom Lane
Robert Haas writes: > On Mon, Jan 23, 2012 at 9:59 AM, Marko Kreen wrote: >> Now that I think about it, same applies to bytea_output? > Probably so. But I think we need not introduce quite so many new > threads on this patch. This is, I think, at least thread #4, and > that's making the discus

Re: [HACKERS] Inline Extension

2012-01-23 Thread Dimitri Fontaine
Robert Haas writes: >> Ok, but then, what about .so files?  Wouldn't it make sense to be able >> to ship also the executable modules needed, and if not, why not? > > Sure, that would be as useful as any other part of this feature. We'd > have to think carefully about how to make it secure, though

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Noah Misch
On Mon, Jan 23, 2012 at 10:34:12AM -0500, Robert Haas wrote: > On Mon, Jan 23, 2012 at 9:36 AM, Florian Weimer wrote: > > * Robert Haas: > >>> In this particular case, I knew that the change was coming and could > >>> push updated Java and Perl client libraries well before the server-side > >>> ch

Re: Removing freelist (was Re: [HACKERS] Should I implement DROP INDEX CONCURRENTLY?)

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 11:01 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Jan 23, 2012 at 12:12 AM, Tom Lane wrote: The expensive part of what we do while holding BufFreelistLock is, I think, iterating through buffers taking and releasing a spinlock on each one (!). > >>>

Re: Removing freelist (was Re: [HACKERS] Should I implement DROP INDEX CONCURRENTLY?)

2012-01-23 Thread Tom Lane
Robert Haas writes: > On Mon, Jan 23, 2012 at 12:12 AM, Tom Lane wrote: >>> The expensive part of what >>> we do while holding BufFreelistLock is, I think, iterating through >>> buffers taking and releasing a spinlock on each one (!). >> Yeah ... spinlocks that, by definition, will be unconteste

Re: [HACKERS] Collect frequency statistics for arrays

2012-01-23 Thread Noah Misch
On Mon, Jan 23, 2012 at 01:21:20AM +0400, Alexander Korotkov wrote: > Updated patch is attached. I've updated comment > of mcelem_array_contained_selec with more detailed description of > probability distribution assumption. Also, I found that "rest" behavious > should be better described by Poisso

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 9:59 AM, Marko Kreen wrote: > On Sun, Jan 22, 2012 at 11:47 PM, Mikko Tiihonen > wrote: >> * introduced a new GUC variable array_output copying the current >>  bytea_output type, with values "full" (old value) and >>  "smallfixed" (new default) >> * added documentation for

Re: [HACKERS] Inline Extension

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 10:46 AM, Dimitri Fontaine wrote: > Robert Haas writes: >> Hmm, I don't think I like that design.  I think we should view this as >> a way to embed the SQL and control files needed by the extension in >> the server, rather than a separate thing called an inline extension.

Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 10:35 AM, Simon Riggs wrote: > freelist_ok was a prototype for testing/discussion, which contained an > arguable heuristic. I guess that means its also "in play", but I > wasn't thinking we'd be able to assemble clear evidence for 9.2. OK, that one is still in the test run

Re: [HACKERS] Inline Extension

2012-01-23 Thread Dimitri Fontaine
Robert Haas writes: > Hmm, I don't think I like that design. I think we should view this as > a way to embed the SQL and control files needed by the extension in > the server, rather than a separate thing called an inline extension. > If pg_dump is going to dump those files, it ought to dump them

Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-01-23 Thread Simon Riggs
On Mon, Jan 23, 2012 at 3:09 PM, Robert Haas wrote: > I'm working on it. Good, thanks for the update. >> The remaining patch you tested was withdrawn and not submitted to the CF. > > Oh.  Which one was that?  I thought all of these were in play. freelist_ok was a prototype for testing/discuss

Re: [HACKERS] Inline Extension

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 10:26 AM, Dimitri Fontaine wrote: > Robert Haas writes: >> Hmm.  But CREATE EXTENSION / ALTER EXTENSION doesn't seem right, >> because the files in the directory correspond to *available* >> extensions, not already-created ones.  We need some way of dumping and > > I would

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 9:36 AM, Florian Weimer wrote: > * Robert Haas: >>> In this particular case, I knew that the change was coming and could >>> push updated Java and Perl client libraries well before the server-side >>> change hit our internal repository, but I really don't want to have to >>

Re: [HACKERS] Inline Extension

2012-01-23 Thread Dimitri Fontaine
Robert Haas writes: > Hmm. But CREATE EXTENSION / ALTER EXTENSION doesn't seem right, > because the files in the directory correspond to *available* > extensions, not already-created ones. We need some way of dumping and I would have limited the dump query to only known installed extensions, ri

Re: [HACKERS] Inline Extension

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 10:04 AM, Dimitri Fontaine wrote: > And then basebackup and pg_upgrade would just work, and for dump and > restore we still need to find something not violating the POLA. > > I think that would mean offering a backend function that list all files > from a given extension an

Re: [HACKERS] patch: ALTER TABLE IF EXISTS

2012-01-23 Thread Robert Haas
On Tue, Jan 3, 2012 at 2:49 PM, Pavel Stehule wrote: > jup, we can continue in enhancing step by step. > > I change a patch and now ALTER TABLE, ALTER INDEX, ALTER SEQUENCE and > ALTER VIEW has IF EXISTS clause ALTER FOREIGN TABLE should be parallel as well, I think. -- Robert Haas EnterpriseDB

Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 9:31 AM, Simon Riggs wrote: > Test duration is important for tests that don't relate to pure > contention reduction, which is every patch apart from XLogInsert. Yes, I know. I already said that I was working on more tests to address other use cases. > I'm very happy to s

Re: [HACKERS] Inline Extension

2012-01-23 Thread Dimitri Fontaine
Robert Haas writes: > I am pretty concerned that we find a design that does not involve > pg_dump needing to dump out the extension contents, though: that seems > to me to be missing the point of having extensions in the first place. I was just trying to explain where I'm coming from, I'm not wed

Re: [HACKERS] patch: ALTER TABLE IF EXISTS

2012-01-23 Thread Simon Riggs
On Tue, Jan 3, 2012 at 7:49 PM, Pavel Stehule wrote: > I change a patch and now ALTER TABLE, ALTER INDEX, ALTER SEQUENCE and > ALTER VIEW has IF EXISTS clause Patch no longer applies. Pls update. --  Simon Riggs   http://www.2ndQuadrant.com/  PostgreSQL Development, 24x7 Suppor

GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread Marko Kreen
On Sun, Jan 22, 2012 at 11:47 PM, Mikko Tiihonen wrote: > * introduced a new GUC variable array_output copying the current >  bytea_output type, with values "full" (old value) and >  "smallfixed" (new default) > * added documentation for the new GUC variable If this variable changes protocol-leve

Re: [HACKERS] PG-Strom - A GPU optimized asynchronous executor module

2012-01-23 Thread Simon Riggs
On Mon, Jan 23, 2012 at 2:49 PM, Kohei KaiGai wrote: >> Also, the query you mention is probably the best performing query you >> can come up with. It looks like a GIS query, yet isn't. Would it be >> possible to run tests on the TPC-H suite and do a full comparison of >> strengths/weaknesses so w

Re: [HACKERS] PG-Strom - A GPU optimized asynchronous executor module

2012-01-23 Thread Kohei KaiGai
2012/1/23 Simon Riggs : > On Sun, Jan 22, 2012 at 3:48 PM, Kohei KaiGai wrote: > >> I tried to implement a fdw module that is designed to utilize GPU >> devices to execute >> qualifiers of sequential-scan on foreign tables managed by this module. >> >> It was named PG-Strom, and the following wiki

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Florian Weimer
* Robert Haas: >> In this particular case, I knew that the change was coming and could >> push updated Java and Perl client libraries well before the server-side >> change hit our internal repository, but I really don't want to have to >> pay attention to such details. > > But if we *don't* turn t

Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-01-23 Thread Simon Riggs
On Mon, Jan 23, 2012 at 1:53 PM, Robert Haas wrote: > Results are the median of three five-minute test runs > checkpoint_timeout = 15min Test duration is important for tests that don't relate to pure contention reduction, which is every patch apart from XLogInsert. We've discussed that before,

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 4:03 AM, Florian Weimer wrote: > * Ants Aasma: >> I had a run in with this. JDBC driver versions < 9.0 with the default >> configuration resulted in silent data corruption. The fix was easy, but not >> having an useful error was what really bothered me. > > Same for the DBD

Re: [HACKERS] Inline Extension

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 8:25 AM, Dimitri Fontaine wrote: > Robert Haas writes: >> "virtual directory" - e.g. CREATE TABLE pg_extension_virtualdir >> (filename text, content text) which would be modifiable by the DBA and >> would be searched either before or after the filesystem itself.  This >> c

[HACKERS] basic pgbench runs with various performance-related patches

2012-01-23 Thread Robert Haas
There was finally some time available on Nate Boley's server, which he has been kind enough to make highly available for performance testing throughout this cycle, and I got a chance to run some benchmarks against a bunch of the perfomance-related patches in the current CommitFest. Specifically, I

Re: [HACKERS] PG-Strom - A GPU optimized asynchronous executor module

2012-01-23 Thread Simon Riggs
On Sun, Jan 22, 2012 at 3:48 PM, Kohei KaiGai wrote: > I tried to implement a fdw module that is designed to utilize GPU > devices to execute > qualifiers of sequential-scan on foreign tables managed by this module. > > It was named PG-Strom, and the following wikipage gives a brief > overview of

Re: [HACKERS] Inline Extension

2012-01-23 Thread Dimitri Fontaine
Robert Haas writes: > "virtual directory" - e.g. CREATE TABLE pg_extension_virtualdir > (filename text, content text) which would be modifiable by the DBA and > would be searched either before or after the filesystem itself. This > catalog wouldn't be dumped by pg_dump, and there would be no chan

Re: Removing freelist (was Re: [HACKERS] Should I implement DROP INDEX CONCURRENTLY?)

2012-01-23 Thread Simon Riggs
On Mon, Jan 23, 2012 at 1:06 PM, Robert Haas wrote: > It's pretty trivial to prove that there is a very serious problem with > BufFreelistLock.  I'll admit I can't prove what the right fix is just > yet, and certainly measurement is warranted. I agree there is a problem with BufFreelistLock (so

Re: [HACKERS] Online base backup from the hot-standby

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 8:11 AM, Robert Haas wrote: > On Mon, Jan 23, 2012 at 5:29 AM, Fujii Masao wrote: >> If many people think the patch is not acceptable without such a safeguard, >> I will do that right now. > > That's my view.  I think we ought to resolve this issue before commit, > especia

Re: [HACKERS] Online base backup from the hot-standby

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 5:29 AM, Fujii Masao wrote: > If many people think the patch is not acceptable without such a safeguard, > I will do that right now. That's my view. I think we ought to resolve this issue before commit, especially since it seems unclear that we know how to fix it. -- Ro

Re: [HACKERS] Online base backup from the hot-standby

2012-01-23 Thread Simon Riggs
On Mon, Jan 23, 2012 at 10:29 AM, Fujii Masao wrote: > On Fri, Jan 20, 2012 at 11:34 PM, Simon Riggs wrote: >> On Fri, Jan 20, 2012 at 12:54 PM, Fujii Masao wrote: >>> Thanks for the review! >>> >>> On Fri, Jan 20, 2012 at 8:15 PM, Simon Riggs wrote: I'm looking at this patch and wondering

Re: [HACKERS] PG-Strom - A GPU optimized asynchronous executor module

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 1:38 AM, Kohei KaiGai wrote: > What options are available to see rate of workloads of components > within a particular query? I usually use oprofile, though I'm given to understand it's been superseded by a new tool called perf. I haven't had a chance to experiment with p

Re: Removing freelist (was Re: [HACKERS] Should I implement DROP INDEX CONCURRENTLY?)

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 12:12 AM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Jan 21, 2012 at 5:29 PM, Jim Nasby wrote: >>> We should also look at having the freelist do something useful, instead of >>> just dropping it completely. Unfortunately that's probably more work... > >> That's kin

Re: [HACKERS] WAL Restore process during recovery

2012-01-23 Thread Simon Riggs
On Mon, Jan 23, 2012 at 12:23 PM, Fujii Masao wrote: > I've not reviewed the patch enough yet. Will review the patch tomorrow again. Thanks very much. I'm sure that's enough to keep me busy a few days. --  Simon Riggs   http://www.2ndQuadrant.com/  PostgreSQL Development, 24x7

Re: [HACKERS] Inline Extension

2012-01-23 Thread Robert Haas
On Mon, Jan 23, 2012 at 5:53 AM, Dimitri Fontaine wrote: >> Probably the worst issue with that is that in typical installations, >> the share/extension/ directory would be read-only to the server, and a >> lot of people might be uncomfortable with making it writable.  Not sure >> whether we should

Re: [HACKERS] New replication mode: write

2012-01-23 Thread Simon Riggs
On Mon, Jan 23, 2012 at 10:03 AM, Fujii Masao wrote: >>> To make the walreceiver call WaitLatchOrSocket(), we would need to >>> merge it and libpq_select() into one function. But the former is the backend >>> function and the latter is the frontend one. Now I have no good idea to >>> merge them c

Re: [HACKERS] WAL Restore process during recovery

2012-01-23 Thread Fujii Masao
On Fri, Jan 20, 2012 at 7:50 PM, Fujii Masao wrote: > On Fri, Jan 20, 2012 at 7:38 PM, Simon Riggs wrote: >> On Fri, Jan 20, 2012 at 3:43 AM, Fujii Masao wrote: >> >> Requested update > > Thanks! Will review. In StartChildProcess(), the code which emits an error when fork of walrestore fails is

Re: [HACKERS] Multithread Query Planner

2012-01-23 Thread Yeb Havinga
On 2012-01-13 21:14, Frederico wrote: Hi folks. Is there any restriction in create and start threads inside Postgres? I'm trying to develop a multithread planner, and some times is raised a exception of access memory. I'm debugging the code to see if is a bug in the planner, but until now, I

Re: [HACKERS] Patch: Allow SQL-language functions to reference parameters by parameter name

2012-01-23 Thread Hitoshi Harada
On Thu, Jan 19, 2012 at 1:58 AM, Hitoshi Harada wrote: > On Wed, Jan 18, 2012 at 1:11 AM, Hitoshi Harada wrote: >> On Sat, Jan 14, 2012 at 8:10 AM, Matthew Draper wrote: >>> >>> I just remembered to make time to advance this from WIP to proposed >>> patch this week... and then worked out I'm rud

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-23 Thread Marko Kreen
On Sat, Jan 21, 2012 at 1:52 PM, Marc Mamin wrote: >> > >> > c. Refine error handling of dblink.c. I think it preserves the >> >    previous behavior for column number mismatch and type >> >    conversion exception. > > Hello, > > I don't know if this cover following issue. > I just mention it for

Re: [HACKERS] Finer Extension dependencies

2012-01-23 Thread Hitoshi Harada
On Mon, Jan 23, 2012 at 2:00 AM, Dimitri Fontaine wrote: > Hitoshi Harada writes: - What happens if DROP EXTENSION ... CASCADE? Does it work? >>> >>> It should, what happens when you try? :) >> >> I just tried DROP EXTENSION now, and found it broken :( >> >> db1=# create extension kmeans; >>

Re: [HACKERS] Inline Extension

2012-01-23 Thread Cédric Villemain
Le 23 janvier 2012 11:53, Dimitri Fontaine a écrit : > Tom Lane writes: >> On reflection it seems like this patch is simply offering the wrong >> solution for the problem.  I agree that it could be useful to install >> extensions without having direct access to the server's filesystem, >> but it

[HACKERS] database schema quality survey & postgresql pitfalls

2012-01-23 Thread Fabien COELHO
Dear pgdevs, I've just completed the final version of a survey of database schema quality in open source software. The survey covers 512 projects which use MySQL and/or PostgreSQL for storing their data. Automatic analyses are performed by querying the information schema. Statistical validations

Re: [HACKERS] Inline Extension

2012-01-23 Thread Dimitri Fontaine
Tom Lane writes: > On reflection it seems like this patch is simply offering the wrong > solution for the problem. I agree that it could be useful to install > extensions without having direct access to the server's filesystem, > but it doesn't seem to follow that we must lobotomize existing exte

Re: [HACKERS] Inline Extension

2012-01-23 Thread Cédric Villemain
> Is it such a bad idea to store the literal text of the extension's > pieces (control file and corresponding SQL program) in catalogs?  I'm > not sure if I understand why everyone is so interested in a special > interaction with the file system in some way.  By the same token, > extensions can be

Re: [HACKERS] Online base backup from the hot-standby

2012-01-23 Thread Fujii Masao
On Fri, Jan 20, 2012 at 11:34 PM, Simon Riggs wrote: > On Fri, Jan 20, 2012 at 12:54 PM, Fujii Masao wrote: >> Thanks for the review! >> >> On Fri, Jan 20, 2012 at 8:15 PM, Simon Riggs wrote: >>> I'm looking at this patch and wondering why we're doing so many >>> press-ups to ensure full_page_wr

Re: [HACKERS] pgstat wait timeout

2012-01-23 Thread pratikchirania
Hi, Any ideas on this? -- View this message in context: http://postgresql.1045698.n5.nabble.com/pgstat-wait-timeout-tp5078125p5165651.html Sent from the PostgreSQL - hackers mailing list archive at Nabble.com. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make change

Re: [HACKERS] New replication mode: write

2012-01-23 Thread Fujii Masao
On Mon, Jan 23, 2012 at 6:28 PM, Simon Riggs wrote: > On Mon, Jan 23, 2012 at 9:02 AM, Fujii Masao wrote: >> On Mon, Jan 23, 2012 at 4:58 PM, Simon Riggs wrote: >>> On Mon, Jan 16, 2012 at 12:45 PM, Fujii Masao wrote: >>> >> Please add the Apply mode. > > OK, will do. Done

Re: [HACKERS] Finer Extension dependencies

2012-01-23 Thread Dimitri Fontaine
Hitoshi Harada writes: >>>     "pg_extension_feature_index" UNIQUE, btree (extoid, extfeature) > Do you mean you want UNIQUE constraint by this index? I found the > usage is to search feature by (only) its name, so I wondered if extoid > is not necessary. I guess you're right and that's something

Re: [HACKERS] Finer Extension dependencies

2012-01-23 Thread Hitoshi Harada
On Sat, Jan 21, 2012 at 9:20 AM, Dimitri Fontaine wrote: > Hi, > > Thank you for reviewing this patch! > > Hitoshi Harada writes: >> The patch applies with one reject, which I could fix easily. The make >> check passed. > > Bitrot happens fast in this season…  will produce another version of the

Re: [HACKERS] New replication mode: write

2012-01-23 Thread Simon Riggs
On Mon, Jan 23, 2012 at 9:02 AM, Fujii Masao wrote: > On Mon, Jan 23, 2012 at 4:58 PM, Simon Riggs wrote: >> On Mon, Jan 16, 2012 at 12:45 PM, Fujii Masao wrote: >> > Please add the Apply mode. OK, will do. >>> >>> Done. Attached is the updated version of the patch. >> >> I notice

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Florian Weimer
* Ants Aasma: > I had a run in with this. JDBC driver versions < 9.0 with the default > configuration resulted in silent data corruption. The fix was easy, but not > having an useful error was what really bothered me. Same for the DBD::Pg driver. In this particular case, I knew that the change w

Re: [HACKERS] New replication mode: write

2012-01-23 Thread Fujii Masao
On Mon, Jan 23, 2012 at 4:58 PM, Simon Riggs wrote: > On Mon, Jan 16, 2012 at 12:45 PM, Fujii Masao wrote: > Please add the Apply mode. >>> >>> OK, will do. >> >> Done. Attached is the updated version of the patch. > > I notice that the Apply mode isn't fully implemented. I had in mind > tha

Re: [HACKERS] Inline Extension

2012-01-23 Thread Daniel Farina
On Sun, Jan 22, 2012 at 8:42 PM, Robert Haas wrote: > On Sun, Jan 22, 2012 at 3:20 PM, Daniel Farina wrote: >> A few anecdotes does not constitute evidence, but it does look like >> some people pay attention to any additional versioning foothold they >> can get. > > Sure, but just because some pe

Re: [HACKERS] Vacuum rate limit in KBps

2012-01-23 Thread Benedikt Grundmann
On 19/01/12 17:39, Greg Smith wrote: > On 1/19/12 1:10 PM, Robert Haas wrote: > >I have to say that I find that intensely counterintuitive. The > >current settings are not entirely easy to tune correctly, but at least > >they're easy to explain. > > If there's anyone out there who has run a large