Re: [HACKERS] Enabling Checksums

2013-03-08 Thread Heikki Linnakangas
On 08.03.2013 05:31, Bruce Momjian wrote: Also, don't all modern storage drives have built-in checksums, and report problems to the system administrator? Does smartctl help report storage corruption? Let me take a guess at answering this --- we have several layers in a database server:

Re: [HACKERS] Enabling Checksums

2013-03-08 Thread Heikki Linnakangas
On 07.03.2013 23:45, Jeff Davis wrote: By the way, I can not find any trace of XLogCheckBufferNeedsBackup(), was that a typo? Ah, sorry, that was a new function introduced by another patch I was reviewing at the same time, and I conflated the two. - Heikki -- Sent via pgsql-hackers mailing

[HACKERS] Re: [BUGS] BUG #7873: pg_restore --clean tries to drop tables that don't exist

2013-03-08 Thread Pavel Stehule
2013/3/8 Josh Kupershmidt : > [Moving to -hackers] > > On Sat, Feb 23, 2013 at 2:51 PM, Pavel Stehule > wrote: > >> so >> >> * --conditional-drops replaced by --if-exists > > Thanks for the fixes, I played around with the patch a bit. I was sort > of expecting this example to work (after setting

Re: [HACKERS] REFRESH MATERIALIZED VIEW locklevel

2013-03-08 Thread Nicolas Barbier
2013/3/8 Andres Freund : > On 2013-03-07 15:21:35 -0800, Josh Berkus wrote: > >> This limitation is in no way crippling for this feature, or even a major >> detraction. I still intend to promote the heck out of this feature. > > Thats scaring me. Because the current state of the feature isn't > s

Re: [HACKERS] [v9.3] OAT_POST_ALTER object access hooks

2013-03-08 Thread Kohei KaiGai
Thanks for your reviewing. 2013/3/7 Robert Haas : > On Sun, Jan 27, 2013 at 1:55 AM, Kohei KaiGai wrote: >> The part-2 patch adds new OAT_POST_ALTER event type, and >> its relevant permission checks on contrib/sepgsql. > > This documentation hunk is unclear: > > +On , install permission > +

Re: [HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

2013-03-08 Thread Heikki Linnakangas
On 03.03.2013 19:42, Alexander Korotkov wrote: This patch only adds one more operator to already committed new opclass. Tests already cover this case. Without patch corresponding test leads to sequential scan instead of index scan. Thanks, committed with some trivial cleanup. However, I can't

Re: [HACKERS] sql_drop Event Triggerg

2013-03-08 Thread Robert Haas
On Tue, Mar 5, 2013 at 12:45 PM, Alvaro Herrera wrote: > Hmm, maybe I should be considering a pair of macros instead -- > UTILITY_START_DROP and UTILITY_END_DROP. I'll give this a try. Other > ideas are welcome. That seems like a possibly promising idea. I do wonder how well any of this is goi

Re: [HACKERS] Index Unqiueness

2013-03-08 Thread Robert Haas
On Tue, Mar 5, 2013 at 10:41 PM, abhinav batra wrote: > Hey > I want to work towards the follwing feature in TODO list: > Prevent index uniqueness checks when UPDATE does not modify the > columnUniqueness (index) checks are done when updating a column even if the > column is not modified by the UP

Re: [HACKERS] Index Unqiueness

2013-03-08 Thread Andres Freund
On 2013-03-06 09:11:38 +0530, abhinav batra wrote: > Hey > I want to work towards the follwing feature in TODO list: > Prevent index uniqueness checks when UPDATE does not modify the > columnUniqueness > (index) checks are done when updating a column even if the column is not > modified by the UPDA

Re: [HACKERS] Materialized views WIP patch

2013-03-08 Thread Bruce Momjian
On Wed, Mar 6, 2013 at 09:16:59AM -0500, Tom Lane wrote: > Simon Riggs writes: > > On 5 March 2013 22:02, Tom Lane wrote: > >> FWIW, my opinion is that doing anything like this in the planner is > >> going to be enormously expensive. > > > As we already said: no MVs => zero overhead => no probl

Re: [HACKERS] sql_drop Event Triggerg

2013-03-08 Thread Alvaro Herrera
Robert Haas escribió: > On Tue, Mar 5, 2013 at 12:45 PM, Alvaro Herrera > wrote: > > Hmm, maybe I should be considering a pair of macros instead -- > > UTILITY_START_DROP and UTILITY_END_DROP. I'll give this a try. Other > > ideas are welcome. > > That seems like a possibly promising idea. I d

Re: [HACKERS] Parameterized paths vs index clauses extracted from OR clauses

2013-03-08 Thread Robert Haas
On Tue, Mar 5, 2013 at 11:53 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Mar 5, 2013 at 3:44 PM, Tom Lane wrote: >>> If "foo OR bar" is useful as an indexqual condition in the inner scan, >>> that's one thing. But if it isn't, the cycles expended to check it in >>> the inner scan are p

Re: [HACKERS] Materialized views WIP patch

2013-03-08 Thread Bruce Momjian
On Tue, Mar 5, 2013 at 08:50:39AM +, Simon Riggs wrote: > Its not a different feature, its what most people expect a feature > called MV to deliver. That's not a matter of opinion, its simply how > every other database works currently - Oracle, Teradata, SQLServer at > least. The fact that we

Re: [HACKERS] Index Unqiueness

2013-03-08 Thread Robert Haas
On Fri, Mar 8, 2013 at 8:58 AM, Andres Freund wrote: > I'd be interested in something slightly related that is doing HOT on a > per-index basis. Currently we don't do hot if any index is updated even > though quite possibly most of the indexes don't change. > I think that might result in some quit

Re: [HACKERS] Materialized views WIP patch

2013-03-08 Thread Robert Haas
On Thu, Mar 7, 2013 at 12:14 PM, David E. Wheeler wrote: > On Mar 7, 2013, at 7:55 AM, Kevin Grittner wrote: >>> If the answer to both those questions is “yes,” I think the term >>> should remain “table,” with a few mentions that the term includes >>> materialized views (and excludes foreign tabl

Re: [HACKERS] sql_drop Event Triggerg

2013-03-08 Thread Robert Haas
On Fri, Mar 8, 2013 at 9:18 AM, Alvaro Herrera wrote: > Robert Haas escribió: >> On Tue, Mar 5, 2013 at 12:45 PM, Alvaro Herrera >> wrote: >> > Hmm, maybe I should be considering a pair of macros instead -- >> > UTILITY_START_DROP and UTILITY_END_DROP. I'll give this a try. Other >> > ideas are

[HACKERS] Re: [BUGS] BUG #7873: pg_restore --clean tries to drop tables that don't exist

2013-03-08 Thread Josh Kupershmidt
On Fri, Mar 8, 2013 at 2:23 AM, Pavel Stehule wrote: > 2013/3/8 Josh Kupershmidt : >> Cool. I think it would also be useful to check that --clean may only >> be used with --format=p to avoid any confusion there. (This issue >> could be addressed in a separate patch if you'd rather not lard this >

Re: [HACKERS] sql_drop Event Triggerg

2013-03-08 Thread Alvaro Herrera
Robert Haas escribió: > On Fri, Mar 8, 2013 at 9:18 AM, Alvaro Herrera > wrote: > > Robert Haas escribió: > >> On Tue, Mar 5, 2013 at 12:45 PM, Alvaro Herrera > >> wrote: > >> > Hmm, maybe I should be considering a pair of macros instead -- > >> > UTILITY_START_DROP and UTILITY_END_DROP. I'll g

Re: [HACKERS] Support for REINDEX CONCURRENTLY

2013-03-08 Thread Fujii Masao
On Fri, Mar 8, 2013 at 10:00 PM, Michael Paquier wrote: > > > On Fri, Mar 8, 2013 at 1:41 AM, Fujii Masao wrote: >> >> On Thu, Mar 7, 2013 at 7:19 AM, Andres Freund >> wrote: >> > The strange think about "hoge_pkey_cct_cct" is that it seems to imply >> > that an invalid index was reindexed concu

[HACKERS] Re: [BUGS] BUG #7873: pg_restore --clean tries to drop tables that don't exist

2013-03-08 Thread Pavel Stehule
2013/3/8 Josh Kupershmidt : > On Fri, Mar 8, 2013 at 2:23 AM, Pavel Stehule wrote: >> 2013/3/8 Josh Kupershmidt : > >>> Cool. I think it would also be useful to check that --clean may only >>> be used with --format=p to avoid any confusion there. (This issue >>> could be addressed in a separate pa

Re: [HACKERS] [DOCS] Contrib module "xml2" status

2013-03-08 Thread Bruce Momjian
On Fri, Feb 22, 2013 at 08:25:31AM +0900, Ian Lawrence Barwick wrote: > > The point is we can > > remove the module when someone fixes and replaces the functionality that's > > left in there that some people rely on. So far nobody has stepped up to the > > plate, although now that we have lateral a

Re: [HACKERS] Enabling Checksums

2013-03-08 Thread Josh Berkus
> I also see the checksum patch is taking a beating. I wanted to step > back and ask what pertntage of known corruptions cases will this > checksum patch detect? I'm pretty sure that early on Jeff posted some statstics which indicated that the current approach would detect 99% of corruption intr

Re: [HACKERS] Identity projection

2013-03-08 Thread Heikki Linnakangas
On 12.02.2013 11:03, Amit Kapila wrote: + /* + * equivalent_tlists + * returns whether two traget lists are equivalent + * + * We consider two target lists equivalent if both have + * only Var entries and resjunk of each target entry is same. + * + * This function is used to decide wh

Re: [HACKERS] REFRESH MATERIALIZED VIEW locklevel

2013-03-08 Thread Josh Berkus
Andres, Crossing this over to pgsql-advocacy, because this is really an Advocacy discussion. > The point is that > a) refreshing is the only way to update materialized views. There's no >incremental support. > b) refreshing will take a long time (otherwise you wouldn't have >create a mate

Re: [HACKERS] REFRESH MATERIALIZED VIEW locklevel

2013-03-08 Thread Merlin Moncure
On Fri, Mar 8, 2013 at 11:59 AM, Josh Berkus wrote: > Andres, > > Crossing this over to pgsql-advocacy, because this is really an Advocacy > discussion. > >> The point is that >> a) refreshing is the only way to update materialized views. There's no >>incremental support. >> b) refreshing will

Re: [HACKERS] REFRESH MATERIALIZED VIEW locklevel

2013-03-08 Thread Joshua D. Drake
On 03/08/2013 10:09 AM, Merlin Moncure wrote: On Fri, Mar 8, 2013 at 11:59 AM, Josh Berkus wrote: Andres, Further, we get pretty much one and only one chance to promote a new major feature, which is when that feature is first introduced. Improving the feature in the next version of Postgre

Re: [HACKERS] Why do we still perform a check for pre-sorted input within qsort variants?

2013-03-08 Thread Bruce Momjian
On Mon, Feb 25, 2013 at 02:31:21PM +, Peter Geoghegan wrote: > On 25 February 2013 11:49, Robert Haas wrote: > > I did attempt to do some tinkering with this while I was playing with > > it, but I didn't come up with anything really compelling. You can > > reduce the number of comparisons on

Re: [HACKERS] Why do we still perform a check for pre-sorted input within qsort variants?

2013-03-08 Thread Dann Corbit
>-Original Message- >From: pgsql-hackers-ow...@postgresql.org >[mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Bruce Momjian >Sent: Friday, March 08, 2013 11:22 AM >To: Peter Geoghegan >Cc: Robert Haas; Tom Lane; PG Hackers >Subject: Re: [HACKERS] Why do we still perform a check f

Re: [HACKERS] Why do we still perform a check for pre-sorted input within qsort variants?

2013-03-08 Thread 'Bruce Momjian'
On Fri, Mar 8, 2013 at 07:43:10PM +, Dann Corbit wrote: > I seem to recall that a year or two back some study was done on > quicksort methodology as used in PostgreSQL. As I recall, the > algorithm used in PostgreSQL fared well in the tests. Well, that's good to hear. -- Bruce Momjian

Re: [HACKERS] Why do we still perform a check for pre-sorted input within qsort variants?

2013-03-08 Thread Peter Geoghegan
On 8 March 2013 11:48, Bruce Momjian wrote: > On Fri, Mar 8, 2013 at 07:43:10PM +, Dann Corbit wrote: >> I seem to recall that a year or two back some study was done on >> quicksort methodology as used in PostgreSQL. As I recall, the >> algorithm used in PostgreSQL fared well in the tests. >

Re: [HACKERS] Why do we still perform a check for pre-sorted input within qsort variants?

2013-03-08 Thread Dann Corbit
-Original Message- From: Peter Geoghegan [mailto:peter.geoghega...@gmail.com] Sent: Friday, March 08, 2013 12:00 PM To: Bruce Momjian Cc: Dann Corbit; Robert Haas; Tom Lane; PG Hackers Subject: Re: [HACKERS] Why do we still perform a check for pre-sorted input within qsort variants? On 8

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Robert Haas
On Thu, Mar 7, 2013 at 2:48 PM, David E. Wheeler wrote: > In the spirit of being liberal about what we accept but strict about what we > store, it seems to me that JSON object key uniqueness should be enforced > either by throwing an error on duplicate keys, or by flattening so that the > lates

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Hannu Krosing
On 03/08/2013 09:39 PM, Robert Haas wrote: On Thu, Mar 7, 2013 at 2:48 PM, David E. Wheeler wrote: In the spirit of being liberal about what we accept but strict about what we store, it seems to me that JSON object key uniqueness should be enforced either by throwing an error on duplicate key

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Alvaro Herrera
Hannu Krosing escribió: > On 03/08/2013 09:39 PM, Robert Haas wrote: > >On Thu, Mar 7, 2013 at 2:48 PM, David E. Wheeler > >wrote: > >>In the spirit of being liberal about what we accept but strict about what > >>we store, it seems to me that JSON object key uniqueness should be enforced > >>ei

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread David E. Wheeler
On Mar 8, 2013, at 1:01 PM, Alvaro Herrera wrote: >> If it does not meet these "semantic" constraints, then it is not >> really JSON - it is merely JSON-like. >> >> this sounds very much like MySQLs decision to support timestamp >> "-00-00 00:00" - syntactically correct, but semantically wro

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Gavin Flower
Well I would much prefer to find out sooner rather than later that there is a problem, so I would much prefer know I've created a duplicate as soon as the system can detect it. In general, Postgresql appears much better at this than MySQL On 09/03/13 10:01, Alvaro Herrera wrote: Hannu Krosi

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Andrew Dunstan
On 03/08/2013 04:01 PM, Alvaro Herrera wrote: Hannu Krosing escribió: On 03/08/2013 09:39 PM, Robert Haas wrote: On Thu, Mar 7, 2013 at 2:48 PM, David E. Wheeler wrote: In the spirit of being liberal about what we accept but strict about what we store, it seems to me that JSON object key un

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread David E. Wheeler
On Mar 8, 2013, at 1:21 PM, Andrew Dunstan wrote: > Here's what rfc2119 says about that wording: > > 4. SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that > there may exist valid reasons in particular circumstances when the > particular behavior is acceptable or even useful,

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Hannu Krosing
On 03/08/2013 10:01 PM, Alvaro Herrera wrote: Hannu Krosing escribió: On 03/08/2013 09:39 PM, Robert Haas wrote: On Thu, Mar 7, 2013 at 2:48 PM, David E. Wheeler wrote: In the spirit of being liberal about what we accept but strict about what we store, it seems to me that JSON object key uni

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Andrew Dunstan
On 03/08/2013 04:28 PM, David E. Wheeler wrote: On Mar 8, 2013, at 1:21 PM, Andrew Dunstan wrote: Here's what rfc2119 says about that wording: 4. SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the part

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Hannu Krosing
On 03/08/2013 10:42 PM, Andrew Dunstan wrote: On 03/08/2013 04:28 PM, David E. Wheeler wrote: On Mar 8, 2013, at 1:21 PM, Andrew Dunstan wrote: Here's what rfc2119 says about that wording: 4. SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reaso

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Andrew Dunstan
On 03/08/2013 04:42 PM, Andrew Dunstan wrote: So my order of preference for the options would be: 1. Have the JSON type collapse objects so the last instance of a key wins and is actually stored 2. Throw an error when a JSON type has duplicate keys 3. Have the accessors find the last ins

[HACKERS] pg_dump selectively ignores extension configuration tables

2013-03-08 Thread Joe Conway
(reposting" apparently I used a verboten word the first time around . Sorry for any duplicates) The -t and -n options of pg_dump do not dump anything from an extension configuration table, whereas normal pg_dump will dump the user data. To see what I mean, in psql do (tested on pg9.2): 8<---

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Josh Berkus
> Actually, now I think more about it 3 is the best answer. Here's why: > even the JSON generators can produce JSON with non-unique field names: +1 Also, I think we should add a json_normalize() function to the TODO list. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent v

Re: [HACKERS] scanner/parser minimization

2013-03-08 Thread Bruce Momjian
On Thu, Feb 28, 2013 at 04:09:11PM -0500, Tom Lane wrote: > Robert Haas writes: > > A whole lot of those state transitions are attributable to states > > which have separate transitions for each of many keywords. > > Yeah, that's no surprise. > > The idea that's been in the back of my mind for a

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Hannu Krosing
On 03/08/2013 11:03 PM, Andrew Dunstan wrote: On 03/08/2013 04:42 PM, Andrew Dunstan wrote: So my order of preference for the options would be: 1. Have the JSON type collapse objects so the last instance of a key wins and is actually stored 2. Throw an error when a JSON type has duplicat

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Andrew Dunstan
On 03/08/2013 06:37 PM, Hannu Krosing wrote: I suspect that 99.98% of the time we will get valid and unique JS Object serializations or equivalent as input to json_in() If we want the getter functions to handle the "loose JSON" to Object conversion side assuming our stored JSON can conta

Re: [HACKERS] Index Unqiueness

2013-03-08 Thread Bruce Momjian
On Fri, Mar 8, 2013 at 10:26:21AM -0500, Robert Haas wrote: > overhead seems badly overpriced for insert-only tables. These are not > fundamental truths of the universe, or even of PostgreSQL; they are > specific consequences of the representation we've chosen for heaps. > Many of them are things

Re: [HACKERS] Enabling Checksums

2013-03-08 Thread Greg Stark
On Fri, Mar 8, 2013 at 5:46 PM, Josh Berkus wrote: > After some examination of the systems involved, we conculded that the > issue was the FreeBSD drivers for the new storage, which were unstable > and had custom source patches. However, without PostgreSQL checksums, > we couldn't *prove* it wasn

Re: [HACKERS] Why do we still perform a check for pre-sorted input within qsort variants?

2013-03-08 Thread Greg Stark
On Fri, Mar 8, 2013 at 7:43 PM, Dann Corbit wrote: > Checking for pre-sorted input will not make the routine faster on average. > However, it prevents a common perverse case where the runtime goes quadratic, > so sorting 10^6 elements will take K*10^12th operations when the bad > condition does

Re: [HACKERS] [pgsql-advocacy] Call for Google Summer of Code mentors, admins

2013-03-08 Thread Josh Berkus
Thom. > I don't mind being an admin again. Can you gather together all of the projects suggested on this thread and use them to create updated text for the GSOC page? If you don't have web repo access, I can create a patch, but if you can do the text, that would be a big help. -- Josh Berkus P

Re: [HACKERS] Enabling Checksums

2013-03-08 Thread Greg Smith
On 3/8/13 3:38 AM, Heikki Linnakangas wrote: See https://www.kernel.org/doc/Documentation/block/data-integrity.txt That includes an interesting comment that's along the lines of the MySQL checksum tests already mentioned: "The 16-bit CRC checksum mandated by both the SCSI and SATA specs is s

Re: [HACKERS] Duplicate JSON Object Keys

2013-03-08 Thread Noah Misch
On Fri, Mar 08, 2013 at 10:34:20PM +0100, Hannu Krosing wrote: > On 03/08/2013 10:01 PM, Alvaro Herrera wrote: >> Hannu Krosing escribi?: >>> If it does not meet these "semantic" constraints, then it is not >>> really JSON - it is merely JSON-like. >> Is it wrong? The standard cited says SHOULD,

Re: [HACKERS] Request for vote to move forward with recovery.conf overhaul

2013-03-08 Thread Michael Paquier
On Wed, Mar 6, 2013 at 2:08 PM, Michael Paquier wrote: > Thanks for taking time in typing a complete summary of the situation. That > really helps. > > On Mon, Mar 4, 2013 at 11:25 AM, Greg Smith wrote: > >> On 1/23/13 6:36 AM, Michael Paquier wrote: >> >>> The only problem I see is if the same p

Re: [HACKERS] Request for vote to move forward with recovery.conf overhaul

2013-03-08 Thread Alvaro Herrera
Michael Paquier escribió: > On Wed, Mar 6, 2013 at 2:08 PM, Michael Paquier > wrote: > > Looks good to me too. > > Based on the patch I already sent before, there are a couple of things > > missing: > > - There are no pg_ctl standby/recover commands implemented yet (no that > > difficult to add) >

Re: [HACKERS] Request for vote to move forward with recovery.conf overhaul

2013-03-08 Thread Greg Smith
On 3/8/13 10:34 PM, Michael Paquier wrote: This patch is still in the current commit fest. Any objections in marking it as returned with feedback and put it in the next commit fest? There are currently 20 "Needs Review" and 14 "Waiting on Author" things left in the queue, so it's not quite tha

Re: [HACKERS] Request for vote to move forward with recovery.conf overhaul

2013-03-08 Thread Michael Paquier
On Sat, Mar 9, 2013 at 1:20 PM, Greg Smith wrote: > There are currently 20 "Needs Review" and 14 "Waiting on Author" things > left in the queue, so it's not quite that there's no time left. There > really isn't very much left to do on this. The rough consensus idea from > before takes a while t

Re: [HACKERS] Fix pgstattuple/pgstatindex to use regclass-type as the argument

2013-03-08 Thread Satoshi Nagayasu
(2013/03/05 22:46), Robert Haas wrote: On Sun, Mar 3, 2013 at 5:32 PM, Tom Lane wrote: Maybe this is acceptable collateral damage. I don't know. But we definitely stand a chance of breaking applications if we change pgstatindex like this. It might be better to invent a differently-named func