On Tue, 2011-12-20 at 13:22 +0400, Alexander Korotkov wrote:
> Hi!
>
>
> Studying this question little more I found that current approach of
> range indexing can be dramatically inefficient in some cases. It's not
> because of penalty or split implementation, but because of approach
> itself. Map
On Wed, 2011-12-14 at 01:04 +0400, Alexander Korotkov wrote:
> Hi!
>
Thank you! Attached a few changes:
* Change "ordinal" to "normal" for clarity (at least to me).
* Some comment cleanup
* Change classes_groups to be an enum of SPLIT_LEFT and SPLIT_RIGHT,
rather than using 1 and 2.
* Changed the
On 22.12.2011 01:43, Tom Lane wrote:
A "utility to bump the page version" is equally a whole lot easier said
than done, given that the new version has more overhead space and thus
less payload space than the old. What does it do when the old page is
too full to be converted? "Move some data som
On Thu, Dec 22, 2011 at 12:28 AM, Robert Haas wrote:
> But on the flip side, I feel like your discussion of the problems is a
> bit hand-wavy. I think we need some real test cases that we can look
> at and measure, not just an informal description of what we think is
> happening.
I understand w
On Wed, Dec 21, 2011 at 12:04 PM, Magnus Hagander wrote:
> On Tue, Dec 20, 2011 at 18:15, Simon Riggs wrote:
>> On Tue, Dec 20, 2011 at 1:40 PM, Magnus Hagander wrote:
>>> These days we have pause_at_recovery_target, which lets us pause when
>>> we reach a PITR target. Is there a particular reas
This may be ambitious, but it'd be neat if PostgreSQL supported
parameterizable types. For example, suppose a contrib module defines
a "pair" type. It could be used as follows:
CREATE TABLE my_table (
coord pair(float, float)
);
The "pair" module could define functions like thes
On Thu, Dec 22, 2011 at 12:06 AM, Simon Riggs wrote:
>> Having two different page formats running around in the system at the
>> same time is far from free; in the worst case it means that every single
>> piece of code that touches pages has to know about and be prepared to
>> cope with both vers
On Wed, Dec 21, 2011 at 04:18:33PM -0800, Rob Wultsch wrote:
> On Wed, Dec 21, 2011 at 1:59 PM, David Fetter wrote:
> > One of the things VMware is working on is double writes, per
> > previous discussions of how, for example, InnoDB does things.
>
> The world is moving to flash, and the lifetime
On Wed, Dec 21, 2011 at 8:32 PM, Johann 'Myrkraverk' Oskarsson
wrote:
> I mean to create a typed hstore, called tstore for now. I'm open to
> name suggestions. It'll only support a subset of core Postgres types
> to begin with. Keys are always text, it's the value that's typed.
Unfortunately,
Hi all,
I mean to create a typed hstore, called tstore for now. I'm open to
name suggestions. It'll only support a subset of core Postgres types
to begin with. Keys are always text, it's the value that's typed.
Usage is very similar to hstore; this is not a complete reference.
tstore( text,
On Wed, Dec 21, 2011 at 7:06 PM, Simon Riggs wrote:
> My feeling is it probably depends upon how different the formats are,
> so given we are discussing a 4 byte addition to the header, it might
> be doable.
I agree. When thinking back on Zoltan's patches, it's worth
remembering that he had a nu
On Wed, Dec 21, 2011 at 4:17 PM, Simon Riggs wrote:
> Partitioning will give us more buffers and more LWlocks, to spread the
> contention when we access the buffers. I use that word because its
> what we call the technique already used in the buffer manager and lock
> manager. If you wish to call
On Wed, Dec 21, 2011 at 1:59 PM, David Fetter wrote:
> One of the things VMware is working on is double writes, per previous
> discussions of how, for example, InnoDB does things.
The world is moving to flash, and the lifetime of flash is measured
writes. Potentially doubling the number of writes
On Wed, Dec 21, 2011 at 11:43 PM, Tom Lane wrote:
> It seems like you've forgotten all of the previous discussion of how
> we'd manage a page format version change.
Maybe I've had too much caffeine. It's certainly late here.
> Having two different page formats running around in the system at th
Simon Riggs writes:
> We don't need to use any flag bits at all. We add
> PG_PAGE_LAYOUT_VERSION to the control file, so that CRC checking
> becomes an initdb option. All new pages can be created with
> PG_PAGE_LAYOUT_VERSION from the control file. All existing pages must
> be either the layout ve
On Wed, Dec 21, 2011 at 03:16:39PM -0500, Robert Haas wrote:
> On Tue, Dec 20, 2011 at 8:14 PM, Noah Misch wrote:
> > RangeVarCallbackForAlterRelation() does not preserve ALTER TABLE's refusal
> > to
> > operate on foreign tables.
>
> I should probably fix that, but I'm wondering if I ought to f
On Wed, Dec 21, 2011 at 7:35 PM, Greg Smith wrote:
> And there's even more radical changes in btrfs, since it wasn't starting
> with a fairly robust filesystem as a base. And putting my tin foil hat on,
> I don't feel real happy about assuming *the* solution for this issue in
> PostgreSQL is the
David Fetter writes:
> There's a separate issue we'd like to get clear on, which is whether
> it would be OK to make a new PG_PAGE_LAYOUT_VERSION.
If you're not going to provide pg_upgrade support, I think there is no
chance of getting a new page layout accepted. The people who might want
CRC su
On Wed, Dec 21, 2011 at 10:19 PM, Kevin Grittner
wrote:
> Alvaro Herrera wrote:
>
>> If you get away with a new page format, let's make sure and
>> coordinate so that we can add more info into the header. One
>> thing I wanted was to have an ID struct on each file, so that you
>> know what DB/re
On Wed, Dec 21, 2011 at 09:32:28AM +0100, Leonardo Francalanci wrote:
> I can't help in this discussion, but I have a question:
> how different would this feature be from filesystem-level CRC, such
> as the one available in ZFS and btrfs?
Hmm, filesystems are not magical. If they implement this th
Alvaro Herrera wrote:
> If you get away with a new page format, let's make sure and
> coordinate so that we can add more info into the header. One
> thing I wanted was to have an ID struct on each file, so that you
> know what DB/relation/segment the file corresponds to. So the
> first page's
Excerpts from David Fetter's message of mié dic 21 18:59:13 -0300 2011:
> If not, we'll have to do some extra work on the patch as described
> below. Thanks to Kevin Grittner for coming up with this :)
>
> - Use a header bit to say whether we've got a checksum on the page.
> We're using 3/16
Folks,
One of the things VMware is working on is double writes, per previous
discussions of how, for example, InnoDB does things. I'd initially
thought that introducing just one of the features in $Subject at a
time would help, but I'm starting to see a mutual dependency.
The issue is that doub
On Wed, Dec 21, 2011 at 1:59 PM, Alvaro Herrera
wrote:
> But, well, tuples that are succesfully hinted need no more hint bits.
Not only do they need no more hinting, they also allow the next
client-serving process that hits it avoid the clog lookup to determine
the hint.
a.
--
Aidan Van Dyk
On Tue, Dec 20, 2011 at 8:14 PM, Noah Misch wrote:
>> I also notice that cluster() - which doesn't have a callback - has
>> exactly the same needs as ReindexRelation() - which does. So that
>> case can certainly share code; though I'm not quite sure what to call
>> the shared callback, or which f
On Wed, Dec 21, 2011 at 12:48 PM, Robert Haas wrote:
> On the other hand, if we just want to avoid having more requests
> simultaneously in flight than we have buffers, so that backends don't
> need to wait for an available buffer before beginning their I/O, then
> something on the order of the nu
On 12/21/2011 10:49 AM, Stephen Frost wrote:
* Leonardo Francalanci (m_li...@yahoo.it) wrote:
I think what I meant was: isn't this going to be useless in a couple
of years (if, say, btrfs will be available)? Or it actually gives
something that FS will never be able to give?
Yes, it wi
On Wed, Dec 21, 2011 at 2:05 PM, Simon Riggs wrote:
> On Wed, Dec 21, 2011 at 3:24 PM, Robert Haas wrote:
>> I think there probably are some scalability limits to the current
>> implementation, but also I think we could probably increase the
>> current value modestly with something less than a to
On Wed, Dec 21, 2011 at 3:24 PM, Robert Haas wrote:
> I think there probably are some scalability limits to the current
> implementation, but also I think we could probably increase the
> current value modestly with something less than a total rewrite.
> Linearly scanning the slot array won't sca
Excerpts from Simon Riggs's message of mié dic 21 15:53:20 -0300 2011:
> On Wed, Dec 21, 2011 at 1:42 PM, Alvaro Herrera
> wrote:
>
> > This one I'm not sure about at all:
> >
> >> * "very large number of columns" for statistical data sets where we
> >> automatically vertically partition the hea
Excerpts from Robert Haas's message of mié dic 21 15:47:13 -0300 2011:
> Maybe. But I think we'd need to see some test results showing that it
> helps. I mean, the nice thing about our current system is that we
> don't set hint bits on tuples unless we otherwise have some need to
> look at them
On Wed, Dec 21, 2011 at 1:42 PM, Alvaro Herrera
wrote:
> This one I'm not sure about at all:
>
>> * "very large number of columns" for statistical data sets where we
>> automatically vertically partition the heap when faced with large
>> numbers of column definitions
We currently have pg_attribu
On Wed, Dec 21, 2011 at 1:14 PM, Kevin Grittner
wrote:
> Would it make sense, as suggested by Álvaro in the "CLOG contention"
> thread, to have a background process to set hint bits on tuples in
> dirty pages? Processing could be loosely based around the
> background writer techniques in terms of
On Wed, Dec 21, 2011 at 1:09 PM, Tom Lane wrote:
> It strikes me that one simple thing we could do is extend the current
> heuristic that says "pin the latest page". That is, pin the last K
> pages into SLRU, and apply LRU or some other method across the rest.
> If K is large enough, that should
Would it make sense, as suggested by Álvaro in the "CLOG contention"
thread, to have a background process to set hint bits on tuples in
dirty pages? Processing could be loosely based around the
background writer techniques in terms of sweeping through the cache,
but it would only look at dirty pag
Robert Haas writes:
> On Wed, Dec 21, 2011 at 11:48 AM, Tom Lane wrote:
>> I'm inclined to think that that specific arrangement wouldn't be good.
>> The normal access pattern for CLOG is, I believe, an exponentially
>> decaying probability-of-access for each page as you go further back from
>> cu
On Wed, Dec 21, 2011 at 11:48 AM, Tom Lane wrote:
> Agreed, the question is whether 32 is enough to fix the problem for
> anything except this one benchmark.
Right. My thought on that topic is that it depends on what you mean
by "fix". It's clearly NOT possible to keep enough CLOG buffers
aroun
On Wed, Dec 21, 2011 at 3:28 PM, Robert Haas wrote:
> On Wed, Dec 21, 2011 at 5:17 AM, Simon Riggs wrote:
>> With the increased performance we have now, I don't think increasing
>> that alone will be that useful since it doesn't solve all of the
>> problems and (I am told) likely increases lookup
Robert Haas writes:
> I think there probably are some scalability limits to the current
> implementation, but also I think we could probably increase the
> current value modestly with something less than a total rewrite.
> Linearly scanning the slot array won't scale indefinitely, but I think
> it
On Wed, Dec 21, 2011 at 11:44 AM, amit sehas wrote:
> It seems that the task of fetching next n results without moving the cursor
> seems like too complicated to implement for any query that has
> even a little bit of complication in it...
I think that's probably true. It would also be expensiv
On Thu, Dec 15, 2011 at 4:15 PM, amit sehas wrote:
> I had a question about the cursor internals implementation. When you Fetch
> next 'n' results without moving the cursors, is this kind of functionality
> implemented by firstly executing the whole query and then moving the cursor
> over the r
Heikki Linnakangas writes:
> 4 bytes out of a 8k block is just under 0.05%. I don't think anyone is
> going to notice the extra disk space consumed by this. There's all those
> other issues like the hint bits that make this a non-starter, but disk
> space overhead is not one of them.
The bigge
I think what I meant was: isn't this going to be useless in a couple
of years (if, say, btrfs will be available)? Or it actually gives
something that FS will never be able to give?
Yes, it will help you find/address bugs in the filesystem. These things
are not unheard of...
It sounds to me li
On Wed, Dec 21, 2011 at 18:21, Marti Raudsepp wrote:
> I think the least invasive fix, as proposed by Jeroen, is to fail only
> when ERANGE is set *and* the return value is 0.0 or +/-HUGE_VAL.
> Reading relevant specifications, this seems to be a fairly safe
> assumption. That's what the attached
Excerpts from Robert Haas's message of mié dic 21 13:18:36 -0300 2011:
> There may be workloads where that will help, but it's definitely not
> going to cover all cases. Consider my trusty
> pgbench-at-scale-factor-100 test case: since the working set fits
> inside shared buffers, we're only wri
Hi list,
Back in June we had a discussion about parsing denormal floating-point
values. A float8->text conversion could result in a number that can't
be converted back to float8 anymore for some values. Among other
things, this could break backups (though my searches didn't turn up
any reports of
On Wed, Dec 21, 2011 at 10:51 AM, Kevin Grittner
wrote:
> Robert Haas wrote:
>> Any thoughts on what makes most sense here? I find it fairly
>> tempting to just crank up NUM_CLOG_BUFFERS and call it good,
>
> The only thought I have to add to discussion so far is that the need
> to do anything m
On Thu, Dec 8, 2011 at 5:41 AM, Kyotaro HORIGUCHI
wrote:
> This is the patch to add the documentation of PGresult custom
> storage. It shows in section '31.19. Alternative result
> storage'.
It would be good to consolidate this into the main patch.
I find the names of the functions added here
Robert Haas wrote:
> Any thoughts on what makes most sense here? I find it fairly
> tempting to just crank up NUM_CLOG_BUFFERS and call it good,
The only thought I have to add to discussion so far is that the need
to do anything may be reduced significantly by any work to write
hint bits more
* Leonardo Francalanci (m_li...@yahoo.it) wrote:
> >Depends on how much you trust the filesystem. :)
>
> Ehm I hope that was a joke...
It certainly wasn't..
> I think what I meant was: isn't this going to be useless in a couple
> of years (if, say, btrfs will be available)? Or it actually gives
On Tue, Dec 20, 2011 at 12:12 PM, Christopher Browne wrote:
> This seems to be a frequent problem with this whole "doing CRCs on pages"
> thing.
>
> It's not evident which problems will be "real" ones.
That depends on the implementation. If we have a flaky, broken
implementation such as the one
On 21.12.2011 17:21, Kevin Grittner wrote:
Also, I'm not sure that our shop would want to dedicate any space
per page for this, since we're comparing between databases to ensure
that values actually match, row by row, during idle time.
4 bytes out of a 8k block is just under 0.05%. I don't thin
On 21/12/2011 16.19, Stephen Frost wrote:
* Leonardo Francalanci (m_li...@yahoo.it) wrote:
I can't help in this discussion, but I have a question:
how different would this feature be from filesystem-level CRC, such
as the one available in ZFS and btrfs?
Depends on how much you trust the filesy
On Wednesday, December 21, 2011 04:21:53 PM Kevin Grittner wrote:
> Greg Smith wrote:
> >> Some people think I border on the paranoid on this issue.
> >
> > Those people are also out to get you, just like the hardware.
>
> Hah! I *knew* it!
>
> >> Are you arguing that autovacuum should be dis
On Wed, Dec 21, 2011 at 5:17 AM, Simon Riggs wrote:
> With the increased performance we have now, I don't think increasing
> that alone will be that useful since it doesn't solve all of the
> problems and (I am told) likely increases lookup speed.
I have benchmarks showing that it works, for what
On Wed, Dec 21, 2011 at 12:33 AM, Tom Lane wrote:
> Oh btw, I haven't looked at that code recently, but I have a nasty
> feeling that there are parts of it that assume that the number of
> buffers it is managing is fairly small. Cranking up the number
> might require more work than just changing
Greg Smith wrote:
>> Some people think I border on the paranoid on this issue.
>
> Those people are also out to get you, just like the hardware.
Hah! I *knew* it!
>> Are you arguing that autovacuum should be disabled after crash
>> recovery? I guess if you are arguing that a database VACUU
* Leonardo Francalanci (m_li...@yahoo.it) wrote:
> I can't help in this discussion, but I have a question:
> how different would this feature be from filesystem-level CRC, such
> as the one available in ZFS and btrfs?
Depends on how much you trust the filesystem. :)
Stephen
signature.as
Tom Lane wrote:
> However, it's not a "commit time" trigger exactly -- keep in mind
> that SET CONSTRAINTS can override the trigger's own timing
> specification.
Hmm. Is there a way for trigger code to check whether it is running
deferred (at transaction commit time) versus immediate? (If no
Robert Haas writes:
> Personally, I hate patches that do more than one thing. For me, the
> time required to verify a patch goes as about O(n^2) in its size.
That's exactly why I'm opening that discussion. The main difference
between the approaches I can take is the time it takes to export each
On Wed, Dec 21, 2011 at 5:12 AM, Pavel Stehule wrote:
> This remove a not necessary string to text overhead
Committed, thanks.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To ma
On Tue, Dec 20, 2011 at 10:01 AM, Dimitri Fontaine
wrote:
> Either I develop them separately, with separate branches derived from
> the master one, or I develop them as a stack, one on top of the other.
> The difference is my ability to provide a patch for one of the features
> that can be applied
Excerpts from Simon Riggs's message of mié dic 21 09:44:04 -0300 2011:
> Sounds great.
>
> While you're doing this, I'd like to think about future requirements,
> to see if that changes anything.
>
> Having a unique logical column id is a great thing because it allows
> the physical storage to
On Tue, Dec 20, 2011 at 9:47 PM, Alvaro Herrera
wrote:
>> > The idea described there by Tom, and upon which I formed a vague
>> > implementation plan in my head, is that I was to look for all uses of
>> > an "attnum", and then replace it by either "attlognum" (i.e. the
>> > user-visible sort iden
On Tue, Dec 20, 2011 at 18:15, Simon Riggs wrote:
> On Tue, Dec 20, 2011 at 1:40 PM, Magnus Hagander wrote:
>> These days we have pause_at_recovery_target, which lets us pause when
>> we reach a PITR target. Is there a particular reason we don't have a
>> way to pause at end of recovery if we *di
On Wed, Dec 21, 2011 at 5:33 AM, Tom Lane wrote:
> Robert Haas writes:
>> ... while the main buffer manager is
>> content with some loosey-goosey approximation of recency, the SLRU
>> code makes a fervent attempt at strict LRU (slightly compromised for
>> the sake of reduced locking in SimpleLruR
Hello
This remove a not necessary string to text overhead
Regards
Pavel Stehule
*** ./src/backend/utils/adt/varlena.c.orig 2011-12-21 10:46:33.0 +0100
--- ./src/backend/utils/adt/varlena.c 2011-12-21 11:08:42.583720151 +0100
***
*** 3668,3674
state = PG_ARGISNULL(0) ?
Hello
this patch adds a bytea_agg aggregation.
It allow fast bytea concatetation.
Regards
Pavel Stehule
*** ./doc/src/sgml/func.sgml.orig 2011-12-07 11:04:33.0 +0100
--- ./doc/src/sgml/func.sgml 2011-12-21 11:00:18.255753111 +0100
***
*** 10911,10916
--- 10911,10934 ---
I can't help in this discussion, but I have a question:
how different would this feature be from filesystem-level CRC, such as
the one available in ZFS and btrfs?
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.o
On Wed, Dec 21, 2011 at 03:03, Tom Lane wrote:
> I've applied a revised version of this patch that factors things in a
> way I found nicer.
Nice, thanks!
Regards,
Marti
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgr
70 matches
Mail list logo
