On Fri, Jul 5, 2013 at 11:27 AM, Andres Freund wrote:
> Hi Robert,
>
> On 2013-07-02 09:31:23 -0400, Robert Haas wrote:
>> I have a few ideas for getting rid of the remaining uses of
>> SnapshotNow that I'd like to throw out there:
>
> Is your current plan to get rid of SnapshotNow entirely? I am
Hi Robert,
On 2013-07-02 09:31:23 -0400, Robert Haas wrote:
> I have a few ideas for getting rid of the remaining uses of
> SnapshotNow that I'd like to throw out there:
Is your current plan to get rid of SnapshotNow entirely? I am wonder
because the changeset extraction needs to care and how the
On 02.07.2013 18:24, Andres Freund wrote:
I've quickly verified that it indeed uses them. I wish I hadn't. Brrr. I
can't even guess what that should do from the surrounding code/function
names. Except that it looks broken under concurrency as long as
SnapshotNow is used (because the query's snaps
On 2013-07-02 10:38:17 -0400, Robert Haas wrote:
> On Tue, Jul 2, 2013 at 9:52 AM, Andres Freund wrote:
> >> > * possibly paranoid, but I'd add a Assert to heap_beginscan_catalog or
> >> > GetCatalogSnapshot ensuring we're dealing with a catalog relation. The
> >> > consistency mechanisms in G
On Tue, Jul 2, 2013 at 9:52 AM, Andres Freund wrote:
>> > * possibly paranoid, but I'd add a Assert to heap_beginscan_catalog or
>> > GetCatalogSnapshot ensuring we're dealing with a catalog relation. The
>> > consistency mechanisms in GetCatalogSnapshot() only work for those, so
>> > there
On 2013-07-02 09:31:23 -0400, Robert Haas wrote:
> On Tue, Jul 2, 2013 at 9:02 AM, Andres Freund wrote:
> > Some things that might be worth changing when committing:
> > * Could you add a Assert(!RelationHasSysCache(relid)) to
> > RelationInvalidatesSnapshotsOnly? It's not unlikely that it will
On Tue, Jul 2, 2013 at 9:02 AM, Andres Freund wrote:
> Some things that might be worth changing when committing:
> * Could you add a Assert(!RelationHasSysCache(relid)) to
> RelationInvalidatesSnapshotsOnly? It's not unlikely that it will be
> missed by the next person adding a syscache and th
On 2013-07-01 15:02:41 -0400, Robert Haas wrote:
> >> * These are updated by GetSnapshotData. We initialize them this way
> >> @@ -177,6 +188,9 @@ GetTransactionSnapshot(void)
> >> else
> >> CurrentSnapshot =
> >> GetSnapshotData(&CurrentSnapshotData);
> >>
>
On Mon, Jul 1, 2013 at 10:04 AM, Andres Freund wrote:
> This is really cool stuff.
Thanks.
> I have to say, if the thing that primarily suffers is pretty extreme DDL
> in extreme situations I am not really worried. Anybody running anything
> close to the territory of such concurrency won't perfo
On 2013-06-28 23:14:23 -0400, Robert Haas wrote:
> Here's a further update of this patch. In this version, I added some
> mechanism to send a new kind of sinval message that is sent when a
> catalog without catcaches is updated; it doesn't apply to all
> catalogs, just to whichever ones we want to
Robert Haas escribió:
> All right, so here's a patch that does something along those lines.
> We have to always take a new snapshot when somebody scans a catalog
> that has no syscache, because there won't be any invalidation messages
> to work off of in that case. The only catalog in that catego
On 2013-06-20 10:58:59 -0400, Robert Haas wrote:
> On Thu, Jun 20, 2013 at 10:35 AM, Andres Freund
> wrote:
> >> But I'm still on the fence about whether this is really a valid test.
> >
> > I think it shows that we need to be careful and do further performance
> > evaluations and/or alleviate th
On Thu, Jun 20, 2013 at 10:35 AM, Andres Freund wrote:
>> But I'm still on the fence about whether this is really a valid test.
>
> I think it shows that we need to be careful and do further performance
> evaluations and/or alleviate the pain by making things cheaper (say, a
> "ddl counter" in sha
On 2013-06-20 09:45:26 -0400, Robert Haas wrote:
> > With that setup one can create quite a noticeable overhead for the mvcc
> > patch (best of 5):
> >
> > master-optimize:
> > tps = 1261.629474 (including connections establishing)
> > tps = 15121.648834 (excluding connections establishing)
> >
> >
On Mon, Jun 17, 2013 at 8:12 AM, Andres Freund wrote:
> So, the biggest issue with the patch seems to be performance worries. I
> tried to create a worst case scenario:
> postgres (patched and HEAD) running with:
> -c shared_buffers=4GB \
> -c max_connections=2000 \
> -c maintenance_work_mem=2GB \
On 2013-06-03 14:57:12 -0400, Robert Haas wrote:
> On Thu, May 30, 2013 at 1:39 AM, Michael Paquier
> wrote:
> > +1.
>
> Here's a more serious patch for MVCC catalog access. This one
> involves more data copying than the last one, I think, because the
> previous version did not register the snap
On 2013-06-06 12:49:14 -0400, Robert Haas wrote:
> On Thu, Jun 6, 2013 at 5:30 AM, Andres Freund wrote:
> >> + * XXX: Now that we have MVCC catalog access, the reasoning above is no
> >> longer
> >> + * true. Are there other good reasons to hard-code this, or should we
> >> revisit
> >> + * tha
On Thu, Jun 6, 2013 at 2:48 PM, Jim Nasby wrote:
> On 6/5/13 3:49 PM, Robert Haas wrote:
>>
>> Now, I did find a couple that I thought should probably stick with
>> SnapshotNow, specifically pgrowlocks and pgstattuple.
>
> FWIW, I've often wished for a way to make all stat access transactional,
>
On 6/5/13 3:49 PM, Robert Haas wrote:
Now, I did find a couple that I thought should probably stick with
SnapshotNow, specifically pgrowlocks and pgstattuple.
FWIW, I've often wished for a way to make all stat access transactional, across all the
stats views. Perhaps that couldn't be done by d
On Thu, Jun 6, 2013 at 5:30 AM, Andres Freund wrote:
>> + * XXX: Now that we have MVCC catalog access, the reasoning above is no
>> longer
>> + * true. Are there other good reasons to hard-code this, or should we
>> revisit
>> + * that decision?
>
> We could just the function by looking in the
Hi Robert,
Took a quick look through the patch to understand what your current
revision is actually doing and to facilitate thinking about possible
pain points.
Here are the notes I made during my reading:
On 2013-06-03 14:57:12 -0400, Robert Haas wrote:
> +++ b/src/backend/catalog/catalog.c
> @
On 2013-06-05 18:56:28 -0400, Tom Lane wrote:
> Robert Haas writes:
> > Now, I did find a couple that I thought should probably stick with
> > SnapshotNow, specifically pgrowlocks and pgstattuple. Those are just
> > gathering statistical information, so there's no harm in having the
> > snapshot c
On Wed, Jun 5, 2013 at 6:56 PM, Tom Lane wrote:
> Robert Haas writes:
>> Now, I did find a couple that I thought should probably stick with
>> SnapshotNow, specifically pgrowlocks and pgstattuple. Those are just
>> gathering statistical information, so there's no harm in having the
>> snapshot ch
Robert Haas writes:
> Now, I did find a couple that I thought should probably stick with
> SnapshotNow, specifically pgrowlocks and pgstattuple. Those are just
> gathering statistical information, so there's no harm in having the
> snapshot change part-way through the scan, and if the scan is long
On Wed, Jun 5, 2013 at 10:28 AM, Greg Stark wrote:
> On Wed, May 22, 2013 at 3:18 AM, Robert Haas wrote:
>> We've had a number of discussions about the evils of SnapshotNow. As
>> far as I can tell, nobody likes it and everybody wants it gone, but
>> there is concern about the performance impact
On 2013-06-05 11:35:58 -0400, Tom Lane wrote:
> Andres Freund writes:
> > On 2013-06-05 15:28:09 +0100, Greg Stark wrote:
> >> I thought there were many call sites that were specifically depending
> >> on seeing dirty reads to avoid race conditions with other backends --
> >> which probably just n
Andres Freund writes:
> On 2013-06-05 15:28:09 +0100, Greg Stark wrote:
>> I thought there were many call sites that were specifically depending
>> on seeing dirty reads to avoid race conditions with other backends --
>> which probably just narrowed the race condition or created different
>> ones.
On 2013-06-05 15:28:09 +0100, Greg Stark wrote:
> On Wed, May 22, 2013 at 3:18 AM, Robert Haas wrote:
> > We've had a number of discussions about the evils of SnapshotNow. As
> > far as I can tell, nobody likes it and everybody wants it gone, but
> > there is concern about the performance impact.
On 06/05/2013 04:28 PM, Greg Stark wrote:
> On Wed, May 22, 2013 at 3:18 AM, Robert Haas wrote:
>> We've had a number of discussions about the evils of SnapshotNow. As
>> far as I can tell, nobody likes it and everybody wants it gone, but
>> there is concern about the performance impact.
> I was
On Wed, May 22, 2013 at 3:18 AM, Robert Haas wrote:
> We've had a number of discussions about the evils of SnapshotNow. As
> far as I can tell, nobody likes it and everybody wants it gone, but
> there is concern about the performance impact.
I was always under the impression that the problem was
On Tue, Jun 4, 2013 at 3:57 AM, Robert Haas wrote:
> On Thu, May 30, 2013 at 1:39 AM, Michael Paquier
> wrote:
> > +1.
>
> Here's a more serious patch for MVCC catalog access. This one
> involves more data copying than the last one, I think, because the
> previous version did not register the s
On Tue, May 28, 2013 at 10:39 PM, Robert Haas wrote:
> IMHO, we should press forward with this approach. Considering that
> these are pretty extreme test cases, I'm inclined to view the
> performance loss as acceptable. We've never really viewed DDL as
> something that needs to be micro-optimiz
On Sun, May 26, 2013 at 9:10 PM, Michael Paquier
wrote:
> Perhaps we see little difference in performance because PGPROC has been
> separated into PGPROC and PGXACT, reducing lock contention with getting
> snapshot data?
>
> By the way, I grabbed a 32-core machine and did some more performance tes
Perhaps we see little difference in performance because PGPROC has been
separated into PGPROC and PGXACT, reducing lock contention with getting
snapshot data?
By the way, I grabbed a 32-core machine and did some more performance tests
with some open connections with XIDs assigned using pg_cxn v2 g
On Wed, May 22, 2013 at 11:11 PM, Andres Freund wrote:
> Make that actually having acquired an xid. We skip a large part of the
> work if a transaction doesn't yet have one afair. I don't think the mere
> presence of 600 idle connections without an xid in contrast to just
> having max_connection a
On 2013-05-22 23:05:40 -0400, Robert Haas wrote:
> On Wed, May 22, 2013 at 11:02 PM, Andres Freund
> wrote:
> > On 2013-05-22 22:51:13 -0400, Robert Haas wrote:
> >> In discussions today, Tom Lane suggested testing the time to start up
> >> a backend and run a simple query such as "SELECT 2+2" in
On Wed, May 22, 2013 at 11:02 PM, Andres Freund wrote:
> On 2013-05-22 22:51:13 -0400, Robert Haas wrote:
>> In discussions today, Tom Lane suggested testing the time to start up
>> a backend and run a simple query such as "SELECT 2+2" in the absence
>> of a relcache file.
>
>> I did this and can'
On 2013-05-22 22:51:13 -0400, Robert Haas wrote:
> In discussions today, Tom Lane suggested testing the time to start up
> a backend and run a simple query such as "SELECT 2+2" in the absence
> of a relcache file.
> I did this and can't measure any overhead as a result of MVCC catalog
> access. I
On Tue, May 21, 2013 at 10:18 PM, Robert Haas wrote:
> [ MVCC catalog access seems to be pretty cheap ]
In discussions today, Tom Lane suggested testing the time to start up
a backend and run a simple query such as "SELECT 2+2" in the absence
of a relcache file.
I did this and can't measure any
39 matches
Mail list logo
