At 2013-10-24 11:33:13 +0530, a...@2ndquadrant.com wrote:
>
> >From /proc/$pid/status, VmPTE was 2880kb with huge_tlb_pages=off, and
> 56kb with it turned on.
(VmPTE is the size of the process's page tables.)
-- Abhijit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To m
Hi.
This is a slightly reworked version of the patch submitted by Richard
Poole last month, which was based on Christian Kruse's earlier patch.
Apart from doing various minor cleanups and documentation fixes, I also
tested this patch against HEAD on a machine with 256GB of RAM. Here's an
overview
On Thu, Oct 24, 2013 at 4:58 AM, Thomas Munro wrote:
> Hi
> I noticed that CLUSTER doesn't have a FREEZE option. Here is a patch to add
> that, for consistency with VACUUM. Is it useful?
I wonder why anyone would like to freeze during CLUSTER command when
they already have separate way (VACUUM
Hi all
I'm interested in an opinion: How should RLS treat cascading RI constraints?
The current tests check ON DELETE NO ACTION constraints, causing
insertion to fail if a PK row exists but is not visible, and deletion to
fail if there's still an invisible FK reference to a visible PK.
That seem
Gordon Mohr wrote:
> Thanks for this! I decided to give the patch a try at the bleeding
> edge with some high-dimensional vectors, specifically the 1.4
> million 1000-dimensional Freebase entity vectors from the Google
> 'word2vec' project:
>
> https://code.google.com/p/word2vec/#Pre-trained_enti
Agree that windowing function will return all the rows compared to max and
group by returing only max rows per group. But even while arriving at the
aggregate/sorting windowing function seems to spend more effort than group
by/order by.
I am just trying to see if we could somehow optimize the way
Peter Geoghegan escribió:
> I am interested in making it store richer statistics,
> provided we're very careful about the costs. Every time those counters
> are incremented, a spinlock is held.
Hmm, now if we had portable atomic addition, so that we could spare the
spinlock ...
--
Álvaro Herrer
[bikeshed]
I'm not totally comfortable with the details of ALTER TABLE ... SET ROW
SECURITY FOR ALL .
If you read:
ALTER TABLE document SET ROW SECURITY FOR ALL
TO (...query...)
is it obvious to you that "FOR ALL" here refers to all _command types_?
I'd like to make that more explicit and
On Wed, Oct 23, 2013 at 2:44 PM, Nigel Heron wrote:
> Hi, thanks, I'm still actively working on this patch. I've gotten the
> traffic counters working when using SSL enabled clients (includes the
> ssl overhead now) but I still have the walsender transfers under SSL
> to work on.
> I'll post an up
Hi all
I'm going to be contributing a fair bit of time to RLS for 2ndQuadrant,
courtesy of the EU AXLE project (http://axleproject.eu/).
I've been catching up on Kohei KaiGai's work and I've been really
impressed by what's already done and working. I'm currently reading the
patches, mailing list
Sameer Kumar wrote
> edb=# explain analyze select max(score) from student_score group by
> course;
This query returns 6 records. The window one returns 123,000. Why do you
expect these to have anywhere near the same performance or plan?
You can enable/disable indexes/scans to see what alternati
On 9/22/13 4:38 PM, Stas Kelvich wrote:
Hello, hackers.
Here is the patch that introduces kNN search for cubes with
euclidean, taxicab and chebyshev distances.
Thanks for this! I decided to give the patch a try at the bleeding edge
with some high-dimensional vectors, specifically the 1.4 mill
Mike Blackwell writes:
> While reviewing the Network Stats Traffic patch I discovered the current
> regression test for rules depends on the system view definitions not
> changing:
Yes, this is standard. We just update the expected output anytime
somebody changes a system view.
(Now, if the sub
On Wed, Oct 23, 2013 at 5:15 PM, Gavin Flower
wrote:
> Anyhow, I was suggesting the faclity be off by default - I see no point in
> enabling where people don't need it, even if the resource RAM, processor,
> whatever, were minimal.
As long as any new field in the Counters struct needs to be prote
On 24/10/13 12:58, Peter Geoghegan wrote:
On Wed, Oct 23, 2013 at 4:48 PM, Gavin Flower
wrote:
32 int64 buckets is only 256 bytes, so a thousand histograms would be less
than a quarter of a MB. Any machine that busy, would likely have many GB's
of RAM. I have 32 GB on my development machine.
On Wed, Oct 23, 2013 at 4:51 PM, Peter Geoghegan wrote:
> No, I'm not. I'm suggesting storing the query texts externally, in a
> file. They usually use 1024 bytes of shared memory per entry,
> regardless of how long the query text is.
I should add that I think that that's about the useful limit o
On Wed, Oct 23, 2013 at 4:48 PM, Gavin Flower
wrote:
> 32 int64 buckets is only 256 bytes, so a thousand histograms would be less
> than a quarter of a MB. Any machine that busy, would likely have many GB's
> of RAM. I have 32 GB on my development machine.
Who wants to just run with a thousand
On 24/10/13 12:46, Josh Berkus wrote:
On 10/23/2013 01:26 PM, Peter Geoghegan wrote:
So fixing that problem would go a long
way towards resolving these concerns. It would also probably have the
benefit of making it possible for query texts to be arbitrarily long -
we'd be storing them in files (
On Wed, Oct 23, 2013 at 4:46 PM, Josh Berkus wrote:
> So you're suggesting that instead of storing the aggregates as we
> currently do, we store a buffer of the last N queries (in normal form)
> and their stats? And then aggregate when the user asks for it?
No, I'm not. I'm suggesting storing th
On 24/10/13 12:24, Peter Geoghegan wrote:
On Wed, Oct 23, 2013 at 4:14 PM, Jeff Janes wrote:
The last bucket would be limited to 8ms < x <= 16 ms. If you find something
16ms, then you have to rescale *before* you increment any of the buckets.
Once you do, there is now room to hold it.
How i
On Wed, Oct 23, 2013 at 4:34 PM, Stephen Frost wrote:
> Any chance we could accumulate into the histogram in local memory and
> only push to the shared memory on an "infrequent" or at least "less
> frequent" basis? Apologies, I've not looked into the pg_stat_statements
> bits all that closely, bu
On 10/23/2013 01:26 PM, Peter Geoghegan wrote:
> So fixing that problem would go a long
> way towards resolving these concerns. It would also probably have the
> benefit of making it possible for query texts to be arbitrarily long -
> we'd be storing them in files (with a shared memory buffer). I g
* Peter Geoghegan (p...@heroku.com) wrote:
> On Wed, Oct 23, 2013 at 4:14 PM, Jeff Janes wrote:
> > The last bucket would be limited to 8ms < x <= 16 ms. If you find something
> >> 16ms, then you have to rescale *before* you increment any of the buckets.
> > Once you do, there is now room to hold
On 24/10/13 12:14, Jeff Janes wrote:
On Wed, Oct 23, 2013 at 4:00 PM, Gavin Flower
mailto:gavinflo...@archidevsys.co.nz>>
wrote:
On 24/10/13 11:26, Peter Geoghegan wrote:
On Wed, Oct 23, 2013 at 2:57 PM, Gavin Flower
mailto:gavinflo...@archidevsys.co.nz>> wrote:
Hi
I noticed that CLUSTER doesn't have a FREEZE option. Here is a patch to
add that, for consistency with VACUUM. Is it useful?
Thanks
Thomas Munro
cluster-freeze.patch
Description: Binary data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subsc
* Martijn van Oosterhout (klep...@svana.org) wrote:
> If I had to guess a distribution for query runtimes I'd go for Poisson,
> which would mean you'd expect the mean to equal the variance. Don't
> have enough experience with such measurements to say whether that is
> reasonable.
I was thinking th
On Wed, Oct 23, 2013 at 4:14 PM, Jeff Janes wrote:
> The last bucket would be limited to 8ms < x <= 16 ms. If you find something
>> 16ms, then you have to rescale *before* you increment any of the buckets.
> Once you do, there is now room to hold it.
How is that laid out in shared memory? If the
On Wed, Oct 23, 2013 at 4:00 PM, Gavin Flower wrote:
> On 24/10/13 11:26, Peter Geoghegan wrote:
>
>> On Wed, Oct 23, 2013 at 2:57 PM, Gavin Flower
>> wrote:
>>
>>> Looks definitely bimodal in the log version, very clear!
>>>
>>> Yes, I feel that having a 32 log binary binned histogram (as Alvar
On 24/10/13 12:00, Gavin Flower wrote:
On 24/10/13 11:26, Peter Geoghegan wrote:
On Wed, Oct 23, 2013 at 2:57 PM, Gavin Flower
wrote:
Looks definitely bimodal in the log version, very clear!
Yes, I feel that having a 32 log binary binned histogram (as Alvaro
Herrera
suggested) would be very
On Wed, Oct 23, 2013 at 3:26 PM, Peter Geoghegan wrote:
> On Wed, Oct 23, 2013 at 2:57 PM, Gavin Flower
> wrote:
> > Looks definitely bimodal in the log version, very clear!
> >
> > Yes, I feel that having a 32 log binary binned histogram (as Alvaro
> Herrera
> > suggested) would be very useful.
On 24/10/13 11:26, Peter Geoghegan wrote:
On Wed, Oct 23, 2013 at 2:57 PM, Gavin Flower
wrote:
Looks definitely bimodal in the log version, very clear!
Yes, I feel that having a 32 log binary binned histogram (as Alvaro Herrera
suggested) would be very useful.
I'm having a hard time imagining
On Wed, Oct 23, 2013 at 2:57 PM, Gavin Flower
wrote:
> Looks definitely bimodal in the log version, very clear!
>
> Yes, I feel that having a 32 log binary binned histogram (as Alvaro Herrera
> suggested) would be very useful.
I'm having a hard time imagining how you'd actually implement that.
Fo
On 24/10/13 10:34, Marc Mamin wrote:
Oscillating plan changes may fit multimodal but I don't feel that's
typical. My experience has been it's either an extremely rare plan
difference or it's a shift from one plan to another over time.
After all, all of avg, min, max and stdev are only numerical
On Wed, Oct 23, 2013 at 2:46 PM, Martijn van Oosterhout
wrote:
> Well, the standard deviation is the square root of the variance, which
> is the second moment of the distribution. The first moment being the
> mean. No matter what distribution it is, these are useful numbers.
I'm inclined to agree
Folks,
Of course, this survey is completely unscientific. However, it does
give us more information than we had before:
http://www.postgresql.org/community/survey/89-what-is-the-smallest-amount-of-ram-you-have-on-a-production-postgresql-server/
So going by this, only 20% of our users have any p
On Tue, Oct 22, 2013 at 11:16:19AM -0700, Jeff Janes wrote:
> On Mon, Oct 21, 2013 at 4:29 PM, Tom Lane wrote:
> > Hm. It's been a long time since college statistics, but doesn't the
> > entire concept of standard deviation depend on the assumption that the
> > underlying distribution is more-or-
On Wed, Oct 23, 2013 at 1:07 PM, Jeff Janes wrote:
> How does max not answer "is this query ever really slow?"? But good point,
> if we have a max, then I think a time-stamp for when that max was obtained
> would also be very useful.
I'm concerned about the cost of all of this. And like Stephen,
On 23 October 2013 21:08, Andres Freund wrote:
> On 2013-10-23 20:51:27 +0100, Dean Rasheed wrote:
>> On 23 October 2013 02:18, Andres Freund wrote:
>> > Hi,
>> >
>> > Using the same debugging hack^Wpatch (0001) as in the matview patch
>> > (0002) an hour or so ago I noticed that INSERT INTO view
* Jeff Janes (jeff.ja...@gmail.com) wrote:
> On Wed, Oct 23, 2013 at 9:20 AM, Stephen Frost wrote:
> > * Josh Berkus (j...@agliodbs.com) wrote:
> > > On the other hand, it's still true that a high STDDEV indicates a high
> > > variance in the response times of a particular query, whereas a low one
On 2013-10-23 20:51:27 +0100, Dean Rasheed wrote:
> On 23 October 2013 02:18, Andres Freund wrote:
> > Hi,
> >
> > Using the same debugging hack^Wpatch (0001) as in the matview patch
> > (0002) an hour or so ago I noticed that INSERT INTO view WITH CHECK
> > doesn't lock the underlying relations p
On Wed, Oct 23, 2013 at 9:20 AM, Stephen Frost wrote:
> Josh,
>
> * Josh Berkus (j...@agliodbs.com) wrote:
> > On the other hand, it's still true that a high STDDEV indicates a high
> > variance in the response times of a particular query, whereas a low one
> > indicates that most are close to th
On 23 October 2013 02:18, Andres Freund wrote:
> Hi,
>
> Using the same debugging hack^Wpatch (0001) as in the matview patch
> (0002) an hour or so ago I noticed that INSERT INTO view WITH CHECK
> doesn't lock the underlying relations properly.
>
> I've attached a sort-of-working (0003) hack but I
On Wed, Oct 23, 2013 at 2:10 PM, Atri Sharma wrote:
>
> Adding myself as the co reviewer specifically for the testing
> purposes, if its ok with you.
>
It's perfectly fine with me. Please do!
__
*Mike Blackw
On Thu, Oct 24, 2013 at 12:30 AM, Mike Blackwell wrote:
>
> On Wed, Oct 23, 2013 at 1:58 PM, Atri Sharma wrote:
>
>>
>> IMO, the idea is pretty good. Its just that we need to do some wide
>> spectrum performance testing. Thats only my thought though.
>
>
>
> I'm looking at trying to do some perfo
On Wed, Oct 23, 2013 at 2:58 PM, Atri Sharma wrote:
> On Thu, Oct 24, 2013 at 12:23 AM, Mike Blackwell
> wrote:
>> Sounds good. I personally don't have any interest in log file i/o counters,
>> but that's just me. I wonder if stats collector counters might be useful...
>> I seem to recall an e
On Wed, Oct 23, 2013 at 1:58 PM, Atri Sharma wrote:
>
> IMO, the idea is pretty good. Its just that we need to do some wide
> spectrum performance testing. Thats only my thought though.
I'm looking at trying to do some performance testing on this. Any
suggestions on test scenarios, etc?
_
On Thu, Oct 24, 2013 at 12:23 AM, Mike Blackwell wrote:
> Sounds good. I personally don't have any interest in log file i/o counters,
> but that's just me. I wonder if stats collector counters might be useful...
> I seem to recall an effort to improve that area. Maybe not enough use to
> take t
Sounds good. I personally don't have any interest in log file i/o
counters, but that's just me. I wonder if stats collector counters might
be useful... I seem to recall an effort to improve that area. Maybe not
enough use to take the performance hit on a regular basis, though.
_
Hi, thanks, I'm still actively working on this patch. I've gotten the
traffic counters working when using SSL enabled clients (includes the
ssl overhead now) but I still have the walsender transfers under SSL
to work on.
I'll post an updated patch when i have it figured out.
Since the patch changes
On Sat, Oct 19, 2013 at 5:44 AM, Dean Rasheed wrote:
> On 18 October 2013 16:41, Robert Haas wrote:
>> On Fri, Oct 18, 2013 at 1:34 AM, Dean Rasheed
>> wrote:
Personally, I think this is too fancy anyway. I'd just complete all
views and foreign tables and be done with it. We don't i
I added this to the current CF, and am starting to review it as I have time.
__
*Mike Blackwell | Technical Analyst, Distribution Services/Rollout
Management | RR Donnelley*
1750 Wallace Ave | St Charles, IL 60174-3401
On Wed, Oct 23, 2013 at 12:24 PM, Josh Berkus wrote:
> Dimitri,
>
>> Josh Berkus writes:
>>> pg_partman has several external (python) scripts which help the
>>> extension, located in /extras/ in its source. The problem currently is
>>> that if you install pg_partman via pgxn or package, you don'
On 10/23/2013 05:38 AM, Noah Misch wrote:
> We say little about
> the correct aspects of a patch; it's usually a given that things not mentioned
> are satisfactory and have self-evident value. That's not such an effective
> discussion pattern when the topic is management strategies.
It's not an e
While reviewing the Network Stats Traffic patch I discovered the current
regression test for rules depends on the system view definitions not
changing:
--
-- Check that ruleutils are working
--
SELECT viewname, definition FROM pg_views WHERE schemaname <>
'information_schema' ORDER BY viewname;
On Wed, Oct 23, 2013 at 10:58 PM, Tom Lane wrote:
> Sameer Kumar writes:
> > I am not sure why but my PostgreSQL does not seem to be using indexes for
> > ORDER BY clause or PARTITION BY CLAUSE which I use with windowing
> function.
>
> When the entire contents of the table have to be read, a se
Dimitri,
> Josh Berkus writes:
>> pg_partman has several external (python) scripts which help the
>> extension, located in /extras/ in its source. The problem currently is
>> that if you install pg_partman via pgxn or package, you don't get those
>> scripts, because there's no "install" location
Josh,
* Josh Berkus (j...@agliodbs.com) wrote:
> On the other hand, it's still true that a high STDDEV indicates a high
> variance in the response times of a particular query, whereas a low one
> indicates that most are close to the average. While precision math
> might not work if we don't have
* Robert Haas (robertmh...@gmail.com) wrote:
> On Wed, Oct 23, 2013 at 11:35 AM, Stephen Frost wrote:
> > Would this make sense as a configure-time check, rather than initdb, to
> > try blocking SIGSYS and checking for an ENOSYS from shm_open()? Seems
> > preferrable to do that in a configure che
All,
Bringing this down to Earth: yes, it would be useful to have min and
max for pg_stat_statement, and even if we add more stats to
pg_stat_statement, it would be useful to have those two. So can we
approve this patch on that basis?
For my part, I generally use the 9-part percentiles for quer
On Mon, Oct 21, 2013 at 11:10:09AM -0400, Tom Lane wrote:
> Heikki Linnakangas writes:
> > On 21.10.2013 16:15, Peter Eisentraut wrote:
> >> What is the alternative?
>
> > If no-one really cares enough about a patch to review it, mark it as
> > "rejected, because no-one but the patch author care
On Wed, Oct 23, 2013 at 8:38 AM, Noah Misch wrote:
> On Tue, Oct 22, 2013 at 12:27:13PM -0700, Josh Berkus wrote:
>> When I came up with the idea of CommitFests they were supposed to be an
>> incremental improvement for us to build on. Instead it's remained
>> frozen in amber, and steadily becomi
On Wed, Oct 23, 2013 at 11:35 AM, Stephen Frost wrote:
> * Tom Lane (t...@sss.pgh.pa.us) wrote:
>> I agree with Robert that it's odd and obnoxious that the call doesn't just
>> return with errno = ENOSYS. However, looking in the archives turns up
>> this interesting historical info:
>> http://www
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> I agree with Robert that it's odd and obnoxious that the call doesn't just
> return with errno = ENOSYS. However, looking in the archives turns up
> this interesting historical info:
> http://www.postgresql.org/message-id/25564.962066...@sss.pgh.pa.us
Wow,
Stephen Frost writes:
> I'm going to guess this idea is a non-starter, but any hope there's some
> other system call which would tell us we're on a platform where
> shm_open() will hit us with SIGSYS? What happens when shm_unlink() is
> called on this platform? Or mmap()?
For context's sake, th
Sameer Kumar writes:
> I am not sure why but my PostgreSQL does not seem to be using indexes for
> ORDER BY clause or PARTITION BY CLAUSE which I use with windowing function.
When the entire contents of the table have to be read, a seqscan-and-sort
will frequently be estimated as cheaper than an
* Robert Haas (robertmh...@gmail.com) wrote:
> I don't think a configure-time test is a good idea because there's no
> guarantee that the configure-time machine and the run-time machine
> have the same behavior. But having initdb fork a child process to run
> the test seems like a reasonable way f
Robert Haas writes:
> On Tue, Oct 22, 2013 at 10:07 PM, Tom Lane wrote:
>> selecting dynamic shared memory implementation ... Bad system call(coredump)
> Well, geez. That's obnoxious.
I quite agree :-(. If it were just this old HPUX version, maybe we could
write it off as something we don't c
On Tue, Oct 22, 2013 at 10:07 PM, Tom Lane wrote:
> initdb.c quoth:
>
> * ... but the fact that a platform has shm_open
> * doesn't guarantee that that call will succeed when attempted.
>
> Indeed:
>
> $ initdb
> The files belonging to this database system will be owned by user "postgres".
> Thi
Hi,
I was dealing with windowing function recently. I feel they are pretty
useful and quite handy in lot of operations.
I am not sure why but my PostgreSQL does not seem to be using indexes for
ORDER BY clause or PARTITION BY CLAUSE which I use with windowing function.
I have tried ORDER BY and G
David Rowley writes:
> On Wed, Oct 23, 2013 at 4:00 AM, Tom Lane wrote:
>> Yeah. As a separate matter, it might be useful to revise stringinfo.c
>> and the asprintf code so that *if* the returned value is larger than the
>> given buffer size, we use it as a guide to resizing, avoiding the possib
Florian Weimer writes:
> Do you care about the snprintf behavior on very large buffers (larger
> than INT_MAX)? Then there's further complication, and it's an area
> where glibc behavior is likely to change in the future (because it is
> claimed that C99 and POSIX conflict, and glibc implement
On Tue, Oct 22, 2013 at 12:27:13PM -0700, Josh Berkus wrote:
> When I came up with the idea of CommitFests they were supposed to be an
> incremental improvement for us to build on. Instead it's remained
> frozen in amber, and steadily becoming less and less effective. I've
> suggested a number of
On 22.10.2013 14:14, Andres Freund wrote:
Hi Robert, Heikki,
On 2013-09-24 13:25:41 +0200, Andres Freund wrote:
I'm afraid this patch was a few bricks shy of a load. The
log_newpage_buffer() function asserts that:
/* We should be in a critical section. */
Assert(CritSectionCou
On Wed, Oct 23, 2013 at 4:00 AM, Tom Lane wrote:
> Yeah. As a separate matter, it might be useful to revise stringinfo.c
> and the asprintf code so that *if* the returned value is larger than the
> given buffer size, we use it as a guide to resizing, avoiding the possible
> need to loop multiple
On Wed, Oct 23, 2013 at 2:54 PM, Michael Paquier
wrote:
> Hi,
>
> Thanks for the feedback. Btw, pg_rewind is not a project included in
> Postgres core as a contrib module or anything, so could you send your
> feedback and the issues you find directly on github instead? The URL
> of the project is
Hi,
Thanks for the feedback. Btw, pg_rewind is not a project included in
Postgres core as a contrib module or anything, so could you send your
feedback and the issues you find directly on github instead? The URL
of the project is https://github.com/vmware/pg_rewind.
Either way, here are some comm
While testing pg_rewind I encountered following problem.
I used following process to do the testing, Please correct me if I am doing
it in wrong way.
Problem-1:
pg_rewind gives error (target master must be shut down cleanly.) when
master crashed unexpectedly.
1. Setup Streaming Replication (stan
On 10/22/2013 11:06 PM, Tom Lane wrote:
Attached is a draft, which compiles though I've not yet actually tested it.
nprinted = vsnprintf(buf, len, fmt, args);
Assert(buf[len - 1] == '\0');
The assert may fire if len > INT_MAX and the system returns with errno
== EOVERFLOW, a
78 matches
Mail list logo
