On Mon, May 13, 2013 at 11:28 PM, Noah Misch wrote:
> * Identifying Parallel-Compatible Functions
>
> Not all functions can reasonably run on a worker backend. We should not
> presume that a VOLATILE function can tolerate the unstable execution order
> imposed by parallelism, though a function l
On Mon, May 13, 2013 at 3:02 PM, Peter Geoghegan wrote:
> Has anyone else thought about approaches to mitigating the problems
> that arise when an archive_command continually fails, and the DBA must
> manually clean up the mess?
Notably, the most common problem in this vein suffered at Heroku has
On Mon, 2013-05-13 at 22:38 -0400, Tom Lane wrote:
> Peter Eisentraut writes:
> > In the last two years, the first commit fest started in June, which is
> > about a month from now. If we are going to do that again, we should
> > clarify that as soon as possible. And if we are not, then we should
On 05/13/2013 10:27 PM, Peter Eisentraut wrote:
On Tue, 2013-05-14 at 04:12 +0300, Marti Raudsepp wrote:
It's caused by this common pattern in extension makefiles:
DATA = $(wildcard sql/*--*.sql) sql/$(EXTENSION)--$(EXTVERSION).sql
What is the point of this? Why have the wildcard and then th
On Mon, May 13, 2013 at 08:54:39PM -0500, Jon Nelson wrote:
> Pertinent to another thread titled
> [HACKERS] corrupt pages detected by enabling checksums
> I hope to explore the possibility of using fallocate (or
> posix_fallocate) for new WAL file creation.
>
> Most modern Linux filesystems suppo
Peter Eisentraut writes:
> In the last two years, the first commit fest started in June, which is
> about a month from now. If we are going to do that again, we should
> clarify that as soon as possible. And if we are not, then we should
> also clarify that, because some people are probably expe
In the last two years, the first commit fest started in June, which is
about a month from now. If we are going to do that again, we should
clarify that as soon as possible. And if we are not, then we should
also clarify that, because some people are probably expecting that we
are.
So, any though
On Tue, 2013-05-14 at 04:12 +0300, Marti Raudsepp wrote:
> It's caused by this common pattern in extension makefiles:
> DATA = $(wildcard sql/*--*.sql) sql/$(EXTENSION)--$(EXTVERSION).sql
>
What is the point of this? Why have the wildcard and then the
non-wildcard term?
I think using wildcard is
> Well, if we are allowed to use a bit ugry way, the attached patch
> seems to cope with this issue. As far as I can see there's no
> problem since pg_fe_sendauth() refueses to send empty password.
>
> Any suggestions?
That seems to do the trick. This probably solves the problem that I
originally
Marti Raudsepp writes:
> While testing out PostgreSQL 9.3beta1, I stumbled upon a problem
> installing some extensions (pgTAP and semver among others):
> ...
> I traced the problem down to commit
> 9db7ccae2000524b72a4052352cbb5407fb53b02 "Use system install program
> when available and usable". I
Pertinent to another thread titled
[HACKERS] corrupt pages detected by enabling checksums
I hope to explore the possibility of using fallocate (or
posix_fallocate) for new WAL file creation.
Most modern Linux filesystems support fast fallocate/posix_fallocate,
reducing extent fragmentation (where
> In fact, this is the behavior with all the authentication methods that
> require a password. But, it is only in the case of PAM authentication
> that auth_failed() logs error when first connection attempt is made
> (without password), since the STATUS_EOF is not passed to it in that
> case.
Well
Hi list,
While testing out PostgreSQL 9.3beta1, I stumbled upon a problem
installing some extensions (pgTAP and semver among others):
% make
[...]
% make DESTDIR=/tmp/foo install
[...]
/usr/bin/install -c -m 644 ./sql/semver--unpackaged--0.2.1.sql
./sql/semver--0.2.4--0.3.0.sql ./sql/semver--0.2.
* Marko Kreen (mark...@gmail.com) wrote:
> On Sat, May 04, 2013 at 10:57:44PM +0200, Dimitri Fontaine wrote:
> > Other than adminpack, I know of PGQ installing their objects in
> > pg_catalog. They only began doing that when switching to the CREATE
> > EXTENSION facility. And they set relocatable t
Hi,
Patch to add more regression tests to ASYNC (/src/backend/commands/async).
Take code-coverage from 39% to 75%.
Any and all feedback is obviously welcome.
p.s.: Please let me know whether these tests are useless or would not be
committed owing to unrelated reasons. As also, if these tests nee
Could you please provide a bit more detailed explanation on how it works?
And how could postgres write at the middle of archiving files during an
active pg_start_backup? if it could, here might be a case when a part of
archived data file contains an overridden information "from the future",
while
On Tue, May 7, 2013 at 7:01 PM, Tom Lane wrote:
> It seems reasonable to me to reduce it to DEBUG1 level.
Attached patch renders all "loaded library..." messages DEBUG1,
regardless of whether local_preload_libraries or
shared_preload_libraries is involved, and regardless of EXEC_BACKEND.
--
Pet
The documentation says of continuous archiving:
"While designing your archiving setup, consider what will happen if
the archive command fails repeatedly because some aspect requires
operator intervention or the archive runs out of space. For example,
this could occur if you write to tape without a
On 13 May 2013 14:45, Jon Nelson wrote:
> I should not derail this thread any further. Perhaps, if interested
> parties would like to discuss the use of fallocate/posix_fallocate, a
> new thread might be more appropriate?
Sounds like a good idea. Always nice to see a fresh take on earlier ideas.
On 13 May 2013 15:57, Tom Lane wrote:
> I think you need to think in terms of restricting the problem space
+1
> One obvious suggestion is to forbid the workers from
> doing any database access of their own at all --- the parent would
> have to do any required catalog lookups for sort functions
2013/5/13 Noah Misch
> * Planner & Similar Issues
>
> We should decide whether to actually sort in parallel based on the
> comparator
> cost and the data size. The system currently has no information on
> comparator
> cost: bt*cmp (and indeed almost all built-in functions) all have procost=1,
>
Andres Freund writes:
> On 2013-05-13 14:48:52 -0400, Tom Lane wrote:
>> Andres Freund writes:
>>> DROPing and recreating a new index in --single mode isn't that
>>> uncommon...
>> Surely you'd just REINDEX it instead. Moreover, if it isn't a system
>> index already, why are you doing this in -
Would try to revert with a faster script (preferably with minimal
CREATE/DROP).
Yes. I just checked with a few create database/drop database. One cycle
takes about 1 full second on my laptop, so I must agree that it is slow.
--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers
Hello
I am not sure so replanning prepared plans due change of search_path is
expected in all use cases. Minimally it should be mentioned as possible
incompatibility.
Use case - move record from schema A to schema B with prepared statements
SET search_path = 'A';
PREPARE x(int) AS DELETE FROM ta
On Sat, May 04, 2013 at 10:57:44PM +0200, Dimitri Fontaine wrote:
> Other than adminpack, I know of PGQ installing their objects in
> pg_catalog. They only began doing that when switching to the CREATE
> EXTENSION facility. And they set relocatable to false.
FYI - PgQ and related modules install n
On 13 May 2013 15:28, Noah Misch wrote:
> The heavyweight locking mechanism will need to be aware of the association
> between the master and its workers.
Not sure I can see why that would be true.
ISTM that the workers need to be restricted in various ways from a
full-functioned master. If the
On 2013-05-13 14:48:52 -0400, Tom Lane wrote:
> Andres Freund writes:
> > On 2013-05-13 14:35:47 -0400, Tom Lane wrote:
> >> That is, in fact, exactly what we want to do and must do during initdb.
> >> If you change anything about this code you'll break the way the
> >> post-bootstrap initdb steps
Andres Freund writes:
> On 2013-05-13 14:35:47 -0400, Tom Lane wrote:
>> That is, in fact, exactly what we want to do and must do during initdb.
>> If you change anything about this code you'll break the way the
>> post-bootstrap initdb steps assign OIDs.
> Well, then we should use some other way
On 2013-05-13 14:35:47 -0400, Tom Lane wrote:
> Andres Freund writes:
> > I think we shouldn't check IsPostmasterEnvironment here but instead
> > IsBootstrapProcessingMode() since we otherwise can generate oids below
> > FirstNormalObjectId in --single mode.
>
> That is, in fact, exactly what we
Andres Freund writes:
> I think we shouldn't check IsPostmasterEnvironment here but instead
> IsBootstrapProcessingMode() since we otherwise can generate oids below
> FirstNormalObjectId in --single mode.
That is, in fact, exactly what we want to do and must do during initdb.
If you change anythi
On Mon, May 13, 2013 at 04:39:01PM +0200, Andres Freund wrote:
> On 2013-05-13 10:28:59 -0400, Noah Misch wrote:
> > Each worker needs to make SnapshotNow visibility decisions coherent with the
> > master. For sorting, this allows us to look up comparison functions, even
> > when the current trans
On 2013-05-13 13:40:57 -0400, Tom Lane wrote:
> Robert Haas writes:
> > On Mon, May 13, 2013 at 1:03 PM, Andres Freund
> > wrote:
> >> Why don't we just prohibit deletion/modification for anything below
> >> FirstNormalObjectId instead of using the schema as a restriction? Then
> >> we can allow
Robert Haas writes:
> On Mon, May 13, 2013 at 1:03 PM, Andres Freund wrote:
>> Why don't we just prohibit deletion/modification for anything below
>> FirstNormalObjectId instead of using the schema as a restriction? Then
>> we can allow creation for tables as well.
> We currently do, but that le
On 13.05.2013 19:59, Robert Haas wrote:
On Mon, May 13, 2013 at 12:35 PM, Tom Lane wrote:
I wrote:
Another way to fix that inconsistency is to consider that
allow_system_table_mods should gate table creations not just drops in
pg_catalog. I'm not real sure why this wasn't the case all along .
Robert Haas writes:
> On Mon, May 13, 2013 at 12:35 PM, Tom Lane wrote:
>> Uh, scratch that last comment: actually, allow_system_table_mods *did*
>> gate that, in every existing release. I bitched upthread about the fact
>> that this was changed in 9.3, and did not hear any very satisfactory
>>
On 2013-05-13 13:04:52 -0400, Robert Haas wrote:
> On Mon, May 13, 2013 at 1:03 PM, Andres Freund wrote:
> >> It disallowed it only for tables, and not for any other object type.
> >> I found that completely arbitrary. It's perfectly obvious that people
> >> want to be able to create objects in p
On Mon, May 13, 2013 at 1:03 PM, Andres Freund wrote:
>> It disallowed it only for tables, and not for any other object type.
>> I found that completely arbitrary. It's perfectly obvious that people
>> want to be able to create objects in pg_catalog; shall we adopt a rule
>> that you can put exte
On 2013-05-13 12:59:47 -0400, Robert Haas wrote:
> On Mon, May 13, 2013 at 12:35 PM, Tom Lane wrote:
> > I wrote:
> >> Another way to fix that inconsistency is to consider that
> >> allow_system_table_mods should gate table creations not just drops in
> >> pg_catalog. I'm not real sure why this w
As for the cost, if the proposed tests are indeed too costly, what is not
necessarily the case for what I have seen, I do not think that it would be a
great problem to have two set of tests, with one a superset of the other,
with some convention.
Well, tests like permission tests aren't the ex
On Mon, May 13, 2013 at 12:35 PM, Tom Lane wrote:
> I wrote:
>> Another way to fix that inconsistency is to consider that
>> allow_system_table_mods should gate table creations not just drops in
>> pg_catalog. I'm not real sure why this wasn't the case all along ...
>
> Uh, scratch that last comm
Bruce Momjian writes:
> On Fri, May 10, 2013 at 08:03:38PM -0400, Bruce Momjian wrote:
>
>> On Fri, May 10, 2013 at 12:36:21PM -0400, Evan D. Hoffman wrote:
>> > "pg.dropped.16" INTEGER /* dummy */,
>> > "pg.dropped.17" INTEGER /* dummy */,
>> > "..
I wrote:
> Another way to fix that inconsistency is to consider that
> allow_system_table_mods should gate table creations not just drops in
> pg_catalog. I'm not real sure why this wasn't the case all along ...
Uh, scratch that last comment: actually, allow_system_table_mods *did*
gate that, in
Robert Haas writes:
> On Mon, May 13, 2013 at 11:48 AM, Tom Lane wrote:
>> Heikki Linnakangas writes:
>>> Do we really want to forbid that?
>> The only alternative I see is the one proposed in
>> http://www.postgresql.org/message-id/12365.1358098...@sss.pgh.pa.us:
> Let me propose another alte
On Mon, May 13, 2013 at 11:48 AM, Tom Lane wrote:
> Heikki Linnakangas writes:
>> On 09.05.2013 18:24, Robert Haas wrote:
>>> In the attached new version of the patch, I added an explicit check to
>>> prevent relocatable extensions from being created in pg_catalog.
>
>> Do we really want to forbi
On Mon, May 13, 2013 at 10:57 AM, Tom Lane wrote:
> This approach seems to me to be likely to guarantee that the startup
> overhead for any parallel sort is so large that only fantastically
> enormous sorts will come out ahead.
>
> I think you need to think in terms of restricting the problem spac
Heikki Linnakangas writes:
> On 13.05.2013 18:14, Tom Lane wrote:
>> Couldn't we just do
>> -#if defined(__arm__) || defined(__arm)
>> +#if defined(__arm__) || defined(__arm) || defined(__aarch64__)
> That would imply falling back to swpb instruction also on aarch64, which
> won't work.
It does
Heikki Linnakangas writes:
> On 09.05.2013 18:24, Robert Haas wrote:
>> In the attached new version of the patch, I added an explicit check to
>> prevent relocatable extensions from being created in pg_catalog.
> Do we really want to forbid that?
The only alternative I see is the one proposed in
On 13.05.2013 18:14, Tom Lane wrote:
Mark Salter writes:
I used the following patch to add lock support aarch64. It is just a
copy of the arm support based on gcc builtins. Postgresql built with
this patch passes the various tests.
Couldn't we just do
-#if defined(__arm__) || defined(__arm)
I believe Tom / Andres and Fabien all have valid points.
Net-net, I believe the tests although non-negotiable, are not required to
be in make-check. For now, its the slow tests that are the pain points
here, and then I would soon try to prune them and commit once again.
Whether it goes in make-ch
+1
Pavel
Dne 13.5.2013 16:29 "Noah Misch" napsal(a):
> A memory chunk allocated through the existing palloc.h interfaces is
> limited
> to MaxAllocSize (~1 GiB). This is best for most callers; SET_VARSIZE()
> need
> not check its own 1 GiB limit, and algorithms that grow a buffer by
> doubling
On 13.05.2013 17:26, Merlin Moncure wrote:
I'm starting to wonder why we don't always use gcc atomics if they are
available and assembly implementation is not (any maybe, even if it
is).
That was discussed a while ago, but there were a lot of claims that
__sync_lock_test_and_set is broken on m
Mark Salter writes:
> I used the following patch to add lock support aarch64. It is just a
> copy of the arm support based on gcc builtins. Postgresql built with
> this patch passes the various tests.
Couldn't we just do
-#if defined(__arm__) || defined(__arm)
+#if defined(__arm__) || defined(__
On 2013-05-13 16:03:11 +0100, Greg Stark wrote:
> On Mon, May 13, 2013 at 2:49 PM, Andres Freund wrote:
> > Sure, the initial file creation will be faster. But are the actual
> > individual wal writes (small, frequently fdatasync()ed) still faster?
> > That's the critical path currently.
> > Wheth
On 2013-05-13 10:57:39 -0400, Tom Lane wrote:
> Noah Misch writes:
> > Each worker needs to make SnapshotNow visibility decisions coherent with the
> > master. For sorting, this allows us to look up comparison functions, even
> > when the current transaction created or modified those functions.
On Mon, May 13, 2013 at 2:49 PM, Andres Freund wrote:
> Sure, the initial file creation will be faster. But are the actual
> individual wal writes (small, frequently fdatasync()ed) still faster?
> That's the critical path currently.
> Whether it is pretty much depends on how the filesystem manages
On 2013-05-13 16:52:08 +0200, Fabien COELHO wrote:
>
> Hello,
>
> >>Would you be okay if there is one/a few effective create/drop (some tests
> >>check that the create or drop fails e.g. depending on permissions, which
> >>ISTM is not tested anywhere else), so that tests for various ALTER
> >>DAT
Noah Misch writes:
> Each worker needs to make SnapshotNow visibility decisions coherent with the
> master. For sorting, this allows us to look up comparison functions, even
> when the current transaction created or modified those functions. This will
> also be an essential building block for an
Hello,
Would you be okay if there is one/a few effective create/drop (some tests
check that the create or drop fails e.g. depending on permissions, which
ISTM is not tested anywhere else), so that tests for various ALTER
DATABASE commands are combined together onto these databases?
TBH, I do
Hi,
Interesting! Need to think about most, but one piece immediately came to
mind:
On 2013-05-13 10:28:59 -0400, Noah Misch wrote:
> Each worker needs to make SnapshotNow visibility decisions coherent with the
> master. For sorting, this allows us to look up comparison functions, even
> when the
It would be great if one client session could take advantage of multiple CPU
cores. EnterpriseDB wishes to start the trek into this problem space for 9.4
by implementing parallel internal (i.e. not spilling to disk) sort. This
touches on a notable subset of the infrastructure components we'll nee
A memory chunk allocated through the existing palloc.h interfaces is limited
to MaxAllocSize (~1 GiB). This is best for most callers; SET_VARSIZE() need
not check its own 1 GiB limit, and algorithms that grow a buffer by doubling
need not check for overflow. However, a handful of callers are quit
On Mon, May 13, 2013 at 8:15 AM, Heikki Linnakangas
wrote:
> On 13.05.2013 15:39, Mark Salter wrote:
>>
>> I used the following patch to add lock support aarch64. It is just a
>> copy of the arm support based on gcc builtins. Postgresql built with
>> this patch passes the various tests.
>
>
> I th
On Mon, May 13, 2013 at 7:50 AM, Heikki Linnakangas
wrote:
> The attached patch is still work-in-progress. There needs to be a configure
> test and fallback to spinlock if a CAS instruction is not available. I used
> the gcc __sync_val_compare_and_swap() builtin directly, that needs to be
> abstra
Fabien COELHO writes:
> Would you be okay if there is one/a few effective create/drop (some tests
> check that the create or drop fails e.g. depending on permissions, which
> ISTM is not tested anywhere else), so that tests for various ALTER
> DATABASE commands are combined together onto these
On Mon, May 13, 2013 at 09:36:51AM -0400, Steve Singer wrote:
> On 05/11/2013 04:58 PM, Bruce Momjian wrote:
> >On Fri, May 10, 2013 at 08:03:38PM -0400, Bruce Momjian wrote:
> >>OK, this verifies that the table had a lot of DDL churn. I have no idea
> >>how to pursue this further because I am uns
On Sun, May 12, 2013 at 10:43:41PM -0400, Evan D. Hoffman wrote:
> Adding & dropping a column resolved the problem. Currently vacuuming the new
> cluster. Thanks for your help everybody!
Great! Someday we will figure out what caused it.
-
On 2013-05-13 08:45:41 -0500, Jon Nelson wrote:
> On Mon, May 13, 2013 at 8:32 AM, Andres Freund wrote:
> > On 2013-05-12 19:41:26 -0500, Jon Nelson wrote:
> >> On Sun, May 12, 2013 at 3:46 PM, Jim Nasby wrote:
> >> > On 5/10/13 1:06 PM, Jeff Janes wrote:
> >> >>
> >> >> Of course the paranoid DB
On Mon, 2013-05-13 at 16:15 +0300, Heikki Linnakangas wrote:
> On 13.05.2013 15:39, Mark Salter wrote:
> > I used the following patch to add lock support aarch64. It is just a
> > copy of the arm support based on gcc builtins. Postgresql built with
> > this patch passes the various tests.
>
> I th
On Mon, May 13, 2013 at 8:32 AM, Andres Freund wrote:
> On 2013-05-12 19:41:26 -0500, Jon Nelson wrote:
>> On Sun, May 12, 2013 at 3:46 PM, Jim Nasby wrote:
>> > On 5/10/13 1:06 PM, Jeff Janes wrote:
>> >>
>> >> Of course the paranoid DBA could turn off restart_after_crash and do a
>> >> manual i
On 05/11/2013 04:58 PM, Bruce Momjian wrote:
On Fri, May 10, 2013 at 08:03:38PM -0400, Bruce Momjian wrote:
OK, this verifies that the table had a lot of DDL churn. I have no idea
how to pursue this further because I am unsure how we are going to
replicate the operations performed on this table
On 2013-05-12 19:41:26 -0500, Jon Nelson wrote:
> On Sun, May 12, 2013 at 3:46 PM, Jim Nasby wrote:
> > On 5/10/13 1:06 PM, Jeff Janes wrote:
> >>
> >> Of course the paranoid DBA could turn off restart_after_crash and do a
> >> manual investigation on every crash, but in that case the database wou
On Mon, May 13, 2013 at 7:49 AM, k...@rice.edu wrote:
> On Sun, May 12, 2013 at 07:41:26PM -0500, Jon Nelson wrote:
>> On Sun, May 12, 2013 at 3:46 PM, Jim Nasby wrote:
>> > On 5/10/13 1:06 PM, Jeff Janes wrote:
>> >>
>> >> Of course the paranoid DBA could turn off restart_after_crash and do a
>>
On 13.05.2013 15:39, Mark Salter wrote:
I used the following patch to add lock support aarch64. It is just a
copy of the arm support based on gcc builtins. Postgresql built with
this patch passes the various tests.
I think this needs an "#ifdef HAVE_GCC_INT_ATOMICS", like the ARM codepath.
If
I've been working on-and-off on the WAL-insert scaling patch. It's in
pretty good shape now, and I'll post it shortly, but one thing I noticed
is that it benefits a lot from using an atomic compare-and-swap
instruction for the contention-critical part.
I realized that we could also use compare
On Sun, May 12, 2013 at 07:41:26PM -0500, Jon Nelson wrote:
> On Sun, May 12, 2013 at 3:46 PM, Jim Nasby wrote:
> > On 5/10/13 1:06 PM, Jeff Janes wrote:
> >>
> >> Of course the paranoid DBA could turn off restart_after_crash and do a
> >> manual investigation on every crash, but in that case the
On Sun, May 12, 2013 at 03:46:00PM -0500, Jim Nasby wrote:
> On 5/10/13 1:06 PM, Jeff Janes wrote:
> >Of course the paranoid DBA could turn off restart_after_crash and do a
> >manual investigation on every crash, but in that case the database would
> >refuse to restart even in the case where it p
I used the following patch to add lock support aarch64. It is just a
copy of the arm support based on gcc builtins. Postgresql built with
this patch passes the various tests.
diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h
index d4a783f..624a73b 100644
--- a/src/include/st
> This code seems to me expecting for psql to send password without
> closing current connnection.On the other hand psql does as
> follows.
>
> bin/psql/startup.c: 227
>>pset.db = PQconnectdbParams(keywords, values, true);
>>free(keywords);
>>free(values);
>>
>>if (PQstatus(pset.db)
On Monday, May 13, 2013 1:13 PM Heikki Linnakangas wrote:
> On 13.05.2013 06:07, Amit Kapila wrote:
> > On Monday, May 13, 2013 5:54 AM Kyotaro HORIGUCHI wrote:
> >> Heikki said in the fist message in this thread that he suspected
> >> the cause of the failure he had seen to be wrong TLI on whitch
On 09.05.2013 18:24, Robert Haas wrote:
On Sat, May 4, 2013 at 3:59 PM, Tom Lane wrote:
Given the lack of any good alternative proposals, I think we should
press ahead with this approach. We still need doc updates and fixes
for the affected contrib module(s), though. Also, in view of point 2,
On 13.05.2013 06:07, Amit Kapila wrote:
On Monday, May 13, 2013 5:54 AM Kyotaro HORIGUCHI wrote:
Heikki said in the fist message in this thread that he suspected
the cause of the failure he had seen to be wrong TLI on whitch
checkpointer runs. Nevertheless, the patch you suggested for me
looks f
Hello,
> > auth_failed() in src/backend/libpq/auth.c intentionally logs nothing for
> > STATUS_EOF status (ie, client closed the connection without responding).
> > But it looks like the PAM code path doesn't have a way to return that
> > status code, even when pam_passwd_conv_proc() knows that th
82 matches
Mail list logo
