[HACKERS] PREPARE dynamic SQL in plpgsql

2016-03-11 Thread Koichi Suzuki
Hi, Does someone know how to prepare a synamic SQL statement in plpgsql? All the examples, PG documents describe only about preparing static SQL statement. Thank you; -- Koichi Suzuki

Re: [HACKERS] Documenting the Frontend/Backend Protocol update criteria

2014-06-02 Thread Koichi Suzuki
e.pdf Hope it helps; --- Koichi Suzuki 2014-06-02 7:22 GMT+09:00 Mikko Tiihonen : > Hi, > > Currently the criteria on updating the F/B protocol is undefined. This makes > it hard to update the protocol going forward. It makes it also hard to write > library/driver/application implementat

Re: [HACKERS] [pgsql-cluster-hackers] 3rd Cluster Hackers Summit, May 15th in Ottawa

2012-02-28 Thread Koichi Suzuki
Hi, This is Koichi. I'm joining the summit. I've heard at least two or more people will join from NTT.I'm expecting a couple of people from EDB too. Regards; ------ Koichi Suzuki 2012/2/29 Josh Berkus : > All, > > First, I've added a wiki page: > &

[HACKERS] Re: [pgsql-cluster-hackers] 3rd Cluster Hackers Summit, May 15th in Ottawa

2012-02-27 Thread Koichi Suzuki
Josh; I belive this is an important agenda too, though I'm not sure if we can assing 90minutes to him. We need to organize the schedule shortly. >From XC, we will present progress, release schedule and further technical >challanges, including adding/removing nodes on-the-fly and HA feature.

Re: [HACKERS] LOCK for non-tables

2011-01-11 Thread Koichi Suzuki
At present, XC does not seem to need locks to maintain cluster-wide integrity because it is maintained centrally in GTM. If application needs to do this, for example, to synchronize between different clusters, XC needs this capability obviously. -- Koichi Suzuki 2011/1/11 Tatsuo Ishii

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-14 Thread Koichi Suzuki
Robert; Thank you very much for your advice. Indeed, I'm considering to change the license to PostgreSQL's one. It may take a bit more though... ------ Koichi Suzuki 2010/12/15 Robert Haas : > On Tue, Dec 7, 2010 at 3:23 AM, Koichi Suzuki wrote: >> This is what P

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-07 Thread Koichi Suzuki
This is what Postgres-XC is doing between a coordinator and a datanode.Coordinator may correspond to poolers/loadbalancers. Does anyone think it makes sense to extract XC implementation of snapshot shipping to PostgreSQL itself? Cheers; -- Koichi Suzuki 2010/12/7 Stefan

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Koichi Suzuki
We may need other means to ensure that the snapshot is available on the slave. It could be a bit too early to use the snapshot on the slave depending upon the delay of WAL replay. -- Koichi Suzuki 2010/12/7 Tom Lane : > marcin mank writes: >> On Sun, Dec 5, 2010 at 7:28 PM,

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-05 Thread Koichi Suzuki
Thank you Joachim; Yes, and the current patch requires the original (publisher) transaction is alive to prevent RecentXmin updated. I hope this restriction is acceptable if publishing/subscribing is provided via functions, not statements. Cheers; -- Koichi Suzuki 2010/12/6 Joachim

Re: [HACKERS] Horizontal Write Scaling

2010-11-25 Thread Koichi Suzuki
l > copy and only exchange locking information and transactional changes > sounds like much less traffic overall. > > Regards > > Markus Wanner > > > [1]: The Dangers of Replication and a Solution, Gray et al, In Proc. of > the SIGMOD Conf., 1996, > http://research.microsoft.com/apps/pubs/default.aspx?id=68247 > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > Cheers; --- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] pg_lesslog 1.4.2 beta now available

2010-06-07 Thread Koichi Suzuki
8.4, version 1.4.1 is available. Toward the final release, I will continue to add the test scripts to test as many types of WAL record as possible. Regards; -- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

[HACKERS] How to generate specific WAL records again for PostgreSQL 9.0?

2010-06-07 Thread Koichi Suzuki
Hi, I've ported pg_lesslog to PostgreSQL 9.0 and I'm testing it. I'm not successful to produce the following WAL records. XLOG_HEAP_INIT_PAGE XLOG_BTREE_REUSE_PAGE XLOG_GIST_PAGE_DELETE XLOG_RELMAP_UPDATE Any suggestions are welcome. Thank you very much; ------ Koichi Suzuk

Re: [HACKERS] Re: [ANNOUNCE] Bug-fix and new feature of pg_lesslog is released

2010-05-12 Thread Koichi Suzuki
transfer to transfer and it is not simple to handle. Regards; -- Koichi Suzuki 2010/5/13 Takahiro Itagaki : > > Tom Lane wrote: > >> Bruce Momjian writes: >> > Yes, I would love to get this into /contrib for PG 9.1! >> >> How much are people really goi

Re: [HACKERS] Re: [ANNOUNCE] Bug-fix and new feature of pg_lesslog is released

2010-05-12 Thread Koichi Suzuki
uated this two years ago, recovery speed was as good as those with full page image, depending upon application and tuning, of course. This is a separate tool and can be used in various scenes. Regards; -- Koichi Suzuki 2010/5/13 Takahiro Itagaki : > > Tom Lane wrote: > &g

[HACKERS] Bug-fix and new feature of pg_lesslog is released

2010-05-12 Thread Koichi Suzuki
included in contrib module if it is ported to PostgreSQL 9.1. Best Regards; -- Koichi Suzuki # pg_lesslog is a project to compress large WAL segments when they're stored and restore then for recovery. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] How to generate specific WAL records?

2010-04-18 Thread Koichi Suzuki
gests me how. Thank you very much in advance; -- Koichi Suzuki 2010/4/15 Bruce Momjian : > Kevin Grittner wrote: >> Koichi Suzuki wrote: >> > 2010/4/14 Simon Riggs : >> >> >> It would be a very useful test case to publish. >> >> > I&#

Re: [HACKERS] How to generate specific WAL records?

2010-04-14 Thread Koichi Suzuki
Thanks for encouraging comment. I'm still struggling to generate remaing WAL records. -- Koichi Suzuki 2010/4/14 Simon Riggs : > On Wed, 2010-04-14 at 11:13 +0900, Koichi Suzuki wrote: > >> Thank you for a great advice.   I successfully generated all the WAL >&g

Re: [HACKERS] How to generate specific WAL records?

2010-04-13 Thread Koichi Suzuki
sk Oleg-san how to generate GIN-related WALs. Thank you very much; ------ Koichi Suzuki 2010/4/14 Alvaro Herrera : > Koichi Suzuki escribió: >> Hi, >> >> Does anyone know how to generate the following WAL records from psql? >> >> I'm now fixing pg_lesslog,

[HACKERS] How to generate specific WAL records?

2010-04-13 Thread Koichi Suzuki
TE So far, I'm using conventional BTREE, btree_gin and btree_gist for the test, as well as 2PC and savepoint. Any information is welcome. Thank you very much in advance; -- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your sub

Re: [GENERAL] [HACKERS] Bug on pg_lesslog

2010-02-11 Thread Koichi Suzuki
both each WAL record and whole WAL segment. -- Koichi Suzuki 2010/2/12 Koichi Suzuki : > Thank you very much for the advice.   Yes I think it should go to > announce.   I will post a message. > -- > Koichi Suzuki > > > > 2010/2/12 Karl Denninger : >>

Re: [GENERAL] [HACKERS] Bug on pg_lesslog

2010-02-11 Thread Koichi Suzuki
Thank you very much for the advice. Yes I think it should go to announce. I will post a message. -- Koichi Suzuki 2010/2/12 Karl Denninger : > Joshua D. Drake wrote: > > On Thu, 2010-02-11 at 23:39 +0900, Koichi Suzuki wrote: > > > Dear Folks; > > A very ser

[HACKERS] Bug on pg_lesslog

2010-02-11 Thread Koichi Suzuki
I'll upload the new version ASAP. Warmest Regards; ------ Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Bug on pg_lesslog

2010-02-11 Thread Koichi Suzuki
I'll upload the new version ASAP. Warmest Regards; ------ Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New PostgreSQL Committers

2009-12-07 Thread Koichi Suzuki
gt;>> >> >> +1 > > Outstanding!  Congratulations, all! +1 Appreciate for their contributions. Congratulations all! Koichi Suzuki > > -Kevin > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscriptio

[HACKERS] xml2 in 8.4 still alive?

2009-05-06 Thread Koichi Suzuki
HI, Although xml2 was announced to be removed from 8.4, I found 8.4beta1 documentation has xml2 description. Does it mean that xml2 is available in 8.4 as well? -- -- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: Remaining items for 8.4 (was Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1710))

2009-03-17 Thread Koichi Suzuki
Sorry I see the comment. I'll continue the work to fulfill the comment. 2009/3/17 Heikki Linnakangas : > Koichi Suzuki wrote: >> >> I believe all the issues pointed out in >> http://archives.postgresql.org//pgsql-hackers/2008-10/msg01387.php as >> been covered in

Re: Remaining items for 8.4 (was Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1710))

2009-03-16 Thread Koichi Suzuki
> > -- >  Heikki Linnakangas >  EnterpriseDB   http://www.enterprisedb.com > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- -- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] V4 of PITR performance improvement for 8.4

2009-03-10 Thread Koichi Suzuki
cords, when you reach the min safe starting point. This arrangement can be done with Hot Standby and Sync.Rep, right? So far, it sounds that we need to add a code to handle if malloc() fails (OOM). In this case, possible way could be to skip whole readahead, although the rest of the recovery may fail b

[HACKERS] Pg_lesslog 1.2 released

2009-03-09 Thread Koichi Suzuki
=1000310 -- -- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] V4 of PITR performance improvement for 8.4

2009-03-04 Thread Koichi Suzuki
Appreciate for your kind help! 2009/3/3 Fujii Masao : > On Tue, Mar 3, 2009 at 1:47 PM, Fujii Masao wrote: >> Hi Suzuki-san, >> >> On Thu, Feb 26, 2009 at 5:03 AM, Koichi Suzuki wrote: >>> My reply to Gregory's comment didn't have any objections.  

Re: [HACKERS] V4 of PITR performance improvement for 8.4

2009-02-25 Thread Koichi Suzuki
Hi, My reply to Gregory's comment didn't have any objections. I believe, as I posted to Wiki page, latest posted patch is okay and waiting for review. 2009/2/24 Robert Haas : > On Sun, Jan 25, 2009 at 7:15 AM, Gregory Stark wrote: >> Koichi Suzuki writes: >> >

Re: [HACKERS] 8.4 release planning

2009-02-05 Thread Koichi Suzuki
hould be assigned to that? > > Regards, > > -- > Fujii Masao > NIPPON TELEGRAPH AND TELEPHONE CORPORATION > NTT Open Source Software Center > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://w

Re: [HACKERS] V4 of PITR performance improvement for 8.4

2009-01-28 Thread Koichi Suzuki
Hi, This is also a reply to your latest post.I'm replying to the older one because I need original text. 2009/1/26 Koichi Suzuki : > Hi, > > It's simply because we should not refer to system catalog during the recovery. This is the reason why I didn't use PrefetchBuf

Re: [HACKERS] V4 of PITR performance improvement for 8.4

2009-01-25 Thread Koichi Suzuki
Hi, It's simply because we should not refer to system catalog during the recovery. 2009/1/25 Gregory Stark : > > Koichi Suzuki writes: > >> Please find enclosed 2nd patch of pg_readahead which include a patch >> to bufer manager to skip prefetch of pages already in sh

Re: [HACKERS] Documenting pglesslog

2009-01-14 Thread Koichi Suzuki
Pg_readahead uses posix_fadvise, which is included in Greg's patch and I've already posted pg_readahead patch integrated into the core. Integration with snc.rep. will be a separate patch which will be posted in a couple of days. 2009/1/14 Bruce Momjian : > Koichi Suzuki wrote: &g

Re: [HACKERS] Documenting pglesslog

2009-01-13 Thread Koichi Suzuki
: > Koichi Suzuki wrote: >> Hi, >> >> I have no intention to make pglesslog to conflict to PostgreSQL >> license. Any advice is welcome to make pglesslog available without >> any license concern. > > I certainly have no concerns. > >> I've a qu

Re: [HACKERS] V4 of PITR performance improvement for 8.4

2009-01-13 Thread Koichi Suzuki
Sorry, I didn't attatch the patch file. This is the second try. 2009/1/12 Koichi Suzuki : > This is V4 patch to improve file read during startup for review. > > Basic algorithm is same as V3 but works with Gregory's fadvise patch. > > http://archives.postgresql.org/pgsql

Re: [HACKERS] Documenting pglesslog

2009-01-12 Thread Koichi Suzuki
ning, Services and Support > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- -- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] V4 of PITR performance improvement for 8.4

2009-01-12 Thread Koichi Suzuki
does not exist. We still need a patch to work this with syncronous replication, which will be posted by the end of this week. Regards; -- -- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresq

Re: [HACKERS] V2 of PITR performance improvement for 8.4

2008-12-26 Thread Koichi Suzuki
I'm now writing v3 patch of PITR improvement, to work with sync.rep and Hot Standby.Would like to change the thread. 2008/12/12 Pavan Deolasee : > On Fri, Dec 12, 2008 at 9:08 AM, Koichi Suzuki wrote: >> Hmmm, it's really like pg_readahead needs to be included in the

Re: [HACKERS] V2 of PITR performance improvement for 8.4

2008-12-11 Thread Koichi Suzuki
Hmmm, it's really like pg_readahead needs to be included in the core. I don't think it's a big work and will try to do this. 2008/12/9 Fujii Masao : > Hi, > > On Mon, Dec 8, 2008 at 2:54 PM, Koichi Suzuki wrote: >> I understood your point. In the case of synch

Re: [HACKERS] V2 of PITR performance improvement for 8.4

2008-12-07 Thread Koichi Suzuki
lp in this case with no modification. 2008/12/4 Simon Riggs <[EMAIL PROTECTED]>: > > On Wed, 2008-12-03 at 14:22 +0900, Koichi Suzuki wrote: > >> > There's clearly a huge gain using prefetch, when we have >> > full_page_writes = off. But that does make me thin

Re: [HACKERS] V2 of PITR performance improvement for 8.4

2008-12-03 Thread Koichi Suzuki
Agreed. I borrowed WAL parsing code from XLogdump and I think WAL parsing should be another candidate. 2008/12/3 Fujii Masao <[EMAIL PROTECTED]>: > Hi, > > On Thu, Nov 27, 2008 at 9:04 PM, Koichi Suzuki <[EMAIL PROTECTED]> wrote: >> Please find enclosed a revised ve

Re: [HACKERS] Hot Standby (commit fest version - v5)

2008-12-02 Thread Koichi Suzuki
Thank you for your advise. I'll edit the Wiki page. 2008/12/3 Pavan Deolasee <[EMAIL PROTECTED]>: > > > On Wed, Dec 3, 2008 at 11:00 AM, Koichi Suzuki <[EMAIL PROTECTED]> wrote: >> >> Hi, >> >> I found that no one is registered as hot standby r

Re: [HACKERS] Hot Standby (commit fest version - v5)

2008-12-02 Thread Koichi Suzuki
ed. Do we still think that? > > * locking correctness around flat file refresh still not enabled > > * need some discussiona round how to handle max_connections changes > cleanly. -- -- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] where is the last hot standby patch?

2008-12-02 Thread Koichi Suzuki
ostgresql.org/pgsql-hackers/2008-11/msg01333.php > > Simon, did you by any chance send something that got eaten by the > message size limit? > > ...Robert > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > ht

Re: [HACKERS] V2 of PITR performance improvement for 8.4

2008-12-02 Thread Koichi Suzuki
Riggs <[EMAIL PROTECTED]>: > > On Thu, 2008-11-27 at 21:04 +0900, Koichi Suzuki wrote: > >> We ran the >> benchmark for on hour with chekpoint timeout 30min and completion_target 0.5. >> Then, coll

Re: [HACKERS] Proposal of PITR performance improvement for 8.4.

2008-10-29 Thread Koichi Suzuki
the duplicates. I wonder how much CPU time is spent >> on that? > > Removing duplicates seems like it will save CPU. If we invoke posix_fadvise() to the blocks already in the kernel cache, this call will just do nothing but consume some overhead in the kernel. I think duplicate removal saves more. > > -- > Simon Riggs www.2ndQuadrant.com > PostgreSQL Training, Services and Support > > -- -- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal of PITR performance improvement for 8.4.

2008-10-29 Thread Koichi Suzuki
e log compressed with pg_compresslog. I'll test if it's okay in the case full_page_writes=off. Anyway, I'd like to keep my proposal for 8.4 and continue the test and evaluation to report to the mailing list. I'll also change the whole code to run in the core. --- Koichi

Re: [HACKERS] Proposal of PITR performance improvement for 8.4.

2008-10-28 Thread Koichi Suzuki
h new record.Improving WAL record format as 2. will help to simplify this as well. 2008/10/28 Heikki Linnakangas <[EMAIL PROTECTED]>: > Gregory Stark wrote: >> >> "Koichi Suzuki" <[EMAIL PROTECTED]> writes: >> >>> This is my first proposal o

Re: [HACKERS] Proposal of PITR performance improvement for 8.4.

2008-10-28 Thread Koichi Suzuki
it. Yes, this is what the code is doing. Prefetch function returns LSN where prefetch was performed. Redo routine has to call prefetch if it is going to read WAL record beyond this. -- ------ Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To mak

Re: [HACKERS] Proposal of PITR performance improvement for 8.4.

2008-10-28 Thread Koichi Suzuki
e to 8.5 or further and we must call for research works.So far, I think it is reasonable to keep improving specific code. I'd like to hear some more about these. I'm more than happy to write all the code inside PG core to avoid overhead to create another process. --- Koichi S

[HACKERS] Proposal of PITR performance improvement for 8.4.

2008-10-28 Thread Koichi Suzuki
done by posix_fadvise() as proposed in index scan improvement. Details of the implementation will be found in README file in the material. -- -- Koichi Suzuki pg_readahead_20081028.tar.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Making pg_standby compression-friendly

2008-10-27 Thread Koichi Suzuki
og in archive recovery, pg_core can call some backend to re-compress the archive log for later use. I'm not sure if archive_commend argument works in this scene too, but very sceptical not. Any further thoughts? ----- Koichi Suzuki 2008/10/25 Charles Duffy <[EMAIL PROTECTED]>:

Re: [HACKERS] Making pg_standby compression-friendly

2008-10-23 Thread Koichi Suzuki
n is attached, intended only to kickstart > discussion; I'm not attached to either its implementation or its > proposed command-line syntax. > > Thoughts? > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscr

[HACKERS] Full page write improvement: new WAL archive command (beta) released

2008-09-01 Thread Koichi Suzuki
ave been much more complicated. -- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_stat_statements

2008-06-15 Thread Koichi Suzuki
s to write statements into disks, 2 would be a solution. > 3 will be needed if sending statements to loggger itself is the reason > of the overhead. > > Regards, > --- > ITAGAKI Takahiro > NTT Open Source Software Center > > > > -- > Sent via pgsql-hackers mailing l

Re: [HACKERS] Core team statement on replication in PostgreSQL

2008-06-09 Thread Koichi Suzuki
at comes > with shipping smaller packets in a real-time implementation. > > -- > * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > h

Re: [HACKERS] Core team statement on replication in PostgreSQL

2008-06-04 Thread Koichi Suzuki
If the version of the master and the slave is different and we'd still like to allow log shipping replication, we need a negotiation if WAL format for the two is compatible. I hope it is not in our scope and I'm worrying too much. 2008/6/5 Tom Lane <[EMAIL PROTECTED]>:

Re: [HACKERS] Core team statement on replication in PostgreSQL

2008-06-04 Thread Koichi Suzuki
seDB http://www.enterprisedb.com > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- -- Koichi Suzuki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] intercepting WAL writes

2008-05-28 Thread Koichi Suzuki
ource code for xlogviewer and > xlogdump; seems like a reasonable place to start. > > Thanks for your help, > > Mike > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpr

Re: [HACKERS] Benchmark for GiST index?

2007-12-20 Thread Koichi Suzuki
Oleg; Thanks for the information. I'll try your code. 2007/12/20, Oleg Bartunov <[EMAIL PROTECTED]>: > On Thu, 20 Dec 2007, Koichi Suzuki wrote: > > > I'm trying to run some GiST activities and see XLOG record structure. > > Koichi, I tested update speed of Gi

Re: [HACKERS] Benchmark for GiST index?

2007-12-19 Thread Koichi Suzuki
search > http://www.sigaev.ru/cvsweb/cvsweb.cgi/ftsbench/ > and several others, based on it. > > > Koichi Suzuki wrote: > > Hi, > > > > Does anybody have an information about GiST index benchmark? I'm > > planning to run GiST benchmark and analyze WAL s

[HACKERS] Benchmark for GiST index?

2007-12-18 Thread Koichi Suzuki
Hi, Does anybody have an information about GiST index benchmark? I'm planning to run GiST benchmark and analyze WAL structure to see PG's dynamic behavior. Regards; ------ Koichi Suzuki ---(end of broadcast)--- TIP 3: Have you c

[HACKERS] GiST crash recovery (potential problems?)

2007-11-15 Thread Koichi Suzuki
Hi, In GiST, I found that after the crash recovery, NSN and right page link are initialized. We can search all the records in this case but performance may become a little worse because we cannot traverse leaves. I'm not sure if it is preffered behavior. -- Koichi S

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-05-21 Thread Koichi Suzuki
ou've added information on btree strip log records, which anables to produce corresponding incremental logs from the full page writes. 2007/5/21, Tom Lane <[EMAIL PROTECTED]>: Koichi Suzuki <[EMAIL PROTECTED]> writes: > As replied to "Patch queue triage" by Tom, he

Re: [HACKERS] Patch queue triage

2007-05-07 Thread Koichi Suzuki
Sorry for late responce due to very long vacation season in Japan. Tom Lane wrote: > > * Re: [PATCHES] [HACKERS] Full page writes improvement, code update > > /Koichi Suzuki/ > > > > I feel that we have to insist that this be modified to not create any WAL > >

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-05-07 Thread Koichi Suzuki
I'd be very grateful if this patch can be considered again. Best Regards; -- - Koichi Suzuki diff -cr pgsql_org/src/backend/access/transam/xlog.c pgsql/src/backend/access/transam/xlog.c *** pgsql_org/src/backend/access/transam/xlog.c 2007-05-02 15:56:38.0 +0900 ---

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-26 Thread Koichi Suzuki
ullpage_optimization quite baffling and I think our general user base will find it even more so. Now that I have Koichi's explanation of the problem, I vote for simply slaving this to the PITR settings and not having a separate option at all. Could I have more specific suggestion on this

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-25 Thread Koichi Suzuki
://www.postgresql.org/docs/faq -- - Koichi Suzuki ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-23 Thread Koichi Suzuki
log-shipping and I agree. If we compare compression with gzip or other general purpose compression, compression ratio, CPU usage and I/O by pg_compresslog are all quite better than those in gzip. Please let me know if you intended defferently. Regards; -- - Koichi Suzuki ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-22 Thread Koichi Suzuki
o make redo routines confused with the dummy full page writes, as Simon suggested. So far, it works fine. Yes, Tom didn't like "LSN replacing" eighter. I withdraw my concern regarding pg_decompresslog. Your work in this area is extremely valuable and I hope

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-20 Thread Koichi Suzuki
gfoundry; with the hope > that it would be merged to contrib or core in 8.4. Frankly the > compress/decompress code needs work anyway before it could be > merged (eg, I noted a distinct lack of I/O error checking). > > regards, tom lane > -- - K

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-20 Thread Koichi Suzuki
space saved by omitting full pages. Agreed. I don't want to start touching something that works so well. We've been thinking about doing this for at least 3 years now, so I don't see any reason to baulk at it now. I'm happy with Koichi-san's patch as-is, assuming f

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-19 Thread Koichi Suzuki
cord position during replay ? Add simple short WAL records in pg_compresslog like: advance LSN by 8192 bytes. Andreas -- - Koichi Suzuki ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-12 Thread Koichi Suzuki
logs which pg_compresslog produces. For recovery, we have to restore LSN as the original WAL. Pg_decompresslog restores removed full page writes as a dumm records so that recovery redo functions won't be confused. Regards; Andreas -- ----- Koichi Suzuki ---(e

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-11 Thread Koichi Suzuki
to be a winner even if it just does not degrade performance. -- Koichi Suzuki ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-11 Thread Koichi Suzuki
The score below was taken based on 8.2 code, not 8.3 code. So I don't think the below measure is introduced only in 8.3 code. Tom Lane wrote: > Koichi Suzuki <[EMAIL PROTECTED]> writes: >> For more information, when checkpoint interval is one hour, the amount >> of th

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-10 Thread Koichi Suzuki
it's reasonable measure. Again, in my proposal, it is not the issue to increase run time performance. Issue is to decrease the size of archive log to save the storage. Regards; Tom Lane wrote: > Koichi Suzuki <[EMAIL PROTECTED]> writes: >> My proposal is to remove unnecessary f

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-05 Thread Koichi Suzuki
. Regards; Simon Riggs wrote: On Tue, 2007-04-03 at 19:45 +0900, Koichi Suzuki wrote: Bruce Momjian wrote: Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches Thank you very much for including. Attached is an update of the

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-03 Thread Koichi Suzuki
Bruce Momjian wrote: Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches Thank you very much for including. Attached is an update of the patch according to Simon Riggs's comment about GUC name. Regards; -- K

Re: [PATCHES] [HACKERS] Full page writes improvement, code update again.

2007-04-03 Thread Koichi Suzuki
Here's third revision of WAL archival optimization patch. GUC parameter name was changed to wal_add_optimization_info. Regards; -- Koichi Suzuki 20070403_pg_lesslog.tar.gz Description: application/gzip ---(end of broadcast)--- TIP

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-04-01 Thread Koichi Suzuki
to indicate number of full page writes included in the record. In my proposal, this unused bit is used to mark that full page writes must not be removed at offline optimization by pg_compresslog. Regards; -- -- Koichi Suzuki -- Koichi Suzuki ---(end of broadcast)-

Re: [HACKERS] [PATCHES] Full page writes improvement, code update

2007-03-30 Thread Koichi Suzuki
thread to continue my post. Sorry for confusion. Please refer to the original thread about this discussion. Best Regards; -- ------ Koichi Suzuki ---(end of broadcast)--- TIP 4: Have you searched our list archives?

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-29 Thread Koichi Suzuki
Hi, Here's a patch reflected some of Simon's comments. 1) Removed an elog call in a critical section. 2) Changed the name of the commands, pg_complesslog and pg_decompresslog. 3) Changed diff option to make a patch. -- Koichi Suzuki pg_lesslog.tgz Description: B

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-29 Thread Koichi Suzuki
sh recovery", "needed in archive recovery" and so on. I don't insist these names. It's very helpful if you have any suggestion to reflect what it really means. Regards; -- Koichi Suzuki ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-29 Thread Koichi Suzuki
Hi, Here're some feedback to the comment: Simon Riggs wrote: On Wed, 2007-03-28 at 10:54 +0900, Koichi Suzuki wrote: As written below, full page write can be categolized as follows: 1) Needed for crash recovery: first page update after each checkpoint. This has to be kept in WAL. 2) N

Re: [PATCHES] [HACKERS] Full page writes improvement, code update

2007-03-27 Thread Koichi Suzuki
Simon; Thanks a lot for your comments/advices. I'd like to write some feedback. Simon Riggs wrote: On Tue, 2007-03-27 at 11:52 +0900, Koichi Suzuki wrote: Here's an update of a code to improve full page writes as proposed in http://archives.postgresql.org/pgsql-hackers/2007-01/ms

[HACKERS] Full page writes improvement, code update

2007-03-26 Thread Koichi Suzuki
oposal but still would like to hear comments/opinions/advices. Regards; -- Koichi Suzuki pg_lesslog.tgz Description: Binary data ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at

Re: [HACKERS] Archive log compression keeping physical log availablein the crash recovery

2007-02-12 Thread Koichi Suzuki
that is what you would tune if you are concerned about fullpage overhead. Andreas ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq -- Koichi Suzuki --

Re: [HACKERS] Archive log compression keeping physical log available in the crash recovery

2007-02-12 Thread Koichi Suzuki
all the data actually hits > disk. That cuts into your compression ratio... > > Have a nice day, -- Koichi Suzuki ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [HACKERS] Archive log compression keeping physical log available in the crash recovery

2007-02-09 Thread Koichi Suzuki
d results by (b|g)zip'ing the WAL files in the archive. I quick test on my laptop shows over a 4x reduction in size. Presumably that'd be even larger if you increased the size of WAL segments. On Jan 29, 2007, at 2:15 AM, Koichi Suzuki wrote: This is a proposal for archive log compres

Re: [HACKERS] [PATCHES] Full page writes improvement

2007-02-08 Thread Koichi Suzuki
two commands can be treated as contrib and the patch itself does work if WAL is simply copied to the archive directory. Regards; Koichi Suzuki Tom Lane wrote: > Koichi Suzuki <[EMAIL PROTECTED]> writes: >> Tom Lane wrote: >>> Doesn't this break crash recovery on PITR sla

Re: [HACKERS] Archive log compression keeping physical log available in the crash recovery

2007-02-08 Thread Koichi Suzuki
results by (b|g)zip'ing the WAL files in the archive. I quick test on my laptop shows over a 4x reduction in size. Presumably that'd be even larger if you increased the size of WAL segments. On Jan 29, 2007, at 2:15 AM, Koichi Suzuki wrote: This is a proposal for archive log compressio

Re: [HACKERS] [PATCHES] Full page writes improvement

2007-02-01 Thread Koichi Suzuki
Tom Lane wrote: > Koichi Suzuki <[EMAIL PROTECTED]> writes: >> Tom Lane wrote: >>> Doesn't this break crash recovery on PITR slaves? > >> Compressed archive log contains the same data as full_page_writes off >> case. So the influence to PIT

Re: [HACKERS] [PATCHES] Full page writes improvement

2007-02-01 Thread Koichi Suzuki
Tom Lane wrote: > Koichi Suzuki <[EMAIL PROTECTED]> writes: >> Here's an idea and a patch for full page writes improvement. > >> Idea: >> (1) keep full page writes for ordinary WAL, make them available during >> the crash recovery, -> recovery from in

[HACKERS] Archive log compression keeping physical log available in the crash recovery

2007-01-28 Thread Koichi Suzuki
as small as the case with full_page_writes off, while the physical log is still available in the crash recovery, maintaining the crash recovery chance. Comments, questions and any input is welcome. - Koichi Suzuki, NTT Open Source Center -- Koichi Suzuki ---

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-07 Thread Koichi Suzuki
Tom Lane wrote: > Zdenek Kotala <[EMAIL PROTECTED]> writes: >> Koichi Suzuki wrote: >>> I've once proposed a patch for 64bit transaction ID, but this causes >>> some overhead to each tuple (XMIN and XMAX). > >> Did you check performance on 32-bit

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Koichi Suzuki
low 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match -- Koichi Suzuki ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org

[HACKERS] A couple of patches for PostgreSQL 64bit support

2005-07-07 Thread Koichi Suzuki
Hi, all, I have posted a couple of patches with regard to 64bit environment support to PATCHES ml. It expands size of shared memory to 64bit space and extends XID to 64bit. Please take a look at it. -- --- Koichi Suzuki Open Source Engineeering

  1   2   >