Re: [HACKERS] New FSM allocation policy
Decibel! wrote: On Sep 5, 2008, at 9:43 PM, Bruce Momjian wrote: One other thing to keep in mind is that VACUUM can reduce a table's size if the trailing blocks are empty, so there is some gain if the earlier parts of the table are preferred for inserts. > Yeah; I would actually really, really like to see a mode you could set on a table that says "I want to try and shrink this table". One of the things that would mean is that the FSM should prefer pages at the beginning of the heap. Not sure how exactly that should work, but it should be pretty easy to do with the new FSM data structure. Perhaps VACUUM should just reset the "next pointers" as it goes. Also related to this is the idea of asking the FSM for pages within a specific range so that you can try and maintain cluster order on a table. You would look in the clustering index for the closest value to your key and where it is in the heap and then ask for a page in that neighborhood. (You'd probably want to look at more than just one index tuple, but you get the idea). The new FSM data structure should make that much easier to implement as well, as it supports naturally the operation "give me page with X bytes of free space, as close as possible to page Y". -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PLUGINS Functionlity in Win32 build scripts
MUHAMMAD ASIF wrote: During the integration of pldebugger ( http://pgfoundry.org/projects/edb-debugger ) with postgres on windows I faced a problem that plugins are not being copied to the lib/plugins directory. Plugins should be copied in (Installation dir)lib/plugins to work properly. To solve this problem I added PLUGINS logic in the Windows Perl build scripts of PostgreSQL 8.3.3. It searches for PLUGINS variable in the contrib Makefile and processes "PLUGINS" as "MODULES" and copies the generated plugin library to the (Installation dir)lib/plugins. Please find the attached plugin.patch file. Thanks. We'll need the same logic on Unix-platforms as well. I've added this to the November commitfest Wiki page. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] rmgr hooks and contrib/rmgr_hook
Simon Riggs wrote: On Tue, 2008-09-02 at 11:39 -0400, Tom Lane wrote: , I'm not seeing the use-case for an rmgr that only executes during recovery; in fact I'm not entirely sure that I see a use-case for this entire patch. Where are the WAL records that the "loadable rmgr" processes going to come from? There is ample reason to do this. I covered this in my first post, please re-read up thread. You have commented on this post already, so I'm suprised by your comments. I think there's two different use cases here: 1. Filter WAL that's already generated, or is being generated by an unmodified PostgreSQL instance. 2. Allow external modules to define new resource managers. The examples you posted with the patch were of type 1. That's a very valid use case, the example of only restoring a single database seems like a useful tool. Another tool like that is pglesslog, although that one couldn't actually be implemented with these hooks. I'm sure there's more tricks like that people would find useful, if the tools existed. The importance of the WAL will only increase as more people start to use it for PITR, replication etc. The 2nd use case, however, I find pretty unconvincing. I can't think of a good example of that. Anything that needs to define its own resource manager is very low-level stuff, and probably needs to go into the core anyway. So, let's focus on the 1st use case. I think a better approach for that is to implement the filters as external programs, like pglesslog. It allows more flexibility, although it also means that you can't rely on existing backend functions to manipulate the WAL. I'd love to see a "WAL toolkit" on pgfoundry, with tools like the filter to only restore a single database, pglesslog, a WAL record viewer etc. A while ago, you also talked about replacing the Slony triggers in the master with a tool that reads the WAL; another good example of an external tool that needs to read WAL. The toolkit could provide some sort of a framework and common user interface to read and write WAL files for all those tools. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Transaction Snapshots and Hot Standby
Heikki Linnakangas wrote: BTW, we haven't talked about how to acquire a snapshot in the slave. You'll somehow need to know which transactions have not yet committed, but will in the future. In the master, we keep track of in-progress transaction in the ProcArray, so I suppose we'll need to do the same in the slave. Very similar to prepared transactions, actually. I believe the Abort records, which are not actually needed for normal operation, become critical here. The slave will need to put an entry to ProcArray for any new XLogRecord.xl_xid it sees in the WAL, and remove the entry at a Commit and Abort record. And clear them all at a shutdown record. For reference, here is how I solved the snapshot problem in my Summer-of-Code project last year, which dealt exactly with executing read-only queries on PITR slaves (But sadly never came out of alpha stage due to both my and Simon's lack of time) The main idea was to invert the meaning of the xid array in the snapshot struct - instead of storing all the xid's between xmin and xmax that are to be considering "in-progress", the array contained all the xid's > xmin that are to be considered "completed". The current read-only snapshot (which "current" meaning the corresponding state on the master at the time the last replayed wal record was generated) was maintained in shared memory. It' xmin field was continually updated with the (newly added) XLogRecord.xl_xmin field, which contained the xid of the oldest running query on the master, with a pruning step after each ReadOnlySnapshot.xmin update to remove all entries < xmin from the xid array. If a commit was seen for an xid, that xid was added to the ReadOnlySnapshot.xid array. The advantage of this concept is that it handles snapshotting on the slave without too much additional work for the master (The only change is the addition of the xl_xmin field to XLogRecord). It especially removes that need to track ShmemVariableCache->nextXid. The downside is that the size of the read-only snapshot is theoretically unbounded, which poses a bit of a problem if it's supposed to live inside shared memory... regards, Florian Pflug smime.p7s Description: S/MIME Cryptographic Signature
Re: [HACKERS] Noisy CVS updates
On Fri, Sep 12, 2008 at 07:27:55PM -0500, Decibel! wrote: > On Sep 5, 2008, at 9:06 AM, D'Arcy J.M. Cain wrote: >> ...etc. Would it be OK if I went in and added .cvsignore files to >> keep the noise level down? I guess I could have my daily script >> filter them out but there may be times when there really is an >> unexpected file and I want to follow up on it. > > +1. It might be possible to get screwed by not doing a distclean, > but the build time savings seems worth it (first thing I do if I get > a build error is make clean/distclean). maintainer-clean is even better as far as removing build errors, and I've never noticed any time difference between it and clean or distclean. Cheers, David. -- David Fetter <[EMAIL PROTECTED]> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] 8.3.3 compiler warnings with gcc 4.3
I'm getting these on Fedora-9: tqual.c:115: warning: inlining failed in call to ‘SetHintBits’: call is unlikely and code size would grow tqual.c:377: warning: called from here tuplesort.c: In function ‘comparetup_index’: tuplesort.c:2423: warning: inlining failed in call to ‘myFunctionCall2’: --param large-stack-frame-growth limit reached tuplesort.c:2474: warning: called from here All these appear a few more times during compilation. I'm not sure we fixed these in 8.3 branch, and wanted to post it -- Devrim GÜNDÜZ, RHCE devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org signature.asc Description: This is a digitally signed message part
Re: [HACKERS] 8.3.3 compiler warnings with gcc 4.3
Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= <[EMAIL PROTECTED]> writes: > I'm getting these on Fedora-9: > tqual.c:115: warning: inlining failed in call to âSetHintBitsâ: call is They're just cosmetic. We don't generally worry about fixing cosmetic warnings in back branches. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposed patch: make SQL interval-literal syntax work per spec
Tom Lane wrote: Ron Mayer <[EMAIL PROTECTED]> writes: interval ... "sql_standard"..."iso_8601"... "backward_compatible" ...depends... on ... DateStyle... ...How about decoupling interval_out's behavior from DateStyle altogether, and instead providing values of IntervalStyle that match all the previous behaviors? Great. That seems much more sane. Any desired names for the existing interval styles? Currently we output intervals in these two styles: '1 year 2 mons 3 days 04:05:06' when the DateStyle is iso. and '@ 1 year 2 mons 3 days 4 hours 5 mins 6 secs' when the DateStyle is sql or postgres, etc. I'm not quite sure where those styles came from so don't know what good names for them might be. Should those ECPG functions be made identical ... ... The palloc and elog dependencies seem to be the hard part. Interesting. So EncodeDateTime and EncodeInterval, guessing 400 or so lines, seem sharable since at first glance they either already do or easily could make their callers deal with all allocation and logging. Agreed that it's a independent patch that I'll try separately. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
