Re: [HACKERS] Improving N-Distinct estimation by ANALYZE

2006-01-16 Thread Manfred Koizar
On Fri, 13 Jan 2006 19:18:29 +, Simon Riggs <[EMAIL PROTECTED]> wrote: >I enclose a patch for checking out block sampling. Can't comment on the merits of block sampling and your implementation thereof. Just some nitpicking: |! * Row Sampling: As of May 2004, we use the Vitter algorithm to c

Re: [HACKERS] Unsplitting btree index leaf pages

2005-12-24 Thread Manfred Koizar
On Thu, 22 Dec 2005 10:40:24 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >If you move items >from one page to the other in the opposite direction from the way the >scan is going, then it will miss those items. AFAIU the (PG implementaion of the) L&Y method is designed to make scans immune against p

Re: [HACKERS] Re: Which qsort is used

2005-12-22 Thread Manfred Koizar
On Thu, 22 Dec 2005 08:01:00 +0100, Martijn van Oosterhout wrote: >But where are you including the cost to check how many cells are >already sorted? That would be O(H), right? Yes. I didn't mention it, because H < N. > This is where we come back >to the issue that comparisons in PostgreSQL are

Re: [HACKERS] Re: Which qsort is used

2005-12-21 Thread Manfred Koizar
On Sat, 17 Dec 2005 00:03:25 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >I've still got a problem with these checks; I think they are a net >waste of cycles on average. [...] > and when they fail, those cycles are entirely wasted; >you have not advanced the state of the sort at all. How can we ma

Re: [HACKERS] Shared locking in slru.c

2005-12-02 Thread Manfred Koizar
On Wed, 30 Nov 2005 13:53:13 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >The way the attached patch attacks this is for the shared-lock access >case to simply set the page's LRU counter to zero, without bumping up >the LRU counters of the other pages as the normal adjustment would do. If you still

Re: [HACKERS] Alternative variable length structure

2005-09-09 Thread Manfred Koizar
On Thu, 08 Sep 2005 18:02:44 +0900, ITAGAKI Takahiro <[EMAIL PROTECTED]> wrote: + * The length of varlena2 is encoded as follows: + * + * | First| Trailing | Total | Max | + * | byte | bytes| bits | length | + * +--+--+---+-+ + * | 0*** |

Re: [HACKERS] Remove xmin and cmin from frozen tuples

2005-09-03 Thread Manfred Koizar
On Fri, 2 Sep 2005 20:41:48 -0400 (EDT), Bruce Momjian wrote: >> Once I had a patch based on 7.4 that stored cmin and cmax in >> backend-local memory. >Interesting idea, but how would you record the cmin/xmin values without >requiring unlimited memory? That's exactly the reason for not sending i

Re: [HACKERS] Remove xmin and cmin from frozen tuples

2005-09-02 Thread Manfred Koizar
On Fri, 2 Sep 2005 15:51:15 -0400 (EDT), Bruce Momjian wrote: > * Merge xmin/xmax/cmin/cmax back into three header fields And don't forget xvac, please. > Before subtransactions, there used to be only three fields needed to > store these four values. ... five value

Re: [HACKERS] Must be owner to truncate?

2005-08-24 Thread Manfred Koizar
On Wed, 24 Aug 2005 07:01:00 +0200, Andreas Seltenreich <[EMAIL PROTECTED]> wrote: >However, a question arose quickly: According to the standard, revoking >INSERT, UPDATE and DELETE after GRANT ALL PRIVILEGES would leave the >relation read-only, but with the TRUNCATE privilege lying around, this >w

Re: [HACKERS] Cost of XLogInsert CRC calculations

2005-05-31 Thread Manfred Koizar
On Tue, 31 May 2005 12:07:53 +0100, "Mark Cave-Ayland" <[EMAIL PROTECTED]> wrote: >Perhaps Manfred can tell us the generator >polynomial that was used to create the lookup tables? 32 26 23 22 16 12 11 10 8 7 5 4 2 1 X + X + X + X + X + X + X + X + X + X + X + X +

Re: [HACKERS] WAL replay failure after file truncation(?)

2005-05-26 Thread Manfred Koizar
On Wed, 25 May 2005 18:19:19 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >> but it keeps a list (hash table, file, whatever) of those blocks. >> [...] Is it sufficient to >> remember just the relation and the block number or do we need the >> contents a well? > >We don't *have* the contents ... tha

Re: [HACKERS] Cost of XLogInsert CRC calculations

2005-05-25 Thread Manfred Koizar
On Wed, 18 May 2005 13:50:22 +0200, I wrote: >The most important figure is, that at MaxSpeed (/O2) 2x32 is almost >twice as fast as CRC32 while only being marginally slower than CRC32. ^ Silly typo! That should have been: The most important figure is, that at MaxSpeed (/O2) 2

Re: [HACKERS] WAL replay failure after file truncation(?)

2005-05-25 Thread Manfred Koizar
On Wed, 25 May 2005 11:02:11 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >Plan B is for WAL replay to always be willing to extend the file to >whatever record number is mentioned in the log, even though this >may require inventing the contents of empty pages; we trust that their >contents won't matt

Re: [HACKERS] Bitmap scan cost model (was Re: bitmap scans, btree scans, and tid order)

2005-05-18 Thread Manfred Koizar
On Tue, 17 May 2005 22:12:17 -0700, "Jeffrey W. Baker" <[EMAIL PROTECTED]> wrote: >Incrementing random_page_cost from 4 (the default) to 5 causes the >planner to make a better decision. We have such a low default random_page_cost primarily to mask other problems in the optimizer, two of which are

Re: [HACKERS] Cost of XLogInsert CRC calculations

2005-05-18 Thread Manfred Koizar
On Wed, 18 May 2005 01:12:26 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >> Wait, par for 32-bit CRCs? Or for 64-bit CRCs calculated using 32-bit ints? > >Right, the latter. We haven't actually tried to measure the cost of >plain 32bit CRCs... although I seem to recall that when we originally >deci

Re: [HACKERS] Learning curves and such

2005-05-17 Thread Manfred Koizar
On Tue, 17 May 2005 14:29:49 -0700, Josh Berkus wrote: > You're not going to win over many people on *this* list with marketing >arguments. Yeah, that's the problem with *my* learning curve ... Servus Manfred ---(end of broadcast)--- TIP 6: Have

Re: Learning curves and such (was Re: [HACKERS] pgFoundry)

2005-05-17 Thread Manfred Koizar
On Tue, 17 May 2005 14:45:00 -0300 (ADT), "Marc G. Fournier" <[EMAIL PROTECTED]> wrote: >Also, how many 'bugs' have we seen go through the lists that someone >hasn't jump'd on and fixed in a couple of days? Just imagine our marketing crew being able to say: "According to our great bug tracking sy

Re: [HACKERS] pgFoundry

2005-05-17 Thread Manfred Koizar
On Mon, 16 May 2005 20:54:15 -0400 (EDT), Bruce Momjian wrote: >I have modifed the TODO HTML so the completed items are in italics. Isn't it a bit misleading to have those items on the TODO list at all? Shouldn't there be a separate list: DONE for the next release? Servus Manfred

Re: [HACKERS] BTW, if anyone wants to work on it...

2005-05-17 Thread Manfred Koizar
On Tue, 03 May 2005 02:45:09 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: > I'm starting to think >it'd be worth setting up a mechanism to handle such changes >automatically. I've been using this skeleton for quite some time now. Magnus' psql ... | while read D might be more robust than my

Re: [HACKERS] Cost of XLogInsert CRC calculations

2005-05-17 Thread Manfred Koizar
On Mon, 16 May 2005 12:35:35 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >Anyone want to try it with non-gcc compilers? MS VC++ 6.0 with various predefined optimizer settings 2x3264 Default (without any /O) 0.828125 0.906250 MinSize (contains /O1)

Re: [HACKERS] Best way to scan on-disk bitmaps

2005-05-15 Thread Manfred Koizar
On Thu, 12 May 2005 17:40:06 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >the planner believes that only >consecutive columns in the index are usable --- that is, if you have >quals for a and c but not for b, it will think that the condition for c >isn't usable with the index. This is true for btre

Re: [HACKERS] Views, views, views! (long)

2005-05-15 Thread Manfred Koizar
On Wed, 4 May 2005 21:37:40 -0700, Josh Berkus wrote: >As stated above, these system views, once incorporated into a pg distribution, >are likely to be with us *forever*. I don't think that this is doable. :-( You might want to put the system views into a version specific schema, say pg_views8

Re: [HACKERS] We are not following the spec for HAVING without GROUP

2005-03-14 Thread Manfred Koizar
On Fri, 11 Mar 2005 10:37:13 +1300, Mark Kirkwood <[EMAIL PROTECTED]> wrote: >Firebird 1.5.1 FreeBSD 5.3 >[correct results] Interbase 6.0: SQL> create table tab (col integer); SQL> select 1 from tab having 1=0; SQL> select 1 from tab having 1=1; 0<---:-)

Re: [HACKERS] Group-count estimation statistics

2005-02-01 Thread Manfred Koizar
On Mon, 31 Jan 2005 14:40:08 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >Manfred Koizar <[EMAIL PROTECTED]> writes: >> That's not what I meant. I tried to say that if we have a GROUP BY >> several columns and one of these columns alone has more than N/10 >>

Re: [HACKERS] Group-count estimation statistics

2005-01-31 Thread Manfred Koizar
On Mon, 31 Jan 2005 11:20:31 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >Already done that way. >if (relvarcount > 1) >clamp *= 0.1; That's not what I meant. I tried to say that if we have a GROUP BY several columns and one of these columns alone has more than N/10 dis

Re: [HACKERS] Refactoring

2005-01-31 Thread Manfred Koizar
On Wed, 19 Jan 2005 18:57:48 +0100, I wrote: > My first vacuum.c >refactoring patch, rev 1.281 2004-06-08, added these comments in >repair_frag(): > >/* > * VACUUM FULL has an exclusive lock on the relation. So > * normally no other transaction can have pending INSERTs or > * DELETEs in this rela

Re: [HACKERS] Group-count estimation statistics

2005-01-31 Thread Manfred Koizar
On Fri, 28 Jan 2005 10:53:33 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: > we should consider >something like "clamp to size of table / 10" instead. ... unless a *single* grouping column is estimated to have more than N/10 distinct values, which should be easy to check. Servus Manfred --

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-24 Thread Manfred Koizar
On Mon, 24 Jan 2005 08:28:09 -0700, "Jonah H. Harris" <[EMAIL PROTECTED]> wrote: >UPDATE pg_user_table_counts >SET rowcount = rowcount + 1 >WHERE schemaname = this_schemaname >AND tablename = TG_RELNAME; This might work for small sing

Re: [HACKERS] Autotuning Group Commit

2005-01-24 Thread Manfred Koizar
On Fri, 21 Jan 2005 23:52:51 +, Simon Riggs <[EMAIL PROTECTED]> wrote: >Currently, we have group commit functionality via GUC parameters > commit_delay >andcommit_siblings And since 7.3 we have ganged WAL writes (c.f. the thread starting at http://archives.postgresql.org/pgsql-hacker

Re: [HACKERS] ARC patent

2005-01-21 Thread Manfred Koizar
On Fri, 21 Jan 2005 02:31:40 +0200, Hannu Krosing <[EMAIL PROTECTED]> wrote: >2) Another simple, but nondeterministic, hack would be using randomness, >i.e. > > 2.1) select a random buffer in LR side half (or 30% or 60%) of > for replacement. > > 2.2) dont last accessed pages to top of L

Re: [HACKERS] Refactoring

2005-01-19 Thread Manfred Koizar
[Sorry, Neil, for abusing your thread. Moving this discussion back to where it belongs.] On Tue, 18 Jan 2005 13:17:17 -0300, Alvaro Herrera <[EMAIL PROTECTED]> wrote: >Hmm. I think this is a good idea on principle, but what happens in case >a previous vacuum was interrupted? Is there a possibil

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-16 Thread Manfred Koizar
On Thu, 13 Jan 2005 00:39:56 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >A would-be deleter of a tuple would have to go and clear the "known >good" bits on all the tuple's index entries before it could commit. >This would bring the tuple back into the "uncertain status" condition >where backends wo

Re: [HACKERS] Shared row locking

2004-12-30 Thread Manfred Koizar
On Thu, 30 Dec 2004 13:36:53 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >Certainly not; indexes depend on locks, not vice versa. You'd not be >able to do that without introducing an infinite recursion into the >system design. Wouldn't you have to face the same sort of problems if you spill part o

Re: [HACKERS] Shared row locking

2004-12-30 Thread Manfred Koizar
On Wed, 29 Dec 2004 19:57:15 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >Manfred Koizar <[EMAIL PROTECTED]> writes: >> I don't see too much of a difference between #1 (an on-disk structure >> buffered in shared memory) and #2 (a shared memory structure spilling &

Re: [HACKERS] Bgwriter behavior

2004-12-29 Thread Manfred Koizar
[I know I'm late and this has already been discussed by Richrad, Tom, et al., but ...] On Tue, 21 Dec 2004 16:17:17 -0600, "Jim C. Nasby" <[EMAIL PROTECTED]> wrote: >look at where the last page you wrote out has ended up in the LRU list >since you last ran, and start scanning from there (by defini

Re: [HACKERS] Shared row locking

2004-12-29 Thread Manfred Koizar
On Mon, 20 Dec 2004 21:44:01 +0100, <[EMAIL PROTECTED]> wrote: >Tom Lane <[EMAIL PROTECTED]> wrote on 20.12.2004, 19:34:21: >> #1 could have a pretty serious performance impact, too. For small >> numbers of FOR UPDATE locks (too few to force spill to disk) I would >> expect #2 to substantially bea

Re: [HACKERS] Shared row locking

2004-12-29 Thread Manfred Koizar
On Thu, 16 Dec 2004 21:54:14 -0300, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > Else, it will have to wait, using XactLockTableWait, for >the first transaction in the array that is still running. We can be >sure that no one will try to share-lock the tuple while we check the >btree because we hol

Re: [HACKERS] Updateable Views?

2004-08-09 Thread Manfred Koizar
On Sat, 07 Aug 2004 10:24:34 -0400, Jan Wieck <[EMAIL PROTECTED]> wrote: >I have not heard of "updatable subselects" yet. http://asktom.oracle.com/pls/ask/f?p=4950:8:6693556430011788783::NO::F4950_P8_DISPLAYID,F4950_P8_CRITERIA:273215737113, | Here we update a join. [...] | [EMAIL PROTECTED]> upd

Re: [HACKERS] Trapping QUERY_CANCELED: yes, no, maybe?

2004-08-06 Thread Manfred Koizar
On Fri, 06 Aug 2004 18:55:49 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >You think there's a serious risk of failure there ;-) ? Not on my hardware... Servus Manfred ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an

Re: [HACKERS] Trapping QUERY_CANCELED: yes, no, maybe?

2004-08-06 Thread Manfred Koizar
On Sat, 31 Jul 2004 21:24:33 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >Exactly. There's a proof-of-concept test at the bottom of >regress/sql/plpgsql.sql, wherein a function gets control back >from a query that would have run for an unreasonably long time. referring to | -- we assume this w

Re: [HACKERS] Quick question regarding tablespaces

2004-08-06 Thread Manfred Koizar
On Thu, 1 Jul 2004 22:55:56 -0400, Mike Rylander <[EMAIL PROTECTED]> wrote: >I was thinking of purely tablespace-based random_page_cost, as that variable >is tied to the access time of a particular filesystem. Strictly speaking we'd also need tablespace-based sequential_page_cost. Servus Manfre

Re: [HACKERS] More vacuum.c refactoring

2004-08-06 Thread Manfred Koizar
[Sorry for the late reply. I'm still struggling to catch up after vacation ...] On Fri, 9 Jul 2004 21:29:52 -0400 (EDT), Bruce Momjian <[EMAIL PROTECTED]> wrote: > >Where are we on this, 2x. :-) Here: >> Tom Lane wrote: >> > Will study these comments later, but it's too late at night here... S

[HACKERS] dbt2-pgsql on OSDL

2004-07-06 Thread Manfred Koizar
Mark, I've tried to run some performance tests on your Scalable Test Platform but the tests failed at the build step. I guess the problem is near line 282 of http://khack.osdl.org/stp/294734/logs/run-log.txt | + wget -nv -t 0 --waitretry=60 -r http://stp/data/dbt-2/postgresql-7.4.tar.gz | 09:45:

Re: [HACKERS] Another unpleasant surprise using inheritance

2004-06-11 Thread Manfred Koizar
On Fri, 11 Jun 2004 14:11:00 +0200, Darko Prenosil <[EMAIL PROTECTED]> wrote: >I think I found bug related to table inheritance (or at least very weird >behavior). This is well known and there's a todo for it: # Allow inherited tables to inherit index, UNIQUE constraint, and primary key, foreig

Re: [HACKERS] More vacuum.c refactoring

2004-06-10 Thread Manfred Koizar
On Thu, 10 Jun 2004 17:19:22 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >This does not make me comfortable. I understand you, honestly. Do I read between your lines that you didn't review my previous vacuum.c refactoring patch? Please do. It'd make *me* more comfortable. > You *think* that tw

[HACKERS] More vacuum.c refactoring

2004-06-10 Thread Manfred Koizar
Near the end of repair_frag() in vacuum.c -- under the comment /* clean moved tuples from last page in Nvacpagelist list */ -- there is code that marks itemids as unused. Itemids affected are those referring to tuples that have been moved off the last page. This code is very similar to vacuum_pag

Re: [HACKERS] SELECT * FROM LIMIT 1; is really slow

2004-06-01 Thread Manfred Koizar
On Fri, 28 May 2004 21:59:53 -0400, Alvaro Herrera <[EMAIL PROTECTED]> wrote: >So the assumption was that when we see that this has >happenned, the Cmin is no longer important (== every future command can >already see the tuple) If it was that simple, we wouldn't have had to invent the CMIN_IS_CMA

Re: [HACKERS] SELECT * FROM LIMIT 1; is really slow

2004-06-01 Thread Manfred Koizar
On Fri, 28 May 2004 14:47:01 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >If putting back xmax is the price we must pay for nested transactions, >then we *will* pay that price. Maybe not in this release, but it will >inevitably happen. "we" = every Postgres user, even those that do not use subtran

Re: [HACKERS] SELECT * FROM LIMIT 1; is really slow

2004-05-28 Thread Manfred Koizar
On Thu, 27 May 2004 16:50:24 -0400, Alvaro Herrera <[EMAIL PROTECTED]> wrote: >Now you are on the subject, can I ask you to take a peek at what I did >regarding tuple headers? I did read your patch, but I didn't understand it. :-( >At first I thought I'd have to add back Xmax as a field on its ow

Re: [HACKERS] SELECT * FROM LIMIT 1; is really slow

2004-05-27 Thread Manfred Koizar
On Thu, 27 May 2004 14:23:07 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >> and when y is updated the new version will be stored in a lower block > >Oh? What makes you think that? I see no guarantee of it. You're right, I see only a tendency, because the majority of free space is before the last

Re: [HACKERS] SELECT * FROM LIMIT 1; is really slow

2004-05-27 Thread Manfred Koizar
On Wed, 26 May 2004 18:17:55 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >(Some days I think we should dump >VACUUM FULL, because it's optimized for a case that's no longer very >interesting...) So we still have to stick with VACUUM FULL for some time, right? The next set of compatibility breakers

Re: [HACKERS] zero-column table behavior

2004-05-24 Thread Manfred Koizar
[resending...] On Sat, 22 May 2004 20:28:43 -0400, Neil Conway <[EMAIL PROTECTED]> wrote: >-- Why is there a blank line before the "--" that indicates the >-- end of the result set? "--" separates the header line from the *start* of the result set. The empty line is the header line, containing z

Re: [HACKERS] New horology failure

2004-05-24 Thread Manfred Koizar
[resending...] On Sun, 23 May 2004 11:38:51 +0800, Christopher Kings-Lynne <[EMAIL PROTECTED]> wrote: >I get this since Tom's commit. >--- ./results/horology.out Sun May 23 11:39:49 2004 >*** >*** 1787,1796 >! | Sat Sep 22 18:19:20 2001 PDT | @ 34 years

Re: [HACKERS] Multiple Xids in PGPROC?

2004-05-05 Thread Manfred Koizar
On Tue, 04 May 2004 23:21:07 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >I thought we had devised a solution that did not require expansible >shared memory for this. Bruce, Manfred, do you recall how that went? AFAIR we did not discuss TransactionIdIsInProgress() specifically. Currently this func

Re: [HACKERS] Number of pages in a random sample

2004-04-29 Thread Manfred Koizar
On Mon, 26 Apr 2004 08:08:16 -0700, Sailesh Krishnamurthy <[EMAIL PROTECTED]> wrote: > "A Bi-Level Bernoulli Scheme for Database Sampling" > Peter Haas, Christian Koenig (SIGMOD 2004) Does this apply to our problem? AFAIK with Bernoulli sampling you don't know the sample size in advance. Anyway

Re: [HACKERS] btbulkdelete

2004-04-28 Thread Manfred Koizar
On Wed, 28 Apr 2004 00:08:48 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >> Is there a special reason for scanning the leaf pages in *logical* >> order, i.e. by following the opaque->btpo_next links? > >Yes. [..] interlocking between indexscans and deletions. Thanks for refreshing my memory. This

Re: [HACKERS] btbulkdelete

2004-04-26 Thread Manfred Koizar
On Mon, 26 Apr 2004 14:29:58 +0100, Simon Riggs <[EMAIL PROTECTED]> wrote: >> Now that FSM >> covers free btree index pages this access pattern might be highly >> nonsequential. > >I had considered implementing a mode where the index doesn't keep trying >to reuse space that was freed by earlier d

[HACKERS] btbulkdelete

2004-04-25 Thread Manfred Koizar
On -performance we have been discussing a configuration where a bulk delete run takes almost a day (and this is not due to crappy hardware or apparent misconfiguration). Unless I misinterpreted the numbers, btbulkdelete() processes 85 index pages per second, while lazy vacuum is able to clean up 6

[HACKERS] Number of pages in a random sample (was: query slows down with more accurate stats)

2004-04-25 Thread Manfred Koizar
On Mon, 19 Apr 2004 12:00:10 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >A possible compromise is to limit the number of pages sampled to >something a bit larger than n, perhaps 2n or 3n. I don't have a feeling >for the shape of the different-pages probability function; would this >make a signific

[HACKERS] Tuple sampling

2004-04-19 Thread Manfred Koizar
The proposed new sampling method (http://archives.postgresql.org/pgsql-hackers/2004-04/msg00036.php and http://archives.postgresql.org/pgsql-patches/2004-04/msg00045.php) basically incorporates two independant changes: (1) Two-stage sampling: Stage one collects a random sample of pages, stage two

Re: [HACKERS] [GENERAL] Large DB

2004-04-02 Thread Manfred Koizar
On Fri, 02 Apr 2004 19:57:47 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >If you like I can send you the Vitter paper off-list (I have a PDF of >it). The comments in the code are not really intended to teach someone >what it's good for ... Yes, please. [Would have sent this off-list. But I'm bla

Re: [HACKERS] [GENERAL] Large DB

2004-04-02 Thread Manfred Koizar
On Fri, 02 Apr 2004 18:06:12 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >You should not need to use the Vitter algorithm for the block-level >selection, since you can know the number of blocks in the table in >advance. You can just use the traditional method of choosing each block >or not with pro

Re: [HACKERS] [GENERAL] Large DB

2004-04-02 Thread Manfred Koizar
On Fri, 02 Apr 2004 14:48:13 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >Manfred Koizar <[EMAIL PROTECTED]> writes: >> What I have in mind is a kind of "Double Vitter" algorithm. [...] >> random sample of sample_size block numbers, and then to sample t

Re: [HACKERS] [GENERAL] Large DB

2004-04-02 Thread Manfred Koizar
[time to move this to -hackers] On Fri, 02 Apr 2004 11:16:21 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >Manfred Koizar <[EMAIL PROTECTED]> writes: >> The first step, however, (acquire_sample_rows() in analyze.c) has to >> read more rows than finally end up in the s

Re: [HACKERS] PostgreSQL block size vs. LVM2 stripe width

2004-03-29 Thread Manfred Koizar
On Mon, 29 Mar 2004 08:50:42 -0800 (PST), [EMAIL PROTECTED] wrote: >In this case, I've only done 1 per each combination. I've found the >results for this test to be reproduceable. Pardon? >>>Linux-2.6.3, LVM2 Stripe Width >>>BLCKSZ >>>(going down)16 KB 32 KB 64 KB

Re: [HACKERS] PostgreSQL block size vs. LVM2 stripe width

2004-03-27 Thread Manfred Koizar
Mark, how often did you run your tests? Are the results reproduceable? On Fri, 26 Mar 2004 14:00:01 -0800 (PST), [EMAIL PROTECTED] wrote: >Linux-2.6.3, LVM2 Stripe Width >(going across) >PostgreSQL >BLCKSZ >(going down)16 KB 32 KB 64 KB 1

Re: [HACKERS] [PATCHES] log_line_info

2004-03-10 Thread Manfred Koizar
On Tue, 09 Mar 2004 10:02:14 -0500, Andrew Dunstan <[EMAIL PROTECTED]> wrote: >>>After this is applied (fingers crossed) and everyone is happy, I will >>>submit a patch to remove log_timestamp, log_pid and (if we are agreed on >>>it) log_source_port. >Is there agreement on removing these 3 confi

Re: [HACKERS] Summary of Changes since last release (7.4.1)

2004-02-19 Thread Manfred Koizar
Simon, On Thu, 19 Feb 2004 00:05:15 -, "Simon Riggs" <[EMAIL PROTECTED]> wrote: >POSTGRESQL: Summary of Changes since last release (7.4.1) >-- >18 Feb 2004 this is getting long over time. If you plan to post it once a week, flagging ite

Re: [HACKERS] [GENERAL] Transaction Question

2003-12-11 Thread Manfred Koizar
On Sat, 6 Dec 2003 10:43:18 -0500 (EST), Bruce Momjian <[EMAIL PROTECTED]> wrote: >Where are we on nested transactions. Is it something we can get for 7.5? I honestly don't know. I've been working on other things lately and have not heard from Alvaro for some time. Servus Manfred

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-12-02 Thread Manfred Koizar
On Mon, 01 Dec 2003 13:32:10 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >Manfred Koizar <[EMAIL PROTECTED]> writes: >> comparetup_index() compares two IndexTuples. The structure >> IndexTupleData consists basically of not much more than an ItemPointer, >> an

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-12-01 Thread Manfred Koizar
On Mon, 1 Dec 2003 00:02:54 -0500 (EST), Bruce Momjian <[EMAIL PROTECTED]> wrote: >Tom Lane wrote: >> Bruce Momjian <[EMAIL PROTECTED]> writes: >> >> And if it doesn't help index >> >> creation speed, at least the resulting index has better correlation. ... which has been shown by the example in

Re: [HACKERS] logical column position

2003-11-20 Thread Manfred Koizar
On Wed, 19 Nov 2003 19:07:23 +0100, Andreas Pflug <[EMAIL PROTECTED]> wrote: >is there >any DB system out there that allows to reshuffle the column ordering? Firebird: ALTER TABLE tname ALTER COLUMN cname POSITION 7; Servus Manfred ---(end of broadcast)-

Re: Defaults for GUC variables (was Re: [HACKERS] pg_ctl reports succes when start fails)

2003-10-28 Thread Manfred Koizar
On Mon, 27 Oct 2003 10:22:32 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >The low-tech solution to this would be to stop listing the default >values as commented-out entries, but just make them ordinary uncommented >entries. Please not. How should we ask a newbie seeking assistance on one of the s

Re: [HACKERS] Cannot dump/restore text value \N

2003-10-08 Thread Manfred Koizar
On Sun, 05 Oct 2003 19:12:50 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >it seems we have to compare the null representation string to the >pre-debackslashing input. Here is a patch that does this and adds a few regression tests. >(This is probably fairly easy to make happen >in CVS tip, but it m

Re: [HACKERS] Cannot dump/restore text value \N

2003-10-05 Thread Manfred Koizar
I have solved my restore problem by editing (the relevant part of) the dump (:%s/^IN^I/^IN ^I/), a one-off solution Anyway, thanks for your investigation. On Sun, 05 Oct 2003 19:12:50 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >it seems we have to compare the null representation string t

[HACKERS] Cannot dump/restore text value \N

2003-10-05 Thread Manfred Koizar
To be clear, this is not about \N as the default external representation for NULL, I'm talking about a string consisting of the two characters backslash and uppercase-N. CREATE TABLE nonu (tx text NOT NULL); INSERT INTO nonu VALUES ('\\N'); SELECT * FROM nonu; COPY nonu TO stdout; This correctly

Re: [HACKERS] ADD FOREIGN KEY

2003-10-01 Thread Manfred Koizar
On Tue, 30 Sep 2003 08:00:07 -0400, Christopher Browne <[EMAIL PROTECTED]> wrote: >I would be pretty "game" for a near-single-user-mode approach that >would turn off some of the usual functionality that we knew we didn't >need because the data source was an already-committed-and-FK-checked >set of

Re: [HACKERS] [PERFORM] osdl-dbt3 run results - puzzled by the execution

2003-09-24 Thread Manfred Koizar
On Fri, 19 Sep 2003 11:35:35 -0700, Jenny Zhang <[EMAIL PROTECTED]> wrote: >I posted more results as you requested: Unfortunately they only confirm what I suspected earlier: >> 2) -> Index Scan using i_ps_suppkey on partsupp >> (cost=0.00..323.16 rows=80 width=34) >>

Re: [HACKERS] Is it a memory leak in PostgreSQL 7.4beta?

2003-09-10 Thread Manfred Koizar
On Wed, 10 Sep 2003 00:18:52 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >I have a suspicion that there is no real leak, but >rather we are seeing some artifact of the way Linux' top(1) reports >memory usage. >From my experience I can confirm that. I have looked a lot at top output when I benchmar

Re: [HACKERS] [SQL] plpgsql doesn't coerce boolean expressions to boolean

2003-09-09 Thread Manfred Koizar
On Mon, 08 Sep 2003 11:40:32 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >4. Use the parser's coerce_to_boolean procedure, so that nonbooleans > will be accepted in exactly the same cases where they'd be accepted > in a boolean-requiring SQL construct (such as CASE). (By default, > none are,

Re: [HACKERS] Index creation takes for ever

2003-09-08 Thread Manfred Koizar
On Mon, 8 Sep 2003 11:31:05 +0200, "Zeugswetter Andreas SB SD" <[EMAIL PROTECTED]> wrote: >> As Tom mentioned, we might not want to keep the tid's in order after the >> index is created because he wants the most recent tid's first, so the >> expired ones migrate to the end. > >But on average this a

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-09-07 Thread Manfred Koizar
On Sun, 07 Sep 2003 12:23:28 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >Maybe so, but it would degrade the performance in the unique-index case >if we do it as the TODO is worded. The patch would only hurt with a unique index, if there are lots of duplicate tuples at CREATE INDEX time. >My own o

Re: [HACKERS] Index creation takes for ever

2003-09-07 Thread Manfred Koizar
On Sun, 7 Sep 2003 11:43:42 -0400 (EDT), Bruce Momjian <[EMAIL PROTECTED]> wrote: >I assume this completes this TODO: > > * Order duplicate index entries by tid for faster heap lookups I don't think so, because the patch does nothing to keep the sort order once the index is initially created

Re: [HACKERS] Index creation takes for ever

2003-09-07 Thread Manfred Koizar
On Mon, 01 Sep 2003 08:46:09 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >[EMAIL PROTECTED] writes: >> it took 69 minutes to finish, 75% of this time was devoted to create 2 >> indexes on varchar(2) with value being 'O', 'N' or null; > >I still say it's either strcoll or qsort's fault. If qsort is

Re: [HACKERS] [GENERAL] Buglist

2003-08-22 Thread Manfred Koizar
On Fri, 22 Aug 2003 12:18:02 -0400, Jan Wieck <[EMAIL PROTECTED]> wrote: >Okay, my proposal would be to have a VACUUM mode where it tells the >buffer manager to only return a page if it is already in memory But how can it know? Yes, we know exactly what we have in PG shared buffers. OTOH we kee

Re: [HACKERS] [GENERAL] Buglist

2003-08-22 Thread Manfred Koizar
On Fri, 22 Aug 2003 10:45:50 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >One big question mark in my mind about these "partial vacuum" >proposals is whether they'd still allow adequate FSM information to be >maintained. If VACUUM isn't looking at most of the pages, there's no >very good way to acq

Re: [HACKERS] [GENERAL] Buglist

2003-08-22 Thread Manfred Koizar
On Fri, 22 Aug 2003 16:27:53 +0530, Shridhar Daithankar <[EMAIL PROTECTED]> wrote: >What does FSM does then? FSM = Free Space Map. VACUUM writes information into the FSM, INSERTs consult the FSM to find pages with free space for new tuples. > I was under impression that FSM stores page >pointer

Re: [HACKERS] Decent VACUUM (was: Buglist)

2003-08-21 Thread Manfred Koizar
On Thu, 21 Aug 2003 17:56:02 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >Conceivably it could be a win, though, >if you could do frequent "vacuum decent"s and only a full-scan vacuum >once in awhile (once a day maybe). That's what I had in mind; similar to the current situation where you can avoid

Re: [HACKERS] Decent VACUUM (was: Buglist)

2003-08-21 Thread Manfred Koizar
[ still brainstorming ... ] On Thu, 21 Aug 2003 17:16:50 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >> Whenever a backend encounters a dead tuple it inserts a reference to >> its page into the RSM. > >This assumes that backends will visit dead tuples with significant >probability. I doubt that as

Re: [HACKERS] [SQL] "SELECT IN" Still Broken in 7.4b

2003-08-21 Thread Manfred Koizar
On Thu, 21 Aug 2003 16:42:20 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >The point is that given > WHERE a = 1 OR b = 1 >you could create a plan that first indexscans on a, then indexscans on >b --- but you mustn't return any tuples in the second scan that you >already returned in the first.

[HACKERS] Decent VACUUM (was: Buglist)

2003-08-21 Thread Manfred Koizar
On Wed, 20 Aug 2003 15:39:26 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >But I think the real point here is that there's no reason to think that >doing tuple deletion on-the-fly in foreground transactions is superior >to doing it in background with a vacuum process. You're taking what >should be n

Re: [GENERAL] [HACKERS] [pgsql-advocacy] Need concrete "Why Postgres not MySQL" bullet list

2003-08-21 Thread Manfred Koizar
On Thu, 21 Aug 2003 15:05:52 +0200, I wrote: >>Just wondering, what other databases has transactable DDLs? > >Firebird. Stop! I withdraw that statement. I must have mis-read some feature list :-( Tests with InterBase 6 showed that you can change metadata within a transaction, but when you ROLLB

Re: [HACKERS] [pgsql-advocacy] Need concrete "Why Postgres not MySQL" bullet list

2003-08-21 Thread Manfred Koizar
On Thu, 21 Aug 2003 14:45:03 +0530, "Shridhar Daithankar" <[EMAIL PROTECTED]> wrote: >Just wondering, what other databases has transactable DDLs? Firebird. Servus Manfred ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choos

[HACKERS] Again on index correlation

2003-08-20 Thread Manfred Koizar
Recent discussion of index cost estimation ("[HACKERS] Correlation in cost_index()" ca. two weeks ago) has lead to the conclusion that the column correlation calculated by VACUUM does not always help when we want to find out how well index access order corresponds to physical tuple position. Most

Re: [HACKERS] Correlation in cost_index()

2003-08-20 Thread Manfred Koizar
On Fri, 8 Aug 2003 16:53:48 -0700, Sean Chittenden <[EMAIL PROTECTED]> wrote: >the problem with your patch was >that it picked an index less often than the current code when there >was low correlation. Maybe bit rot? What version did you apply the patch against? Here is a new version for Postgre

Re: [HACKERS] Correlation in cost_index()

2003-08-14 Thread Manfred Koizar
On Fri, 8 Aug 2003 11:06:56 -0700, Sean Chittenden <[EMAIL PROTECTED]> wrote: >[...] it'd seem as though an avg depth of >nodes in index * tuples_fetched * (random_io_cost * indexCorrelation) >would be closer than where we are now... Index depth does not belong here because we walk down the index

Re: [HACKERS] Correlation in cost_index()

2003-08-11 Thread Manfred Koizar
On Fri, 08 Aug 2003 18:25:41 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: > Two examples: [...] One more example: X Y A A a B A C b A B B b C C A c B C C Cor

Re: [HACKERS] Correlation in cost_index()

2003-08-10 Thread Manfred Koizar
On Fri, 8 Aug 2003 15:10:06 -0700, Sean Chittenden <[EMAIL PROTECTED]> wrote: >> Yes, we knew that already. Oliver had suggested simply dropping the >> division by nKeys, thus pretending that the first-column correlation >> is close enough. That seems to me to be going too far in the other >> dir

Re: [HACKERS] Correlation in cost_index()

2003-08-10 Thread Manfred Koizar
On Thu, 7 Aug 2003 13:44:19 -0700, Sean Chittenden <[EMAIL PROTECTED]> wrote: >> The indexCorrelation^2 algorithm was only a quick hack with no theory >> behind it :-(. I've wanted to find some better method to put in there, >> but have not had any time to research the problem. > >Could we "quick

Re: [HACKERS] Correlation in cost_index()

2003-08-08 Thread Manfred Koizar
On Fri, 8 Aug 2003 16:53:48 -0700, Sean Chittenden <[EMAIL PROTECTED]> wrote: ># SHOW effective_cache_size ; > effective_cache_size >-- > 4456 >(1 row) Only 35 MB? Are you testing on such a small machine? >The stats are attached && bzip2 compressed. Nothing was attached. Di

  1   2   >