[HACKERS] CREATE INDEX CONCURRENTLY?

2014-10-31 Thread Mark Woodward
I have not kept up with PostgreSQL changes and have just been using it. A co-worker recently told me that you need to word "CONCURRENTLY" in "CREATE INDEX" to avoid table locking. I called BS on this because to my knowledge PostgreSQL does not lock tables. I referenced this page in the documentatio

[HACKERS] SSL and USER_CERT_FILE

2008-05-15 Thread Mark Woodward
I am using PostgreSQL's SSL support and the conventions for the key and certifications don't make sense from the client perspective. Especially under Windows. I am proposing a few simple changes: Adding two API void PQsetSSLUserCertFileName(char *filename) { user_crt_filename = strdup(filenam

Re: [HACKERS] Permanent settings

2008-02-21 Thread Mark Woodward
> > > Mark Woodward wrote: >> I have been looking at this thread for a bit and want to interject an >> idea. >> >> A couple years ago, I offered a patch to the GUC system that added a >> number of abilities, two left out were: >> >> (1) Spec

Re: [HACKERS] Permanent settings

2008-02-21 Thread Mark Woodward
I have been looking at this thread for a bit and want to interject an idea. A couple years ago, I offered a patch to the GUC system that added a number of abilities, two left out were: (1) Specify a configuration file on the command line. (2) Allow the inclusion of a configuration file from withi

Re: [HACKERS] Syntax bug? Group by?

2006-10-17 Thread Mark Woodward
> On Oct 17, 2006, at 15:19, Peter Eisentraut wrote: > >> Mark Woodward wrote: >>> Shouldn't this work? >>> >>> select ycis_id, min(tindex), avg(tindex) from y where ycis_id = 15; >>> >>> ERROR: column "y.ycis_id" mu

Re: [HACKERS] Syntax bug? Group by?

2006-10-17 Thread Mark Woodward
> Mark Woodward wrote: >> Shouldn't this work? >> >> select ycis_id, min(tindex), avg(tindex) from y where ycis_id = 15; >> >> ERROR: column "y.ycis_id" must appear in the GROUP BY clause or be >> used in an aggregate function > > This

Re: [HACKERS] Syntax bug? Group by?

2006-10-17 Thread Mark Woodward
> On Tue, Oct 17, 2006 at 02:41:25PM -0400, Mark Woodward wrote: > >> The output column "ycis_id" is unabiguously a single value with regards >> to >> the query. Shouldn't PostgreSQL "know" this? AFAIR, I think I've used >> this >>

Re: [HACKERS] Syntax bug? Group by?

2006-10-17 Thread Mark Woodward
> Mark Woodward wrote: >>>> select ycis_id, min(tindex), avg(tindex) from y where ycis_id = 15; >>>> >> >> I still assert that there will always only be one row to this query. >> This >> is an aggregate query, so all the rows with ycis_i

Re: [HACKERS] Syntax bug? Group by?

2006-10-17 Thread Mark Woodward
> Mark Woodward wrote: >>> Stephen Frost wrote: >>> >>> >>>> select ycis_id, min(tindex), avg(tindex) from y where ycis_id = 15; >>>> >>> But back to the query the issue comes in that the ycis_id value is >>> included with t

Re: [HACKERS] Syntax bug? Group by?

2006-10-17 Thread Mark Woodward
> Hi, Mark, > > Mark Woodward wrote: >> Shouldn't this work? >> >> select ycis_id, min(tindex), avg(tindex) from y where ycis_id = 15; >> >> ERROR: column "y.ycis_id" must appear in the GROUP BY clause or be used >> in an aggregate funct

Re: [HACKERS] Syntax bug? Group by?

2006-10-17 Thread Mark Woodward
> Hi, Mark, > > Mark Woodward wrote: >>> Stephen Frost wrote: >>> >>>> select ycis_id, min(tindex), avg(tindex) from y where ycis_id = 15; >>> But back to the query the issue comes in that the ycis_id value is >>> included with the retur

Re: [HACKERS] Syntax bug? Group by?

2006-10-17 Thread Mark Woodward
> Stephen Frost wrote: > >> select ycis_id, min(tindex), avg(tindex) from y where ycis_id = 15; > > But back to the query the issue comes in that the ycis_id value is > included with the return values requested (a single row value with > aggregate values that isn't grouped) - if ycis_id is not uniq

[HACKERS] Syntax bug? Group by?

2006-10-17 Thread Mark Woodward
Shouldn't this work? select ycis_id, min(tindex), avg(tindex) from y where ycis_id = 15; ERROR: column "y.ycis_id" must appear in the GROUP BY clause or be used in an aggregate function If I am asking for a specific column value, should I, technically speaking, need to group by that column? -

Re: [HACKERS] Hints WAS: Index Tuning Features

2006-10-12 Thread Mark Woodward
> Clinging to sanity, [EMAIL PROTECTED] ("Mark Woodward") mumbled into > her beard: >> What is the point of writing a proposal if there is a threat of >> "will be rejected" if one of the people who would do the rejection >> doesn't at least outline

Re: [HACKERS] Hints WAS: Index Tuning Features

2006-10-11 Thread Mark Woodward
>>> >>> Since you're the one who wants hints, that's kind of up to you to >>> define. >>> Write a specification and make a proposal. >>> >> >> What is the point of writing a proposal if there is a threat of "will be >> rejected" if one of the people who would do the rejection doesn't at >> least >>

Re: [HACKERS] Hints WAS: Index Tuning Features

2006-10-11 Thread Mark Woodward
> Mark, > > First off, I'm going to request that you (and other people) stop hijacking > Simon's thread on hypothetical indexes. Hijacking threads is an > effective way to get your ideas rejected out of hand, just because the > people whose thread you hijacked are angry with you. > > So please ob

Re: [HACKERS] Index Tuning Features

2006-10-11 Thread Mark Woodward
> "Mark Woodward" <[EMAIL PROTECTED]> writes: >> I would say that a "simpler" planner with better hints >> will always be capable of creating a better query plan. > > This is demonstrably false: all you need is an out-of-date hint, and > you can

Re: [HACKERS] Index Tuning Features

2006-10-11 Thread Mark Woodward
> > "Mark Woodward" <[EMAIL PROTECTED]> writes: > >> The analyzer, at least the last time I checked, does not recognize these >> relationships. > > The analyzer is imperfect but arguing from any particular imperfection is > weak > because someone w

Re: [HACKERS] Index Tuning Features

2006-10-11 Thread Mark Woodward
> On 10/10/06, Mark Woodward <[EMAIL PROTECTED]> wrote: >> I think the idea of "virtual indexes" is pretty interesting, but >> ultimately a lesser solution to a more fundimental issue, and that would >> be "hands on" control over the planner. Est

Re: [HACKERS] Index Tuning Features

2006-10-10 Thread Mark Woodward
> Simon Riggs <[EMAIL PROTECTED]> writes: >> - RECOMMEND command > >> Similar in usage to an EXPLAIN, the RECOMMEND command would return a >> list of indexes that need to be added to get the cheapest plan for a >> particular query (no explain plan result though). > > Both of these seem to assume t

Re: [HACKERS] Upgrading a database dump/restore

2006-10-09 Thread Mark Woodward
> Mark, > >> No one could expect that this could happen by 8.2, or the release after >> that, but as a direction for the project, the "directors" of the >> PostgreSQL project must realize that the dump/restore is becomming like >> the old locking vacuum problem. It is a *serious* issue for PostgreS

Re: [HACKERS] Upgrading a database dump/restore

2006-10-09 Thread Mark Woodward
> On Mon, Oct 09, 2006 at 11:50:10AM -0400, Mark Woodward wrote: >> > That one is easy: there are no rules. We already know how to deal >> with >> > catalog restructurings --- you do the equivalent of a pg_dump -s and >> > reload. Any proposed pg_upgra

Re: [HACKERS] Upgrading a database dump/restore

2006-10-09 Thread Mark Woodward
> "Mark Woodward" <[EMAIL PROTECTED]> writes: >>> Whenever someone actually writes a pg_upgrade, we'll institute a policy >>> to restrict changes it can't handle. > >> IMHO, *before* any such tool *can* be written, a set of rules must be >

Re: [HACKERS] Upgrading a database dump/restore

2006-10-08 Thread Mark Woodward
> "Mark Woodward" <[EMAIL PROTECTED]> writes: >> Not to cause any arguments, but this is sort a standard discussion that >> gets brought up periodically and I was wondering if there has been any >> "softening" of the attitudes against an "in pla

Re: [HACKERS] Upgrading a database dump/restore

2006-10-05 Thread Mark Woodward
> > Indeed. The main issue for me is that the dumping and replication > setups require at least 2x the space of one db. That's 2x the > hardware which equals 2x $$$. If there were some tool which modified > the storage while postgres is down, that would save lots of people > lots of money. Its tim

Re: [HACKERS] Upgrading a database dump/restore

2006-10-05 Thread Mark Woodward
> Mark Woodward wrote: >> I am currently building a project that will have a huge number of >> records, >> 1/2tb of data. I can't see how I would ever be able to upgrade >> PostgreSQL >> on this system. >> >> > > Slony will help you upgrad

[HACKERS] Upgrading a database dump/restore

2006-10-05 Thread Mark Woodward
Not to cause any arguments, but this is sort a standard discussion that gets brought up periodically and I was wondering if there has been any "softening" of the attitudes against an "in place" upgrade, or movement to not having to dump and restore for upgrades. I am aware that this is a difficult

Re: [HACKERS] Query Failed, out of memory

2006-10-05 Thread Mark Woodward
> On Thu, 2006-10-05 at 14:53 -0400, Luke Lonergan wrote: >> Is that in the release notes? > > Yes: "Allow COPY to dump a SELECT query (Zoltan Boszormenyi, Karel Zak)" I remember this discussion, it is cool when great features get added. ---(end of broadcast)--

Re: [HACKERS] Query Failed, out of memory

2006-10-05 Thread Mark Woodward
> Tom Lane wrote: >> "Mark Woodward" <[EMAIL PROTECTED]> writes: >> >>> psql -p 5435 -U pgsql -t -A -c "select client, item, rating, day from >>> ratings order by client" netflix > netflix.txt >>> >> >> FWIW, there&

Re: [HACKERS] Query Failed, out of memory

2006-10-05 Thread Mark Woodward
> >> > FWIW, there's a feature in CVS HEAD to instruct psql to try to use a >> > cursor to break up huge query results like this. For the moment I'd >> > suggest using COPY instead. >> >> >> That's sort of what I was afraid off. I am trying to get 100 million >> records into a text file in a speci

Re: [HACKERS] Query Failed, out of memory

2006-10-05 Thread Mark Woodward
> On Thu, Oct 05, 2006 at 11:56:43AM -0400, Mark Woodward wrote: >> The query was executed as: >> psql -p 5435 -U pgsql -t -A -c "select client, item, rating, day from >> ratings order by client" netflix > netflix.txt >> >> >> My question, it l

Re: [HACKERS] Query Failed, out of memory

2006-10-05 Thread Mark Woodward
> "Mark Woodward" <[EMAIL PROTECTED]> writes: >> psql -p 5435 -U pgsql -t -A -c "select client, item, rating, day from >> ratings order by client" netflix > netflix.txt > >> My question, it looks like the kernel killed psql, and not postmaster.

[HACKERS] Query Failed, out of memory

2006-10-05 Thread Mark Woodward
I am using the netflix database: Table "public.ratings" Column | Type | Modifiers +--+--- item | integer | client | integer | day| smallint | rating | smallint | The query was executed as: psql -p 5435 -U pgsql -t -A -c "select client, item, rating, da

Re: [HACKERS] Netflix Prize data

2006-10-04 Thread Mark Woodward
> "Mark Woodward" <[EMAIL PROTECTED]> writes: >> The rating, however, is one char 1~9. Would making it a char(1) buy >> anything? > > No, that would actually hurt because of the length word for the char > field. Even if you used the "char" type,

Re: [HACKERS] Netflix Prize data

2006-10-04 Thread Mark Woodward
> > "Greg Sabino Mullane" <[EMAIL PROTECTED]> writes: > >> CREATE TABLE rating ( >> movie SMALLINT NOT NULL, >> person INTEGER NOT NULL, >> rating SMALLINT NOT NULL, >> viewed DATE NOT NULL >> ); > > You would probably be better off putting the two smallints first followed > by > the

Re: [HACKERS] Netflix Prize data

2006-10-04 Thread Mark Woodward
> "Mark Woodward" <[EMAIL PROTECTED]> writes: >> The one thing I notice is that it is REAL slow. > > How fast is your disk? Counting on my fingers, I estimate you are > scanning the table at about 47MB/sec, which might or might not be > disk-limited... &g

Re: [HACKERS] Netflix Prize data

2006-10-04 Thread Mark Woodward
>> I signed up for the Netflix Prize. (www.netflixprize.com) >> and downloaded their data and have imported it into PostgreSQL. >> Here is how I created the table: > > I signed up as well, but have the table as follows: > > CREATE TABLE rating ( > movie SMALLINT NOT NULL, > person INTEGER NOT

[HACKERS] Netflix Prize data

2006-10-04 Thread Mark Woodward
I signed up for the Netflix Prize. (www.netflixprize.com) and downloaded their data and have imported it into PostgreSQL. Here is how I created the table: Table "public.ratings" Column | Type | Modifiers +-+--- item | integer | client | integer | rating | intege

[HACKERS] Mapping arbitriary and heirachical XML to tuple

2006-09-08 Thread Mark Woodward
I have a system by which I store complex data in PostgreSQL as an XML string. I have a simple function that can return a single value. I would like to return sets and sets of rows from the data. This is not a huge problem, as I've written a few of these functions. The question I'd like to put out

Re: [HACKERS] update/insert,

2006-07-05 Thread Mark Woodward
> On Tue, Jul 04, 2006 at 11:59:27AM +0200, Zdenek Kotala wrote: >> Mark, >> I don't know how it will exactly works in postgres but my expectations >> are: >> >> Mark Woodward wrote: >> >Is there a difference in PostgreSQL performance between these

[HACKERS] update/insert, delete/insert efficiency WRT vacuum and MVCC

2006-07-03 Thread Mark Woodward
Is there a difference in PostgreSQL performance between these two different strategies: if(!exec("update foo set bar='blahblah' where name = 'xx'")) exec("insert into foo(name, bar) values('xx','blahblah'"); or exec("delete from foo where name = 'xx'"); exec("insert into foo(name, bar) values

Re: [HACKERS] SO_SNDBUF size is small on win32?

2006-06-27 Thread Mark Woodward
> We have definitly seen weird timing issues sometimes when both client > and server were on Windows, but have been unable to pin it exactly on > what. From Yoshiykis other mail it looks like this could possibly be it, > since he did experience a speedup in the range we've been looking for in > tho

Re: [HACKERS] SO_SNDBUF size is small on win32?

2006-06-27 Thread Mark Woodward
I would set the SO_SNDBUF to 32768. > Hi, > > I see a performance issue on win32. This problem is causes by the > following URL. > > http://support.microsoft.com/kb/823764/EN-US/ > > On win32, default SO_SNDBUF value is 8192 bytes. And libpq's buffer is > 8192 too. > > pqcomm.c:117 > #define PQ

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-27 Thread Mark Woodward
> On Fri, Jun 23, 2006 at 06:37:01AM -0400, Mark Woodward wrote: >> While we all know session data is, at best, ephemeral, people still want >> some sort of persistence, thus, you need a database. For mcache I have a >> couple plugins that have a wide range of opition

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-27 Thread Mark Woodward
> Ühel kenal päeval, E, 2006-06-26 kell 09:10, kirjutas Mark Woodward: >> > Ãœhel kenal päeval, R, 2006-06-23 kell 17:27, kirjutas Bruce >> Momjian: >> >> Jonah H. Harris wrote: >> >> > On 6/23/06, Tom Lane <[EMAIL PROTECTED]> wrote: >&

Re: [HACKERS] vacuum row?

2006-06-26 Thread Mark Woodward
> On 6/24/06, Mark Woodward <[EMAIL PROTECTED]> wrote: >> I originally suggested a methodology for preserving MVCC and everyone is >> confusing it as update "in place," this isnot what I intended. > > Actually, you should've presented your idea as perf

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-26 Thread Mark Woodward
> Heikki Linnakangas wrote: >> On Mon, 26 Jun 2006, Jan Wieck wrote: >> >> > On 6/25/2006 10:12 PM, Bruce Momjian wrote: >> >> When you are using the update chaining, you can't mark that index row >> as >> >> dead because it actually points to more than one row on the page, >> some >> >> are non-vi

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-26 Thread Mark Woodward
> Ühel kenal päeval, R, 2006-06-23 kell 17:27, kirjutas Bruce Momjian: >> Jonah H. Harris wrote: >> > On 6/23/06, Tom Lane <[EMAIL PROTECTED]> wrote: >> > > What I see in this discussion is a huge amount of "the grass must be >> > > greener on the other side" syndrome, and hardly any recognition

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Mark Woodward
> On 6/24/2006 9:23 AM, Mark Woodward wrote: > >>> On Sat, 24 Jun 2006, Mark Woodward wrote: >>> >>>> I'm probably mistaken, but aren't there already forward references in >>>> tuples to later versions? If so, I'm only sugesting r

[HACKERS] vacuum row?

2006-06-24 Thread Mark Woodward
I originally suggested a methodology for preserving MVCC and everyone is confusing it as update "in place," this isnot what I intended. How about a form of vacuum that targets a particular row? Is this possible? Would if have to be by transaction? ---(end of broadcast)

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-24 Thread Mark Woodward
> On 6/24/06, Mark Woodward <[EMAIL PROTECTED]> wrote: >> > On 6/24/06, Mark Woodward <[EMAIL PROTECTED]> wrote: >> >> In the scenario, as previously outlined: >> >> >> >> ver001->verN->...->ver003->ver2->| >> >>

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-24 Thread Mark Woodward
> On 6/24/06, Mark Woodward <[EMAIL PROTECTED]> wrote: >> In the scenario, as previously outlined: >> >> ver001->verN->...->ver003->ver2->| >> ^-/ > > So you want to always keep an old version around? Prior to

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-24 Thread Mark Woodward
> On 6/24/06, Mark Woodward <[EMAIL PROTECTED]> wrote: >> Currently it looks like this: >> >> ver001->ver002->ver003->...-verN >> >> That's what t_ctid does now, right? Well, that's sort of stupid. Why not >> have it do this: >>

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-24 Thread Mark Woodward
> On Sat, 24 Jun 2006, Mark Woodward wrote: > >> I'm probably mistaken, but aren't there already forward references in >> tuples to later versions? If so, I'm only sugesting reversing the order >> and referencing the latest version. > > I thought I un

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-24 Thread Mark Woodward
> On 6/23/2006 3:10 PM, Mark Woodward wrote: > >> This is NOT an "in-place" update. The whole MVCC strategy of keeping old >> versions around doesn't change. The only thing that does change is one >> level of indirection. Rather than keep references to all v

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-23 Thread Mark Woodward
l that would be a show stopper. > > Rick > > On Jun 22, 2006, at 7:59 AM, Mark Woodward wrote: > >>> After a long battle with technology, [EMAIL PROTECTED] ("Mark >>> Woodward"), an earthling, wrote: >>>>> Clinging to sanity, [EMAIL PROTECTED]

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-23 Thread Mark Woodward
> Tom Lane wrote: >> If you're doing heavy updates of a big table then it's likely to end up >> visiting most of the table anyway, no? There is talk of keeping a map >> of dirty pages, but I think it'd be a win for infrequently-updated >> tables, not ones that need constant vacuuming. >> >> I thin

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-23 Thread Mark Woodward
> On 6/23/06, Mark Woodward <[EMAIL PROTECTED]> wrote: >> I, for one, see a particularly nasty unscalable behavior in the >> implementation of MVCC with regards to updates. > > I think this is a fairly common acceptance. The overhead required to > perform an UPDATE

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-23 Thread Mark Woodward
> > Bottom line: there's still lots of low-hanging fruit. Why are people > feeling that we need to abandon or massively complicate our basic > architecture to make progress? > > regards, tom lane I, for one, see a particularly nasty unscalable behavior in the implementation

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-23 Thread Mark Woodward
> Mark Woodward wrote: > >> > In case of the number of actively modified rows being in only tens or >> > low hundreds of thousands of rows, (i.e. the modified set fits in >> > memory) the continuous vacuum process shows up as just another >> backend, >>

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-23 Thread Mark Woodward
> Ühel kenal päeval, N, 2006-06-22 kell 12:41, kirjutas Mark Woodward: > >> > Depending on exact details and optimisations done, this can be either >> > slower or faster than postgresql's way, but they still need to do >> > something to get transactional

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-23 Thread Mark Woodward
>> Let me ask a question, you have this hundred million row table. OK, how >> much of that table is "read/write?" Would it be posible to divide the >> table into two (or more) tables where one is basically static, only >> infrequent inserts and deletes, and the other is highly updated? > > Well, al

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-23 Thread Mark Woodward
>> I suppose you have a table memberships (user_id, group_id) or something >> like it ; it should have as few columns as possible ; then try regularly >> clustering on group_id (maybe once a week) so that all the records for a >> particular group are close together. Getting the members of a gr

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-23 Thread Mark Woodward
>> The example is a very active web site, the flow is this: >> >> query for session information >> process HTTP request >> update session information >> >> This happens for EVERY http request. Chances are that you won't have >> concurrent requests for the same row, but you may have well over 100 >>

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-22 Thread Mark Woodward
> >> What you seem not to grasp at this point is a large web-farm, about 10 >> or >> more servers running PHP, Java, ASP, or even perl. The database is >> usually >> the most convenient and, aside from the particular issue we are talking >> about, best suited. > > The answer is sticky session

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-22 Thread Mark Woodward
> >> As you can see, in about a minute at high load, this very simple table >> lost about 10% of its performance, and I've seen worse based on update >> frequency. Before you say this is an obscure problem, I can tell you it >> isn't. I have worked with more than a few projects that had to switch

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-22 Thread Mark Woodward
>> > You mean systems that are designed so exactly, that they can't take >> 10% >> > performance change ? >> >> No, that's not really the point, performance degrades over time, in one >> minute it degraded 10%. >> >> The update to session ratio has a HUGE impact on PostgreSQL. If you have >> a >> t

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-22 Thread Mark Woodward
> Christopher Browne <[EMAIL PROTECTED]> writes: >> After a long battle with technology, [EMAIL PROTECTED] ("Mark >> Woodward"), an earthling, wrote: >>> Not true. Oracle does not seem to exhibit this problem. > >> Oracle suffers a problem in

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-22 Thread Mark Woodward
> Ühel kenal päeval, N, 2006-06-22 kell 10:20, kirjutas Jonah H. Harris: >> On 6/22/06, Alvaro Herrera <[EMAIL PROTECTED]> wrote: >> > > Hmm, OK, then the problem is more serious than I suspected. >> > > This means that every index on a row has to be updated on every >> > > transaction that modif

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-22 Thread Mark Woodward
> Ühel kenal päeval, N, 2006-06-22 kell 09:59, kirjutas Mark Woodward: >> > After a long battle with technology, [EMAIL PROTECTED] ("Mark >> > Woodward"), an earthling, wrote: >> >>> Clinging to sanity, [EMAIL PROTECTED] ("Mark Woodward"

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-22 Thread Mark Woodward
> After a long battle with technology, [EMAIL PROTECTED] ("Mark > Woodward"), an earthling, wrote: >>> Clinging to sanity, [EMAIL PROTECTED] ("Mark Woodward") mumbled into >>> her beard: [snip] >>> >>> 1. The index points to all the

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-22 Thread Mark Woodward
> Clinging to sanity, [EMAIL PROTECTED] ("Mark Woodward") mumbled into > her beard: >> We all know that PostgreSQL suffers performance problems when rows are >> updated frequently prior to a vacuum. The most serious example can be >> seen >> by using Postg

[HACKERS] vacuum, performance, and MVCC

2006-06-21 Thread Mark Woodward
We all know that PostgreSQL suffers performance problems when rows are updated frequently prior to a vacuum. The most serious example can be seen by using PostgreSQL as a session handler for a busy we site. You may have thousands or millions of active sessions, each being updated per page hit. Eac

Re: [HACKERS] Preventing DELETE and UPDATE without a WHERE clause?

2006-06-17 Thread Mark Woodward
> On 6/16/06, Mark Woodward <[EMAIL PROTECTED]> wrote: >> > Chris Campbell <[EMAIL PROTECTED]> writes: >> >> I heard an interesting feature request today: preventing the >> >> execution of a DELETE or UPDATE query that does not have a WHERE >>

Re: [HACKERS] Preventing DELETE and UPDATE without a WHERE clause?

2006-06-16 Thread Mark Woodward
> Chris Campbell <[EMAIL PROTECTED]> writes: >> I heard an interesting feature request today: preventing the >> execution of a DELETE or UPDATE query that does not have a WHERE clause. > > These syntaxes are required by the SQL spec. Furthermore, it's easy > to imagine far-more-probable cases in w

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-08 Thread Mark Woodward
> On Wed, Jun 07, 2006 at 07:07:55PM -0400, Mark Woodward wrote: >> I guess what I am saying is that PostgreSQL isn't "smooth," between >> checkpoints and vacuum, it is near impossible to make a product that >> performs consistently under high load. > &g

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-07 Thread Mark Woodward
> On Wed, Jun 07, 2006 at 11:47:45AM -0400, Tom Lane wrote: >> Zdenek Kotala <[EMAIL PROTECTED]> writes: >> > Koichi Suzuki wrote: >> >> I've once proposed a patch for 64bit transaction ID, but this causes >> >> some overhead to each tuple (XMIN and XMAX). >> >> > Did you check performance on 32-bi

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Mark Woodward
> Mark Woodward wrote: >> OK, here's my problem, I have a nature study where we have about 10 >> video >> cameras taking 15 frames per second. >> For each frame we make a few transactions on a PostgreSQL database. > > Maybe if you grouped multiple oper

Re: [HACKERS] AGREGATE FUNCTIONS

2006-06-06 Thread Mark Woodward
> Hello, I would like to know where in the source-code of postgres is > located the code of the aggregate functions min, max, avg. > I wish to develop more statistical aggregate functions, and I prefer to > use C than to write then in the PL/R. There is a library in "contrib" called "intagg." I wr

[HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Mark Woodward
OK, here's my problem, I have a nature study where we have about 10 video cameras taking 15 frames per second. For each frame we make a few transactions on a PostgreSQL database. We want to keep about a years worth of data at any specific time. We have triggers that fire is something interesting is

Re: [HACKERS] [PERFORM] psql -A (unaligned format) eats too much

2006-06-05 Thread Mark Woodward
> Mark Woodward wrote: >>> "Jim C. Nasby" <[EMAIL PROTECTED]> writes: >>> >>>> On Mon, Jun 05, 2006 at 11:27:30AM -0400, Tom Lane wrote: >>>> >>>>> I'm reading this as just another uninformed complaint about libpq

Re: [HACKERS] [PERFORM] psql -A (unaligned format) eats too much

2006-06-05 Thread Mark Woodward
> "Jim C. Nasby" <[EMAIL PROTECTED]> writes: >> On Mon, Jun 05, 2006 at 11:27:30AM -0400, Tom Lane wrote: >>> I'm reading this as just another uninformed complaint about libpq's >>> habit of buffering the whole query result. It's possible that there's >>> a memory leak in the -A path specifically,

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
> >>> Allow COPY to output from views >>> Another idea would be to allow actual SELECT statements in a COPY. >>> >>> Personally I strongly favor the second option as being more flexible >>> than the first. >> >> >> I second that - allowing arbitrary SELECT statements as a COPY source >> seems muc

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
> Mark Woodward wrote: > ... > >>>> pg_dump -t mytable | psql -h target -c "COPY mytable FROM STDIN" >>>> >>>> With a more selective copy, you can use pretty much this mechanism to >>>> limit a copy to a sumset of the records in a

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
> Mark Woodward wrote: > ... >>>>> create table as select ...; followed by a copy of that table >>>>> if it really is faster then just the usual select & fetch? >>>> Why "create table?" >>> Just to simulate and time the pro

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
> Mark Woodward wrote: >>> Mark Woodward wrote: >>>> Tom had posted a question about file compression with copy. I thought >>>> about it, and I want to through this out and see if anyone things it >>>> is >>>> a >>>> good ide

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
> Mark Woodward wrote: >> Tom had posted a question about file compression with copy. I thought >> about it, and I want to through this out and see if anyone things it is >> a >> good idea. >> >> Currently, the COPY command only copies a table, what if

[HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
Tom had posted a question about file compression with copy. I thought about it, and I want to through this out and see if anyone things it is a good idea. Currently, the COPY command only copies a table, what if it could operate with a query, as: COPY (select * from mytable where foo='bar') as BA

Re: [HACKERS] Possible TODO item: copy to/from pipe

2006-06-01 Thread Mark Woodward
> After re-reading what I just wrote to Andreas about how compression of > COPY data would be better done outside the backend than inside, it > struck me that we are missing a feature that's fairly common in Unix > programs. Perhaps COPY ought to have the ability to pipe its output > to a shell co

Re: [HACKERS] LIKE, leading percent, bind parameters and indexes

2006-05-26 Thread Mark Woodward
> On Thu, May 25, 2006 at 08:41:17PM -0300, Rodrigo Hjort wrote: >> > >> >I think more exactly, the planner can't possibly know how to plan an >> >indexscan with a leading '%', because it has nowhere to start. >> > >> >> The fact is that index scan is performed on LIKE expression on a string >> not

Re: [HACKERS] Performance Issues

2006-05-23 Thread Mark Woodward
> Dhanaraj M wrote: >> I have the following doubts. >> >> 1. Does postgres create an index on every primary key? Usually, queries >> are performed against a table on the primary key, so, an index on it >> will be very useful. > > Yes, a unique index is used to enforce the primary-key. Well, here

Re: [pgsql-advocacy] [HACKERS] [OT] MySQL is bad, but THIS bad?

2006-05-22 Thread Mark Woodward
> On Sat, May 20, 2006 at 02:29:01PM +0200, Dawid Kuroczko wrote: >> On 5/20/06, Lukas Smith <[EMAIL PROTECTED]> wrote: >> >The improvements to the installer are great, but there simply needs to >> >be a packaged solution that adds more of the things people are very >> >likely to use. From my under

Re: [HACKERS] String Similarity

2006-05-22 Thread Mark Woodward
en trying best run, total match, and a host of others, but haven't found one I really like. > > Chris > > Mark Woodward wrote: >> I have a side project that needs to "intelligently" know if two strings >> are contextually similar. Think about how CDDB informat

Re: [HACKERS] [OT] MySQL is bad, but THIS bad?

2006-05-20 Thread Mark Woodward
> On Fri, May 19, 2006 at 07:04:47PM -0400, Bruce Momjian wrote: >> > libreadline is not a problem because you can distribute postgresql >> > compiled with readline and comply with all licences involved >> > simultaneously. It doesn't work with openssl because the licence >> > requires things that

Re: [HACKERS] [OT] MySQL is bad, but THIS bad?

2006-05-20 Thread Mark Woodward
> > My question is whether psql using libreadline.so has to be GPL, meaning > the psql source has to be included in a binary distribution. If I understand what I have been told by lawyers, here's what using a GPL, and NOT LGPL, library means: According to RMS, the definition of a derivitive work

Re: [HACKERS] String Similarity

2006-05-20 Thread Mark Woodward
> What I was hoping someone had was a function that could find the substring > runs in something less than a strlen1*strlen2 number of operations and a > numerically sane way of representing the similarity or difference. Acually, it is more like strlen1*strlen2*N, where N is the number of valid r

Re: [HACKERS] String Similarity

2006-05-20 Thread Mark Woodward
> Get pg_trgm http://www.sai.msu.su/~megera/oddmuse/index.cgi/ReadmeTrgm > It doesn't depends on language. That's an interesting approach. This is what I got: apps$ ./stratest "pink floyd dark side of the moon money" "dark side of the moon pink floyd" Match: dark side of the moon Match: pink flo

Re: [HACKERS] String Similarity

2006-05-19 Thread Mark Woodward
> > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > >> I have a side project that needs to "intelligently" know if two strings >> are contextually similar. > > The examples you gave seem heavy on word order and whitespace > consideration, > before applying any algorithms. Here's a quick perl ve

Re: [HACKERS] String Similarity

2006-05-19 Thread Mark Woodward
> Mark Woodward wrote: >> I have a side project that needs to "intelligently" know if two strings >> are contextually similar. Think about how CDDB information is collected >> and sorted. It isn't perfect, but there should be enough information to >

[HACKERS] String Similarity

2006-05-19 Thread Mark Woodward
I have a side project that needs to "intelligently" know if two strings are contextually similar. Think about how CDDB information is collected and sorted. It isn't perfect, but there should be enough information to be usable. Think about this: "pink floyd - dark side of the moon - money" "dark s

  1   2   >