Re: [HACKERS] Make subquery alias optional in FROM clause

2017-02-26 Thread Matthew Woodcraft
On 2017-02-24 07:25, Robert Haas wrote: > I don't think it's only Oracle that allows omitting the > alias; I think there are a number of other systems that behave > similarly. SQLite, for example. Making conversions from SQLite to Postgres easier is a Good Thing. "subquery in FROM must have an al

[HACKERS] [PATCH] Remove TZ entry from postgres CLI doc page.

2016-03-29 Thread Matthew Somerville
is not looked at when the server is started. I am using Test::PostgreSQL to create a test database; it empties the postgresql.conf created by initdb and is then therefore not using the correct timezone, and it took me a while to work out what was happening and why I couldn't use TZ when starti

[HACKERS] Add a hint when postgresql.conf hot_standby = 'off' and recovery.conf standby = 'on'

2015-05-21 Thread Matthew Kelly
If you do the following sequence, the server gives the least helpful error message: initdb data pg_ctl -D data -l logfile start # The following reconfigs are obvious based on error message if you try to take a base backup echo 'local replication all trust’ >> data/pg_hba.conf sed -i 's/

Re: [HACKERS] [GENERAL] 4B row limit for CLOB tables

2015-02-05 Thread Matthew Kelly
> That's assuming that toasting is evenly spread between tables. In my > experience, that's not a great bet... Time to create a test: SELECT chunk_id::bigint/10 as id_range, count(*), count(*)/(10::float) density FROM (SELECT chunk_id FROM pg_toast.pg_toast_39000165 WHERE chunk_id < 100

Re: [HACKERS] [GENERAL] 4B row limit for CLOB tables

2015-02-03 Thread Matthew Kelly
> Hmm 2^32 times aprox. 2kB (as per usual heuristics, ~4 rows per heap > page) is 8796093022208 (~9e13) bytes > ... which results in 8192 1GB segments :O 8192 1GB segments is just 8TB, its not _that_ large. At TripAdvisor we’ve been using a NoSQL solution to do session storage. We are look

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-12 Thread Matthew Woodcraft
On 2014-10-12 13:40, Marko Tiikkaja wrote: > On 10/12/14, 2:36 PM, Matthew Woodcraft wrote: >> On 2014-10-10 19:44, Kevin Grittner wrote: >>> To restate: to do so is conflating the logical definition of the >>> database with a particular implementation detail. As just o

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-12 Thread Matthew Woodcraft
On 2014-10-10 19:44, Kevin Grittner wrote: > Peter Geoghegan wrote: >> People keep remarking that they don't like that you can (optionally) >> name a unique index explicitly, [...] > To restate: to do so is conflating the logical definition of the > database with a particular implementation de

Re: [HACKERS] Collations and Replication; Next Steps

2014-09-17 Thread Matthew Kelly
witched to 'don't trust any change until shown otherwise', so that may have bled into my last email. - Matt K. On Sep 17, 2014, at 8:17 AM, Robert Haas wrote: > On Wed, Sep 17, 2014 at 9:07 AM, Matthew Kelly wrote: >> Here is where I think the timezone and PostGIS cases

Re: [HACKERS] Collations and Replication; Next Steps

2014-09-17 Thread Matthew Kelly
Here is where I think the timezone and PostGIS cases are fundamentally different: I can pretty easily make sure that all my servers run in the same timezone. That's just good practice. I'm also going to install the same version of PostGIS everywhere in a cluster. I'll build PostGIS and its de

[HACKERS] Collations and Replication; Next Steps

2014-09-16 Thread Matthew Kelly
Hello, Last month, I brought up the following issue to the general mailing list about how running streaming replication between machines running different versions of glibc can cause corrupt indexes. http://www.postgresql.org/message-id/ba6132ed-1f6b-4a0b-ac22-81278f5ab...@tripadvisor.com In th

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-12-09 Thread Matthew Gerber
took me quite a while to track down the problem so I hope someone can figure out what is going on under the hood. It seems to be a pretty significant problem. Cheers, Matt On Sun, Nov 11, 2012 at 9:45 PM, Matthew Gerber wrote: > > > On Sun, Nov 11, 2012 at 8:27 PM, Tom Lane wrote: > >

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 8:27 PM, Tom Lane wrote: > Noah Misch writes: > > So, I can reproduce the lower threshold, but the exception type does not > agree > > with the one Matthew observed. > > I finally got around to looking at the link you provided about error > 0x

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 2:43 PM, Noah Misch wrote: > On Sun, Nov 11, 2012 at 10:10:31AM -0500, Matthew Gerber wrote: > > > > Matthew Gerber writes: > > > > >> Here is the command that was executing when the 0xC409 > exception > > > was >

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 12:22 PM, Tom Lane wrote: > Matthew Gerber writes: > > On Sun, Nov 11, 2012 at 11:19 AM, Tom Lane wrote: > >> How long is "long"? > > > I was seeing queries with around 5000-7000 "UNION ALL" statements. > >

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 11:19 AM, Tom Lane wrote: > Matthew Gerber writes: > > On Sun, Nov 11, 2012 at 12:23 AM, Noah Misch wrote: > >> It signifies scribbling past the end of the frame's local variables: > >> http://msdn.microsoft.com/en-us/library/8dbf701c.as

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 12:23 AM, Noah Misch wrote: > On Sun, Nov 04, 2012 at 02:30:38PM -0500, Tom Lane wrote: > > Matthew Gerber writes: > > >> Here is the command that was executing when the 0xC409 exception > was > > >> raised: > > >> IN

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-05 Thread Matthew Gerber
On Sun, Nov 4, 2012 at 3:39 AM, Craig Ringer wrote: > On 11/04/2012 08:47 AM, Matthew Gerber wrote: > > > So I attached the VS debugger, but the server died without raising an > exception in VS. Not sure what's going on here. > >> >> Try creating a direct

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-04 Thread Matthew Gerber
On Sun, Nov 4, 2012 at 3:39 AM, Craig Ringer wrote: > On 11/04/2012 08:47 AM, Matthew Gerber wrote: > > > Here is the command that was executing when the 0xC409 exception was > raised: > > INSERT INTO places (bounding_box,country,full_name,id,name

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-03 Thread Matthew Gerber
On Fri, Nov 2, 2012 at 9:00 PM, Noah Misch wrote: > On Fri, Nov 02, 2012 at 02:05:47PM -0500, Merlin Moncure wrote: > > On Fri, Nov 2, 2012 at 1:25 PM, Matthew Gerber > wrote: > > > I am encountering an error on my Postgres installation for Windows > Server > >

Re: [HACKERS] enhanced error fields

2012-07-03 Thread Matthew Woodcraft
Peter Geoghegan writes: > So I took a look at the patch eelog-2012-05-09.diff today. All of the > following remarks apply to it alone. I've been trying out this patch for my own interest (I'm very pleased to see work on this feature), and I have a couple of suggestions from a user's point of view

Re: [HACKERS] Patch: Allow SQL-language functions to reference parameters by parameter name

2012-01-29 Thread Matthew Draper
they generally are. Maybe it'd be better suited to be explicitly discussed in the comments? Thanks, Matthew -- matt...@trebex.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Allow SQL-language functions to reference parameters by parameter name

2012-01-23 Thread Matthew Draper
first. No better idea from me at the moment, though. By my reading of (a draft of) the spec, Subclause 6.6, "", Syntax Rules 8.b.i-iii, the current behaviour is correct. But I join you in wondering whether we should permit the function name to be schema-qualified anyway. Matthew -- mat

[HACKERS] Patch: Allow SQL-language functions to reference parameters by parameter name

2012-01-14 Thread Matthew Draper
iginal WIP post: http://archives.postgresql.org/pgsql-hackers/2011-03/msg01479.php This is an open TODO: http://wiki.postgresql.org/wiki/Todo#SQL-Language_Functions I've just added the above post to the CF app; I'll update it to point to this one once it appears. Thanks! Matthew --

Re: [HACKERS] Improve lseek scalability v3

2011-09-19 Thread Matthew Wilcox
et the current size of a file before; I'd assumed everybody used fstat. Given this legitimate reason for a high-frequency calling of lseek, I withdraw my earlier objection to the patch series. -- Matthew Wilcox Intel Open Source Technology Centre "Bill, look, we

Re: [HACKERS] Improve lseek scalability v3

2011-09-16 Thread Matthew Wilcox
g to the blog entry it's used for "the query planner". Does the query planner need to know the exact number of bytes in the file, or is it after an order-of-magnitude? Or to-the-nearest-gigabyte? -- Matthew Wilcox Intel Open Source Technology Centre "B

Re: [HACKERS] fsync reliability

2011-04-23 Thread Matthew Woodcraft
On 2011-04-22 21:55, Greg Smith wrote: > On 04/22/2011 09:32 AM, Simon Riggs wrote: >> OK, that's good, but ISTM we still have a hole during >> RemoveOldXlogFiles() where we don't fsync or open/close the file, just >> rename it. > > This is also something that many applications rely upon working a

[HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-03-25 Thread Matthew Draper
; passes `make check` on current HEAD, a4425e3) Thanks! Matthew -- matt...@trebex.net diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c new file mode 100644 index ce3b77b..be71fbb *** a/src/backend/executor/functions.c --- b/src/backend/executor/functions.c

Re: [HACKERS] compile/install of git

2010-09-19 Thread Matthew D. Fuller
oesn't help if you're talking HTTP to just a plain HTTP host. -- Matthew Fuller (MF4839) | fulle...@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. -- Sent via pgsql-hackers mai

Re: [HACKERS] git: uh-oh

2010-08-17 Thread Matthew D. Fuller
ts to svn, but that it's built on (/part of) cvs2svn, so presumably a lot of the "figure out changesets and branch membership" logic and the "get things in the shape svn wants" logic are intertwined. -- Matthew Fuller (MF4839) | fulle...@over-yonder.net Systems/Net

Re: [HACKERS] [JDBC] Trouble with COPY IN

2010-07-29 Thread Matthew Wakeling
, where the copy is still locked, but we know that the CommandComplete and ReadyForQuery messages have been received. We can no longer unlock the copy in processCopyData - we need to do that in endCopy instead, after calling processCopyData to ensure that we wait for a valid CommandComplete an

[HACKERS] Alias to rollback keyword

2010-03-31 Thread Matthew Altus
Hey, After dealing with a production fault and having to rollback all the time, I kept typing a different word instead of rollback. So I created a patch to accept this word as an alias for rollback. Obviously it's not part of the sql standard, but could be a nice extension for postgresql. Se

Re: [HACKERS] 8.5 release timetable, again

2009-08-26 Thread Matthew T. O'Connor
Alvaro Herrera wrote: This seems a good idea. Possibly pushing the betas more aggresively to current users would make them tested not only by PG hackers ... Isn't this the purpose of the new alpha releases, at lease to some extent. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] Visibility map, partial vacuums

2008-11-23 Thread Matthew T. O'Connor
Tom Lane wrote: However, my comment above was too optimistic, because in an insert-only scenario autovac would in fact not trigger VACUUM at all, only ANALYZE. So it seems like we do indeed want to rejigger autovac's rules a bit to account for the possibility of wanting to apply vacuum to get vi

Re: [HACKERS] Block-level CRC checks

2008-11-17 Thread Matthew T. O'Connor
Aidan Van Dyk wrote: * Greg Stark <[EMAIL PROTECTED]> [081117 03:54]: I thought of saying that too but it doesn't really solve the problem. Think of what happens if someone sets a hint bit on a dirty page. If the page is dirty from a "real change", then it has a WAL backup block record alread

Re: [HACKERS] [WIP] In-place upgrade

2008-11-10 Thread Matthew T. O'Connor
Tom Lane wrote: Decibel! <[EMAIL PROTECTED]> writes: I think that's pretty seriously un-desirable. It's not at all uncommon for databases to stick around for a very long time and then jump ahead many versions. I don't think we want to tell people they can't do that. Of course they

Re: [HACKERS] RAM-only temporary tables

2008-11-05 Thread Matthew T. O'Connor
Kevin Grittner wrote: An idea for a possible enhancement to PostgreSQL: allow creation of a temporary table without generating any disk I/O. (Creating and dropping a three-column temporary table within a database transaction currently generates about 150 disk writes). If some circumstances don

Re: [HACKERS] 8.3 .4 + Vista + MingW + initdb = ACCESS_DENIED

2008-10-15 Thread Matthew T. O'Connor
Tom Lane wrote: Andrew Chernow <[EMAIL PROTECTED]> writes: Be careful. From LockFileEx docs: "However, the time it takes for the operating system to unlock these locks depends upon available system resources. Therefore, it is recommended that your process explicitly unlock all files it has

Re: [HACKERS] CREATE DATABASE vs delayed table unlink

2008-10-10 Thread Matthew Wakeling
d the directory 33049747 no longer exists either. I'll try again to get some details next time it happens. Matthew -- $ rm core Segmentation Fault (core dumped) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresq

Re: [HACKERS] CREATE DATABASE vs delayed table unlink

2008-10-09 Thread Matthew Wakeling
ed by FlushDatabaseBuffers? Matthew -- Isn't "Microsoft Works" something of a contradiction? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] CREATE DATABASE vs delayed table unlink

2008-10-09 Thread Matthew Wakeling
On Thu, 9 Oct 2008, Tom Lane wrote: So I'm mystified how Matthew could have seen the expected error and yet had the destination tree (or at least large chunks of it) left behind. Remember I was running 8.3.0, and you mentioned a few changes after that version which would have made sur

Re: [HACKERS] September CommitFest Closed

2008-10-01 Thread Matthew T. O'Connor
Josh Berkus wrote: For the September commitfest, 29 patches were applied (one to pgFoundry) and 18 patches were sent back for more work. More importantly, six *new* reviewers completed reviews of of various patches: Abbas Butt, Alex Hunsaker, Markus Wanner, Ibrar Ahmed, Ryan Bradetich and Gi

Re: [HACKERS] [PATCHES] VACUUM Improvements - WIP Patch

2008-08-24 Thread Matthew T. O'Connor
Tom Lane wrote: "Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: I think everyone agrees that partial vacuums would be useful / *A Good Thing* but it's the implementation that is the issue. I'm not sure how important it will really be once we have su

Re: [HACKERS] [PATCHES] VACUUM Improvements - WIP Patch

2008-08-24 Thread Matthew T. O'Connor
Joshua D. Drake wrote: Merlin Moncure wrote: Well, there doesn't seem to be a TODO for partial/restartable vacuums, which were mentioned upthread. This is a really desirable feature for big databases and removes one of the reasons to partition large tables. I would agree that partial vacuums wo

Re: [HACKERS] Do we really want to migrate plproxy and citext into PG core distribution?

2008-07-23 Thread Matthew T. O'Connor
Tom Lane wrote: "Greg Sabino Mullane" <[EMAIL PROTECTED]> writes: Code outside of core, is, in reality, less reviewed, less likely to work well with recent PG versions, and more likely to cause problems. It's also less likely to be found by people, less likely to be used by people, and less like

Re: [HACKERS] Concurrent VACUUM and ANALYZE

2008-07-21 Thread Matthew T. O'Connor
Jonah H. Harris wrote: On Mon, Jul 21, 2008 at 10:19 PM, Tom Lane <[EMAIL PROTECTED]> wrote: I don't find this a compelling argument, at least not without proof that the various vacuum-improvement projects already on the radar screen (DSM-driven vacuum, etc) aren't going to fix your problem.

Re: [HACKERS] Vacuuming leaked temp tables (once again)

2008-06-27 Thread Matthew T. O'Connor
Tom Lane wrote: We might have to rearrange the logic a bit to make that happen (I'm not sure what order things get tested in), but a log message does seem like a good idea. I'd go for logging anytime an orphaned table is seen, and dropping once it's past the anti-wraparound horizon. Is there a

Re: [HACKERS] XIDs and big boxes again ...

2008-05-12 Thread Matthew T. O'Connor
eed a database wide vacuum to solve the XID wraparound problem, so I think the answer is upgrade to 8.3 and see if you still have this problem. Matthew O'Connor -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.or

Re: [HACKERS] Posting to hackers and patches lists

2008-05-07 Thread Matthew T. O'connor
Alex Hunsaker wrote: A big part of my problem with the split is if there is a discussion taking place on -hackers I want to be able to reply to the discussion and say "well, here is what I was thinking". Sending it to -patches first waiting for it to hit the archive so I can link to it in my rep

Re: [HACKERS] Posting to hackers and patches lists

2008-05-07 Thread Matthew T. O'connor
Alex Hunsaker wrote: In fact I would argue -patches should go away so we dont have that split. +1I think the main argument for the split is to keep the "large" patch emails off the hackers list, but I don't think that limit is so high that it's a problem. People have to gzip their patche

[HACKERS] Visa CISP PCI compliance needs SHA1?

2008-04-02 Thread Matthew Wetmore
md5 hash )still the same as md5 auth We also have global customers so we understand the US v non-US export stuff. Any direction is appreciated. Thanks in advance. /matthew wetmore -- Matthew Wetmore Secom International, Inc 9610 Bellanca, Ave. Los Angeles, CA 90045 310-641-1290 This e-mail

[HACKERS] US VISA CISP PCI comp. needs SHA1

2008-04-02 Thread Matthew Wetmore
data] We just heard back today that they would like to use SHA1 for pwd auth. does anyone have any doco that will support md5 vs. SHA1? We also have global customers so we understand the us v non-US export stuff. Any direction is appreciated. Thanks in advance. /matthew wetmore -- Matthew

[HACKERS] offtopic, historic, in 8.1, how to find relfrozenxid by table?

2007-11-17 Thread Matthew Grosso
in 8.1, how do I find the relfrozenxid of a table? we're running 8.1.9, and have turned off autovacuum for a wee bit too long... thanks to the verbose warnings, we still have a few million transactions left before it locks up, although thats not much time for us. I'd like to focus vacuum on the

[HACKERS] Built PostgreSQL 8.3 BETA on FreeBSD/SPARC64

2007-10-24 Thread Matthew Alton
Machine: Sun Microsystems Ultra 5 (SPARC IIi). OS: FreeBSD 6.2/SPARC64 Compiler: gcc v.3.4.6 PostgreSQL 8.3 Beta builds on my system with no trouble. Matthew Alton

Re: [HACKERS] First steps with 8.3 and autovacuum launcher

2007-10-01 Thread Matthew T. O'Connor
Tom Lane wrote: If you insist on crafting a solution that only fixes this problem for pg_restore's narrow usage, you'll be back revisiting it before beta1 has been out a month. I don't know much about what is involved in crafting these solutions, but it seems we're close to beta and probably

Re: [HACKERS] First steps with 8.3 and autovacuum launcher

2007-10-01 Thread Matthew T. O'Connor
Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: This is an interesting idea, but I think it's attacking the wrong problem. To me, the problem here is that an ANALYZE should not block CREATE INDEX or certain forms of ALTER TABLE. I doubt that that will work; in particular I'

Re: [HACKERS] More logging for autovacuum

2007-08-07 Thread Matthew T. O'Connor
Gregory Stark wrote: I'm having trouble following what's going on with autovacuum and I'm finding the existing logging insufficient. In particular that it's only logging vacuum runs *after* the vacuum finishes makes it hard to see what vacuums are running at any given time. Also, I want to see wh

Re: [HACKERS] Still recommending daily vacuum...

2007-07-06 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Matthew T. O'Connor wrote: Well, if a table has 10 rows, and we keep the current threshold of 1000 rows, then this table must have 1002 dead tuples (99% dead tuples, 1002 dead + 10 live) before being vacuumed. This seems wasteful because there are 500 dead tuples on i

Re: [HACKERS] Still recommending daily vacuum...

2007-07-03 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Jim C. Nasby wrote: FWIW, I normally go with the 8.2 defaults, though I could see dropping vacuum_scale_factor down to 0.1 or 0.15. I also think the thresholds could be decreased further, maybe divide by 10. How about pushing thresholds all the way down to 0? As long a

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Jim C. Nasby escribió: There *is* reason to allow setting the naptime smaller, though (or at least there was; perhaps Alvero's recent changes negate this need): clusters that have a large number of databases. I've worked with folks who are in a hosted environment and give e

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Matthew T. O'Connor
Tom Lane wrote: "Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: How about creating a log-writing-process? Postmaster could write to the log files directly until the log-writer is up and running, then all processes can send their log output through the log-writer. We

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Matthew T. O'Connor
Andrew Dunstan wrote: The situation with this patch is that I now have it in a state where I think it could be applied, but there is one blocker, namely that we do not have a way of preventing the interleaving of log messages from different backends, which leads to garbled logs. This is an exis

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Matthew O'Connor
Michael Paesold wrote: Matthew T. O'Connor schrieb: Do we need a configurable autovacuum naptime at all? I know I put it in the original contrib autovacuum because I had no idea what knobs might be needed. I can't see a good reason to ever have a naptime longer than the default

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-07 Thread Matthew T. O'Connor
Tom Lane wrote: "Andrew Hammond" <[EMAIL PROTECTED]> writes: Hmmm... it seems to me that points new users towards not using autovacuum, which doesn't seem like the best idea. I think it'd be better to say that setting the naptime really high is a Bad Idea. It seems like we should have an upper

Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit

2007-06-07 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: But this is misleading (started postmaster with good value, then edited postgresql.conf and entered "-2"): 17903 LOG: received SIGHUP, reloading configuration files 17903 LOG: -2 is outside the valid range for pa

Re: [HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-06 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Simon Riggs wrote: On Wed, 2007-06-06 at 12:17 -0400, Matthew T. O'Connor wrote: Florian G. Pflug wrote: Work done so far: - .) Don't start autovacuum and bgwriter. Do table stats used by the planner get replicated on a PITR slave? I assume

Re: [HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-06 Thread Matthew T. O'Connor
Florian G. Pflug wrote: Work done so far: - .) Don't start autovacuum and bgwriter. Do table stats used by the planner get replicated on a PITR slave? I assume so, but if not, you would need autovac to do analyzes. ---(end of broadcast)-

Re: [HACKERS] Autovacuum versus rolled-back transactions

2007-06-01 Thread Matthew T. O'Connor
Tom Lane wrote: ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: Our documentation says | analyze threshold = analyze base threshold | + analyze scale factor * number of tuples | is compared to the total number of tuples inserted, updated, or deleted | since the last ANALYZE.

Re: [HACKERS] Autovacuum versus rolled-back transactions

2007-05-26 Thread Matthew O'Connor
Tom Lane wrote: This means that a table could easily be full of dead tuples from failed transactions, and yet autovacuum won't do a thing because it doesn't know there are any. Perhaps this explains some of the reports we've heard of tables bloating despite having autovac on. I think this is o

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-06 Thread Matthew O'Connor
Devrim Gündüz wrote: Hi, On Fri, 2007-04-06 at 01:23 -0400, Matthew T. O'Connor wrote: The other thing to consider is that CentOS 5 has Xen built right in, so you should be able run VMs without VMWare on it. ... if the kernel of the OS has Xen support, there will be no performance pe

Re: [HACKERS] What X86/X64 OS's do we need coverage for?

2007-04-05 Thread Matthew T. O'Connor
Larry Rosenman wrote: I might use that as the base then, since the hardware finishes getting here tomorrow. The other thing to consider is that CentOS 5 has Xen built right in, so you should be able run VMs without VMWare on it. ---(end of broadcast)--

Re: [HACKERS] Documentation access problems.

2007-03-23 Thread Matthew T. O'Connor
Bruce Momjian wrote: Tom Lane wrote: "Matthew T. O'Connor" writes: Lots of projects publish their HTML docs in two formats: One Big HTML file with everything; Broken up into many HTML files that link to each other. This would allow you you have one big searchable document

Re: [HACKERS] Documentation access problems.

2007-03-23 Thread Matthew T. O'Connor
Joshua D. Drake wrote: The big thing for me, is a single document, zero clicks, that is searchable. PDF and plain text are the only thing that give me that. If you are really zealous you can even use Beagle (which I don't) to preindex the PDF for you for easy searching. Lots of projects publish

Re: [HACKERS] autovacuum next steps, take 3

2007-03-09 Thread Matthew T. O'Connor
Tom Lane wrote: "Matthew T. O'Connor" writes: It's not clear to me why a worker cares that there is a new worker, since the new worker is going to ignore all the tables that are already claimed by all worker todo lists. That seems wrong to me, since it means that new

Re: [HACKERS] autovacuum next steps, take 3

2007-03-09 Thread Matthew T. O'Connor
My initial reaction is that this looks good to me, but still a few comments below. Alvaro Herrera wrote: Here is a low-level, very detailed description of the implementation of the autovacuum ideas we have so far. launcher's dealing with databases - [ Snip ]

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-27 Thread Matthew T. O'Connor
Tom Lane wrote: "Simon Riggs" <[EMAIL PROTECTED]> writes: On Tue, 2007-02-27 at 10:37 -0600, Jim C. Nasby wrote: ... The idea would be to give vacuum a target run time, and it would monitor how much time it had remaining, taking into account how long it should take to scan the indexes based on

Re: [HACKERS] autovacuum next steps, take 2

2007-02-27 Thread Matthew T. O'Connor
Jim C. Nasby wrote: On Tue, Feb 27, 2007 at 12:00:41AM -0300, Alvaro Herrera wrote: Jim C. Nasby wrote: The advantage to keying this to autovac_naptime is that it means we don't need another GUC, but after I suggested that before I realized that's probably not the best idea. For example, I've

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: "Matthew T. O'Connor" writes: I'm not sure what you are saying here, are you now saying that partial vacuum won't work for autovac? Or are you saying that saving state as Jim is describing above won't work? I'm saying that I don't li

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: "Jim C. Nasby" <[EMAIL PROTECTED]> writes: The proposal to save enough state to be able to resume a vacuum at pretty much any point in it's cycle might work; we'd have to benchmark it. With the default maintenance_work_mem of 128M it would mean writing out 64M of state every min

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Jim C. Nasby wrote: On Mon, Feb 26, 2007 at 10:18:36PM -0500, Matthew T. O'Connor wrote: Jim C. Nasby wrote: Here is a worst case example: A DB with 6 tables all of which are highly active and will need to be vacuumed constantly. While this is totally hypothetical, it is how I env

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: "Matthew T. O'Connor" writes: That does sounds simpler. Is chunk-at-a-time a realistic option for 8.3? It seems fairly trivial to me to have a scheme where you do one fill-workmem-and-scan-indexes cycle per invocation, and store the next-heap-page-to-scan in s

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: "Matthew T. O'Connor" writes: Tom Lane wrote: I'm inclined to propose an even simpler algorithm in which every worker acts alike; That is what I'm proposing except for one difference, when you catch up to an older worker, exit. No, that's a b

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: BTW, to what extent might this whole problem be simplified if we adopt chunk-at-a-time vacuuming (compare current discussion with Galy Lee)? If the unit of work has a reasonable upper bound regardless of table size, maybe the problem of big tables starving small ones goes away.

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: "Jim C. Nasby" <[EMAIL PROTECTED]> writes: The real problem is trying to set that up in such a fashion that keeps hot tables frequently vacuumed; Are we assuming that no single worker instance will vacuum a given table more than once? (That's not a necessary assumption, certai

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Jim C. Nasby wrote: On Mon, Feb 26, 2007 at 06:23:22PM -0500, Matthew T. O'Connor wrote: I'm not sure how pg_class.relpages is maintained but what happens to a bloated table? For example, a 100 row table that is constantly updated and hasn't been vacuumed in a while (say the

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: Matthew T. O'Connor wrote: I'm not sure it's a good idea to tie this to the vacuum cost delay settings either, so let me as you this, how is this better than just allowing the admin to set a

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Matthew T. O'Connor wrote: I'm not sure how pg_class.relpages is maintained but what happens to a bloated table? For example, a 100 row table that is constantly updated and hasn't been vacuumed in a while (say the admin disabled autovacuum for a while),

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Matthew T. O'Connor wrote: How can you determine what tables can be vacuumed within autovacuum_naptime? My assumption is that pg_class.relpages * vacuum_cost_page_miss * vacuum_cost_delay = time to vacuum This is of course not the reality, because the delay is no

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Jim C. Nasby wrote: That's why I'm thinking it would be best to keep the maximum size of stuff for the second worker small. It probably also makes sense to tie it to time and not size, since the key factor is that you want it to hit the high-update tables every X number of

Re: [HACKERS] SCMS question

2007-02-25 Thread Matthew D. Fuller
simplier conflicts to deal with. A VCS that makes frequent merges easy results in easier conflict handling, not by some magical auto-resolution, but just by letting you do it in ongoing regular and small bites. -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrato

Re: [HACKERS] [Monotone-devel] Re: SCMS question

2007-02-25 Thread Matthew D. Fuller
rate them out so you can know when you're arguing for/against $SYSTEM, and when you're arguing for/against $CLASS_OF_SYSTEMS. -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the

Re: [HACKERS] [Monotone-devel] Re: SCMS question

2007-02-25 Thread Matthew D. Fuller
a year or two, when the performance failings are historical and bzr can be chosen 8-} -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream.

Re: [HACKERS] autovacuum next steps, take 2

2007-02-22 Thread Matthew T. O'Connor
Jim C. Nasby wrote: On Thu, Feb 22, 2007 at 09:32:57AM -0500, Matthew T. O'Connor wrote: So the heuristic would be: * Launcher fires off workers into a database at a given interval (perhaps configurable?) * Each worker works on tables in size order. * If a worker ever catches up

Re: [HACKERS] autovacuum next steps, take 2

2007-02-22 Thread Matthew T. O'Connor
Jim C. Nasby wrote: On Wed, Feb 21, 2007 at 05:40:53PM -0500, Matthew T. O'Connor wrote: My Proposal: If we require admins to identify hot tables tables, then: 1) Launcher fires-off a worker1 into database X. 2) worker1 deals with "hot" tables first, then regular tabl

Re: [HACKERS] autovacuum next steps, take 2

2007-02-21 Thread Matthew T. O'Connor
e this is probably a good idea in the long run, but I agree this is lower on the priority list and should come next. Greg Stark and Matthew O'Connor say that we're misdirected in having more than one worker per tablespace. I say we're not :-) If we consider Ron Mayer's

Re: [HACKERS] GiST Comparing IndexTuples/Datums

2007-02-17 Thread Matthew Campbell
Good news: I think we've got GiST working (somewhat anyways), as we found gistKeyIsEQ(giststate, 0, datum, currdatum) in gistutil.c does the trick of comparing two datums. I swear most of our trouble is just finding our way around the postgres codebase, but we're getting there little by little

Re: [HACKERS] autovacuum next steps

2007-02-16 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Matthew T. O'Connor wrote: On first blush, I'm not sure I like this as it doesn't directly attack the table starvation problem, and I think it could be a net loss of speed. VACUUM is I/O bound, as such, just sending multiple vacuum commands at a DB isn&

Re: [HACKERS] autovacuum next steps

2007-02-16 Thread Matthew T. O'Connor
Alvaro Herrera wrote: After staring at my previous notes for autovac scheduling, it has become clear that this basics of it is not really going to work as specified. So here is a more realistic plan: [Snip Detailed Description] How does this sound? On first blush, I'm not sure I like this a

[HACKERS] GiST Comparing IndexTuples/Datums

2007-02-12 Thread Matthew Campbell
Hey folks: I posted this to the pgsql-novice mailing list but was told that it'd probably be better to repost here. I've been working with a group trying to implement UNIQUE index functionality in GiST (we started with hash, and have branched out to try and understand some of the other indexi

Re: [HACKERS] autovacuum process handling

2007-01-22 Thread Matthew T. O'Connor
Alvaro Herrera wrote: This is how I think autovacuum should change with an eye towards being able to run multiple vacuums simultaneously: [snip details] Does this raise some red flags? It seems straightforward enough to me; I'll submit a patch implementing this, so that scheduling will conti

Re: [HACKERS] Autovacuum improvements

2007-01-16 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Matthew T. O'Connor wrote: This still seems ambiguous to me, how would I handle a maintenance window of Weekends from Friday at 8PM though Monday morning at 6AM? My guess from what said is: mon dom dow starttime endtime null null6 20:00 null null

  1   2   3   4   5   >