Re: [GENERAL] A query planner that learns

2006-10-17 Thread Rafal Pietrak
On Tue, 2006-10-17 at 10:24 -0400, Madison Kelly wrote: > Nor am I a lawyer, but I still hold that hoping "ignorance" will be a > decent defense is very, very risky. In the end I am not a pgSQL > developer so it isn't in my hands either way. If I may. The "hoping, ignorance will save you" line

Re: [GENERAL] A query planner that learns

2006-10-17 Thread AgentM
On Oct 17, 2006, at 10:46 , Madison Kelly wrote: Brian Mathis wrote: I also am NAL, but I know enough about the patent system (in the US) to know that ignorance *IS* a defense. If you are ignorant of the patent, you only have to pay the damages. If you knew about the patent and did it a

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Madison Kelly
Alvaro Herrera wrote: Yeah. I invite you to do all the extra (useless) development work required. But please do not charge other people with it. Whoever investigates patents and lets pgsql-hackers know about them, is charging the Postgres community with that work. We sure don't need it. As

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Alvaro Herrera
Madison Kelly wrote: > Brian Mathis wrote: > >I also am NAL, but I know enough about the patent system (in the US) to > >know that ignorance *IS* a defense. If you are ignorant of the patent, > >you only have to pay the damages. If you knew about the patent and did > >it anyway, you have to pa

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Scott Ribe
> Colour me funny, but wouldn't staying out of the courts in the first > place not be the best option? Yes, however some people feel that given the way the patent office is spewing huge quantities of patents, many on old well-known techniques, and the the absurd difficulty of reading patent claims

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Brian Mathis
On 10/17/06, Madison Kelly <[EMAIL PROTECTED]> wrote: Brian Mathis wrote:> I also am NAL, but I know enough about the patent system (in the US) to> know that ignorance *IS* a defense.  If you are ignorant of the patent,> you only have to pay the damages.  If you knew about the patent and did > it a

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Madison Kelly
Brian Mathis wrote: I also am NAL, but I know enough about the patent system (in the US) to know that ignorance *IS* a defense. If you are ignorant of the patent, you only have to pay the damages. If you knew about the patent and did it anyway, you have to pay *triple* damages. Ignorance wil

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Brian Mathis
On 10/17/06, Madison Kelly <[EMAIL PROTECTED]> wrote: AgentM wrote:> Alvaro's advice is sound. If the patent holder can prove that a> developer looked at a patent (for example, from an email in a mailing> list archive) and the project proceeded with the implementation > regardless, malice can been

Re: [GENERAL] A query planner that learns

2006-10-17 Thread Madison Kelly
AgentM wrote: Alvaro's advice is sound. If the patent holder can prove that a developer looked at a patent (for example, from an email in a mailing list archive) and the project proceeded with the implementation regardless, malice can been shown and "damages" can be substantially higher. You'r

Re: [GENERAL] A query planner that learns

2006-10-17 Thread AgentM
On Oct 16, 2006, at 16:17 , Madison Kelly wrote: Alvaro Herrera wrote: Jochem van Dieten wrote: Scott Marlowe wrote: While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Madison Kelly
Alvaro Herrera wrote: Hasn't IBM release a pile of it's patents for use (or at least stated they won't sue) to OSS projects? If so, is this patent covered by that "amnesty"? This is useless as a policy, because we have plenty of companies basing their proprietary code on PostgreSQL, which woul

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Alvaro Herrera
Madison Kelly wrote: > Alvaro Herrera wrote: > >Jochem van Dieten wrote: > >>Scott Marlowe wrote: > >>>While all the talk of a hinting system over in hackers and perform is > >>>good, and I have a few queries that could live with a simple hint system > >>>pop up now and again, I keep thinking that

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Madison Kelly
Alvaro Herrera wrote: Jochem van Dieten wrote: Scott Marlowe wrote: While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep thinking that a query planner that learns >from its mi

Re: [GENERAL] A query planner that learns

2006-10-16 Thread John D. Burger
Jochem van Dieten wrote: I think you might want to check US Patent 6,763,359 before you start writing any code. http://tinyurl.com/yzjdve - John D. Burger MITRE ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ?

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Alvaro Herrera
Jochem van Dieten wrote: > Scott Marlowe wrote: > >While all the talk of a hinting system over in hackers and perform is > >good, and I have a few queries that could live with a simple hint system > >pop up now and again, I keep thinking that a query planner that learns > >from its mistakes over ti

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Jochem van Dieten
Scott Marlowe wrote: While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep thinking that a query planner that learns from its mistakes over time is far more desirable. Is it reas

Re: [GENERAL] A query planner that learns

2006-10-16 Thread Alban Hertroys
Jim C. Nasby wrote: On Thu, Oct 12, 2006 at 05:39:20PM -0500, Scott Marlowe wrote: It may well be that by first looking at the data collected from problems queries, the solution for how to adjust the planner becomes more obvious. Yeah, that would be useful to have. The problem I see is storing

Re: [GENERAL] A query planner that learns

2006-10-14 Thread Ron Johnson
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/13/06 10:47, John D. Burger wrote: > Erik Jones wrote: > [snip] > But with both approaches, the planner is just using the static > statistics gathered by ANALYZE to estimate the cost of each candidate > plan, and these statistics are based on sa

Re: [GENERAL] A query planner that learns

2006-10-14 Thread Guy Rouillier
Jim C. Nasby wrote: > Just recording the query plan and actual vs estimated rowcounts would > be a good start, though. And useful to DBA's, provided you had some > means to query against it. If the DBMS stores the incoming query (or some parsed version of it) and identifies frequency of usage

Re: [GENERAL] A query planner that learns

2006-10-13 Thread Scott Marlowe
On Fri, 2006-10-13 at 12:48, Jim C. Nasby wrote: > On Thu, Oct 12, 2006 at 05:39:20PM -0500, Scott Marlowe wrote: > > > > It seems to me the first logical step would be having the ability to > > > > flip a switch and when the postmaster hits a slow query, it saves both > > > > the query that ran lo

Re: [GENERAL] A query planner that learns

2006-10-13 Thread Jim C. Nasby
On Fri, Oct 13, 2006 at 11:53:15AM -0400, AgentM wrote: > One simple first step would be to run an ANALYZE whenever a > sequential scan is executed. Is there a reason not to do this? It Yes. You want a seqscan on a small (couple pages) table, and ANALYZE has a very high overhead on some platfo

Re: [GENERAL] A query planner that learns

2006-10-13 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 05:39:20PM -0500, Scott Marlowe wrote: > > > It seems to me the first logical step would be having the ability to > > > flip a switch and when the postmaster hits a slow query, it saves both > > > the query that ran long, as well as the output of explain or explain > > > ana

Re: [GENERAL] A query planner that learns

2006-10-13 Thread AgentM
On Oct 13, 2006, at 11:47 , John D. Burger wrote: Erik Jones wrote: Forgive me if I'm way off here as I'm not all that familiar with the internals of postgres, but isn't this what the genetic query optimizer discussed the one of the manual's appendixes is supposed to do. No - it's not

Re: [GENERAL] A query planner that learns

2006-10-13 Thread John D. Burger
Erik Jones wrote: Forgive me if I'm way off here as I'm not all that familiar with the internals of postgres, but isn't this what the genetic query optimizer discussed the one of the manual's appendixes is supposed to do. No - it's not an "optimizer" in that sense. When there are a small

Re: [GENERAL] A query planner that learns

2006-10-13 Thread Erik Jones
Forgive me if I'm way off here as I'm not all that familiar with the internals of postgres, but isn't this what the genetic query optimizer discussed the one of the manual's appendixes is supposed to do. Or, is that more about using the the known costs of atomic operations that make up a query

Re: [GENERAL] A query planner that learns

2006-10-12 Thread Scott Marlowe
On Thu, 2006-10-12 at 17:14, Jim C. Nasby wrote: > On Thu, Oct 12, 2006 at 03:31:50PM -0500, Scott Marlowe wrote: > > While all the talk of a hinting system over in hackers and perform is > > good, and I have a few queries that could live with a simple hint system > > pop up now and again, I keep t

Re: [GENERAL] A query planner that learns

2006-10-12 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 03:31:50PM -0500, Scott Marlowe wrote: > While all the talk of a hinting system over in hackers and perform is > good, and I have a few queries that could live with a simple hint system > pop up now and again, I keep thinking that a query planner that learns > from its mista

[GENERAL] A query planner that learns

2006-10-12 Thread Scott Marlowe
While all the talk of a hinting system over in hackers and perform is good, and I have a few queries that could live with a simple hint system pop up now and again, I keep thinking that a query planner that learns from its mistakes over time is far more desirable. Is it reasonable or possible for