Re: Having query cache in core

2018-05-11 Thread Andres Freund
On 2018-05-12 08:20:13 +1000, CK Tan wrote: > On Sat, May 12, 2018 at 8:18 AM, Tatsuo Ishii wrote: > > > > > > > How do you handle tables hiding behind views? Also how does cached > > entries > > > in pgpools know if some tables are modified without going thru pgpool, eg > > > pgplsql or trigger

Re: Having query cache in core

2018-05-11 Thread CK Tan
On Sat, May 12, 2018 at 8:18 AM, Tatsuo Ishii wrote: > > > > How do you handle tables hiding behind views? Also how does cached > entries > > in pgpools know if some tables are modified without going thru pgpool, eg > > pgplsql or trigger or via psql directly? > > Pgpool-II do not invalidate cach

Re: Having query cache in core

2018-05-11 Thread Tatsuo Ishii
>> > I think you need to know which tables are involved and if they were >> > modified. >> >> Of course. While creating a cache entry for a SELECT, we need to >> analyze it and extract tables involved in the SELECT. The information >> should be stored along with the cache entry. If any of the table

Re: Having query cache in core

2018-05-11 Thread Tatsuo Ishii
> On 11.05.2018 18:01, Tatsuo Ishii wrote: >> Plus checking username is neccessary (otherwise any user could >> retrieve a cache for a table lookup which is not permitted by other >> users). > > as the tables a cached query operated on is known anyway -- it's > needed > to purge cache entries when

Re: Having query cache in core

2018-05-11 Thread Hartmut Holzgraefe
On 11.05.2018 18:01, Tatsuo Ishii wrote: Plus checking username is neccessary (otherwise any user could retrieve a cache for a table lookup which is not permitted by other users). as the tables a cached query operated on is known anyway -- it's needed to purge cache entries when table content c

Re: Having query cache in core

2018-05-11 Thread Tatsuo Ishii
> that's almost actually how the MySQL query cache works: the query > cache > lookup kicks in even before the query is parsed, to get maximum gain > from cache hits. > > It does not just take the query text into account alone though, > but also the current default database, protocol version, > an

Re: Having query cache in core

2018-05-11 Thread CK Tan
On Fri, May 11, 2018, 10:26 PM Tatsuo Ishii wrote: > > > > > I think you need to know which tables are involved and if they were > > modified. > > Of course. While creating a cache entry for a SELECT, we need to > analyze it and extract tables involved in the SELECT. The information > should be s

Re: Having query cache in core

2018-05-11 Thread Tatsuo Ishii
>> If any of the tables were modified, cache entries using the table must be > removed. >> (these are already implemented in Pgpool-II's in memory query cache) > > How do you identify updates made from a pl/pgsql procedure? Pgpool-II does not invalidate query cache in that case. I think in-core

Re: Having query cache in core

2018-05-11 Thread Hartmut Holzgraefe
On 11.05.2018 14:26, Tatsuo Ishii wrote: If any of the tables were modified, cache entries using the table must be removed. (these are already implemented in Pgpool-II's in memory query cache) ... and this is the expensive part in the MySQL implementation that has rendered the query cache mostl

Re: Having query cache in core

2018-05-11 Thread Vladimir Sitnikov
> If any of the tables were modified, cache entries using the table must be removed. > (these are already implemented in Pgpool-II's in memory query cache) How do you identify updates made from a pl/pgsql procedure? Vladimir

Re: Having query cache in core

2018-05-11 Thread Tatsuo Ishii
>> Thanks for the advice. But I rather thought about bypassing the raw >> parser and the planner. i.e. use the query string (or its hash) as the >> index of the query cache. >> > > I think you need to know which tables are involved and if they were > modified. Of course. While creating a cache en

Re: Having query cache in core

2018-05-11 Thread Hartmut Holzgraefe
On 11.05.2018 11:12, Tatsuo Ishii wrote: Thanks for the advice. But I rather thought about bypassing the raw parser and the planner. i.e. use the query string (or its hash) as the index of the query cache. that's almost actually how the MySQL query cache works: the query cache lookup kicks in e

Re: Having query cache in core

2018-05-11 Thread CK Tan
On Fri, May 11, 2018, 7:13 PM Tatsuo Ishii wrote: > > You could probably write an extension for that, though. I think the > > planner hook and custom scans give you enough flexibility to do that > > without modifying the server code. > > Thanks for the advice. But I rather thought about bypassing

Re: Having query cache in core

2018-05-11 Thread Tatsuo Ishii
> You could probably write an extension for that, though. I think the > planner hook and custom scans give you enough flexibility to do that > without modifying the server code. Thanks for the advice. But I rather thought about bypassing the raw parser and the planner. i.e. use the query string (o

RE: Having query cache in core

2018-05-08 Thread Tsunakawa, Takayuki
From: Robert Haas [mailto:robertmh...@gmail.com] > That's not my experience. I agree that plan caching isn't important > for long-running queries, but I think it *is* potentially important > for fast queries with fast planning time. Even when the planning time > is fast, it can be a significant p

Re: Having query cache in core

2018-05-08 Thread Michael Paquier
On Wed, May 09, 2018 at 09:04:04AM +0900, Tatsuo Ishii wrote: > I have simulated the idea and I wonder how to implement the query > result cache on the streaming standby servers because no DML/DDL are > executed on standby servers, that makes it impossible to invalidate > the query cache. Probably

Re: Having query cache in core

2018-05-08 Thread Tatsuo Ishii
> On 07/05/18 05:47, Tom Lane wrote: >> Tatsuo Ishii writes: >>> Does anybody think having in-memory query result cache in core is a >>> good idea? >> No. > > Agreed. > > You could probably write an extension for that, though. I think the > planner hook and custom scans give you enough flexibili

Re: Having query cache in core

2018-05-08 Thread Robert Haas
On Mon, May 7, 2018 at 2:32 PM, Pavel Stehule wrote: > For interactive application only for one subset of queries the plan cache is > interesting. > > @1 There are long queries - the planning time is not significant (although > can be high), and then plan cache is not important > @2 there are fast

Re: Having query cache in core

2018-05-07 Thread Pavel Stehule
2018-05-07 19:52 GMT+02:00 Robert Haas : > On Sun, May 6, 2018 at 10:32 PM, Tatsuo Ishii wrote: > > Does anybody think having in-memory query result cache in core is a > > good idea? From the experience of implementing the feature in > > Pgpool-II, I would think this is not terribly hard job. But

Re: Having query cache in core

2018-05-07 Thread Andres Freund
On 2018-05-07 13:52:26 -0400, Robert Haas wrote: > On Sun, May 6, 2018 at 10:32 PM, Tatsuo Ishii wrote: > > Does anybody think having in-memory query result cache in core is a > > good idea? From the experience of implementing the feature in > > Pgpool-II, I would think this is not terribly hard j

Re: Having query cache in core

2018-05-07 Thread Robert Haas
On Sun, May 6, 2018 at 10:32 PM, Tatsuo Ishii wrote: > Does anybody think having in-memory query result cache in core is a > good idea? From the experience of implementing the feature in > Pgpool-II, I would think this is not terribly hard job. But first of > all I'm wondering if there's a demand

Re: Having query cache in core

2018-05-07 Thread Konstantin Knizhnik
On 07.05.2018 11:58, Tatsuo Ishii wrote: On 07.05.2018 05:32, Tatsuo Ishii wrote: Does anybody think having in-memory query result cache in core is a good idea? From the experience of implementing the feature in Pgpool-II, I would think this is not terribly hard job. But first of all I'm wonde

Re: Having query cache in core

2018-05-07 Thread Tatsuo Ishii
> On 07.05.2018 05:32, Tatsuo Ishii wrote: >> Does anybody think having in-memory query result cache in core is a >> good idea? From the experience of implementing the feature in >> Pgpool-II, I would think this is not terribly hard job. But first of >> all I'm wondering if there's a demand for the

Re: Having query cache in core

2018-05-07 Thread Konstantin Knizhnik
On 07.05.2018 11:24, Tsunakawa, Takayuki wrote: From: Konstantin Knizhnik [mailto:k.knizh...@postgrespro.ru] But I think it is better to start first with 1. Global prepared statements cache 2. Global catalog cache 3. Global relation cache May I ask why prepared statements need to precede cata

Re: Having query cache in core

2018-05-07 Thread Hartmut Holzgraefe
On 07.05.2018 10:12, Konstantin Knizhnik wrote: Concerning result cache, I think it will be better to ask opinion of mysql users: how useful it is. It isn't useful. I haven't seen a customer case in years where the query cache would have done any good. It is off by default ever since MySQL 5

RE: Having query cache in core

2018-05-07 Thread Tsunakawa, Takayuki
From: Konstantin Knizhnik [mailto:k.knizh...@postgrespro.ru] > But I think it is better to start first with > 1. Global prepared statements cache > 2. Global catalog cache > 3. Global relation cache May I ask why prepared statements need to precede catalog and relation caches? We're suffering fr

Re: Having query cache in core

2018-05-07 Thread Konstantin Knizhnik
On 07.05.2018 05:32, Tatsuo Ishii wrote: Does anybody think having in-memory query result cache in core is a good idea? From the experience of implementing the feature in Pgpool-II, I would think this is not terribly hard job. But first of all I'm wondering if there's a demand for the feature.

Re: Having query cache in core

2018-05-07 Thread Hartmut Holzgraefe
On 07.05.2018 08:23, Laurenz Albe wrote: Having been bitten by the feature on MySQL, I think it's not a good thing. Essentially, it's a band-aid for badly written applications, but it will only help in certain cases and hurts in many others. The MySQL query cache helped quite a bit in the earl

Re: Having query cache in core

2018-05-07 Thread Sergei Kornilov
> Having been bitten by the feature on MySQL, I think it's not a good thing. Even in MySQL itself this feature was already removed.

Re: Having query cache in core

2018-05-07 Thread Heikki Linnakangas
On 07/05/18 05:47, Tom Lane wrote: Tatsuo Ishii writes: Does anybody think having in-memory query result cache in core is a good idea? No. Agreed. You could probably write an extension for that, though. I think the planner hook and custom scans give you enough flexibility to do that with

Re: Having query cache in core

2018-05-06 Thread Laurenz Albe
Tom Lane wrote: > Tatsuo Ishii writes: > > Does anybody think having in-memory query result cache in core is a > > good idea? > > No. I agree. Having been bitten by the feature on MySQL, I think it's not a good thing. Essentially, it's a band-aid for badly written applications, but it will onl

Re: Having query cache in core

2018-05-06 Thread Tom Lane
Tatsuo Ishii writes: > Does anybody think having in-memory query result cache in core is a > good idea? No. regards, tom lane