Re: TupleTableSlot abstraction

2019-02-26 Thread Andres Freund
On 2019-02-27 15:42:50 +0900, Michael Paquier wrote: > On Tue, Feb 26, 2019 at 10:38:45PM -0800, Andres Freund wrote: > > Im not sure I understand. How can adding a memory context + reset to > > ctas and matview receivers negatively impact other dest receivers? > > I don't think you got my point h

Re: TupleTableSlot abstraction

2019-02-26 Thread Michael Paquier
On Tue, Feb 26, 2019 at 10:38:45PM -0800, Andres Freund wrote: > Im not sure I understand. How can adding a memory context + reset to > ctas and matview receivers negatively impact other dest receivers? I don't think you got my point here: imagine that a plugin use the current receiveSlot logic fr

Re: TupleTableSlot abstraction

2019-02-26 Thread Andres Freund
Hi, On 2019-02-27 15:34:07 +0900, Michael Paquier wrote: > On Tue, Feb 26, 2019 at 09:42:38PM -0800, Andres Freund wrote: > > I'm not so sure that's the architecturally correct fix however. Is it > > actually guaranteed, given expanded tuples, toasting, etc, that there's > > no other memory leak h

Re: TupleTableSlot abstraction

2019-02-26 Thread Michael Paquier
On Tue, Feb 26, 2019 at 09:42:38PM -0800, Andres Freund wrote: > I'm not so sure that's the architecturally correct fix however. Is it > actually guaranteed, given expanded tuples, toasting, etc, that there's > no other memory leak here? I wonder if we shouldn't work twoards using a > short lived m

Re: TupleTableSlot abstraction

2019-02-26 Thread Andres Freund
Hi, On 2019-02-27 14:21:52 +0900, Michael Paquier wrote: > On Sat, Feb 16, 2019 at 05:07:44PM -0500, Jeff Janes wrote: > > By blind analogy to the changes made to matview.c, I think that createas.c > > is missing a heap_freetuple, as attached. First, sorry to have been slow answering. I was whack

Re: TupleTableSlot abstraction

2019-02-26 Thread Michael Paquier
On Sat, Feb 16, 2019 at 05:07:44PM -0500, Jeff Janes wrote: > By blind analogy to the changes made to matview.c, I think that createas.c > is missing a heap_freetuple, as attached. I think that you are right. CTAS and materialized views share a lot when it comes to relation creation and initial t

Re: TupleTableSlot abstraction

2019-02-16 Thread Jeff Janes
On Fri, Nov 16, 2018 at 7:46 PM Andres Freund wrote: > Hi, > > On 2018-11-13 15:30:21 -0800, Andres Freund wrote: > > What I'm now planning to do is to go through the big comment in > > tuptable.h and update that to the new world. While I'm not convinced > > that that that's the best place for i

Re: TupleTableSlot abstraction

2018-11-19 Thread Amit Khandekar
On Wed, 14 Nov 2018 at 05:00, Andres Freund wrote: > After this, I hope Amit Khandekar will rebase a patch he's sent me > internally that converts triggers to use slots. I'll work on rebasing > the pluggable storage patch ontop of this. Shared this patch in a separate mail thread : https://www.po

Re: TupleTableSlot abstraction

2018-11-16 Thread Andres Freund
Hi, On 2018-11-13 15:30:21 -0800, Andres Freund wrote: > What I'm now planning to do is to go through the big comment in > tuptable.h and update that to the new world. While I'm not convinced > that that that's the best place for it, it needs to be accurate. > > Furthermore: > - More comment pol

Re: TupleTableSlot abstraction

2018-10-16 Thread Andres Freund
Hi, On 2018-10-15 12:12:03 +0530, Amit Khandekar wrote: > On Sat, 13 Oct 2018 at 04:02, Andres Freund wrote: > > I'd just have ExecInterpExpr() continue calling ExecEvalSysVar. There's > > no reason for it to be inline. > Can you explain more why you think there should be a ExecEvalSysVar() > de

Re: TupleTableSlot abstraction

2018-10-16 Thread Amit Khandekar
On Tue, 9 Oct 2018 at 20:46, Amit Khandekar wrote: > There is still one more regression test failure in polygon.sql which I > am yet to analyze. Below is a narrowed down testcase which reproduces the failure in polygon.sql : create table tab (n int, dist int, id integer); insert into tab values

Re: TupleTableSlot abstraction

2018-10-14 Thread Amit Khandekar
On Sat, 13 Oct 2018 at 04:02, Andres Freund wrote: > > > > @@ -2024,7 +2024,18 @@ FormIndexDatum(IndexInfo *indexInfo, > > > > Datum iDatum; > > > > boolisNull; > > > > > > > > - if (keycol != 0) > > > > + if (keycol < 0) >

Re: TupleTableSlot abstraction

2018-10-12 Thread Andres Freund
On 2018-10-09 20:46:04 +0530, Amit Khandekar wrote: > On Wed, 26 Sep 2018 at 05:35, Andres Freund wrote: > > > + > > > +/* > > > + * This is a function used by all getattr() callbacks which deal with a > > > heap > > > + * tuple or some tuple format which can be represented as a heap tuple > > >

Re: TupleTableSlot abstraction

2018-10-11 Thread Andres Freund
Hi, On 2018-10-01 22:21:58 -0400, Tom Lane wrote: > Kyotaro HORIGUCHI writes: > > At Tue, 25 Sep 2018 16:45:09 -0700, Andres Freund > > wrote in <20180925234509.3hrrf6tmvy5tf...@alap3.anarazel.de> > >> On 2018-09-04 18:35:34 +0530, Amit Khandekar wrote: > >>> Pack the boolean members in TupleTa

Re: TupleTableSlot abstraction

2018-10-10 Thread Amit Khandekar
On Wed, 26 Sep 2018 at 05:15, Andres Freund wrote: > > Hi, > > On 2018-09-04 18:35:34 +0530, Amit Khandekar wrote: > > Subject: [PATCH 05/14] Use tts_flags instead of previous bool members > > > > Pack the boolean members in TupleTableSlot into a 16 bit tts_flags. > > This reduces the size of Tupl

Re: TupleTableSlot abstraction

2018-10-09 Thread Amit Khandekar
On Wed, 26 Sep 2018 at 05:35, Andres Freund wrote: > > + > > +/* > > + * This is a function used by all getattr() callbacks which deal with a > > heap > > + * tuple or some tuple format which can be represented as a heap tuple > > e.g. a > > + * minimal tuple. > > + * > > + * heap_getattr consid

Re: TupleTableSlot abstraction

2018-10-05 Thread Amit Khandekar
On Thu, 4 Oct 2018 at 22:59, Amit Khandekar wrote: > > I have only done the below two changes yet. After doing that and > rebasing with latest master, in the regression I got crashes, and I > suspect the reason being that I have used Virtual tuple slot for the > destination slot of execute_attr_ma

Re: TupleTableSlot abstraction

2018-10-04 Thread Amit Khandekar
I have only done the below two changes yet. After doing that and rebasing with latest master, in the regression I got crashes, and I suspect the reason being that I have used Virtual tuple slot for the destination slot of execute_attr_map_slot(). I am analyzing it. I am anyway attaching the patches

Re: TupleTableSlot abstraction

2018-10-01 Thread Tom Lane
Kyotaro HORIGUCHI writes: > At Tue, 25 Sep 2018 16:45:09 -0700, Andres Freund wrote > in <20180925234509.3hrrf6tmvy5tf...@alap3.anarazel.de> >> On 2018-09-04 18:35:34 +0530, Amit Khandekar wrote: >>> Pack the boolean members in TupleTableSlot into a 16 bit tts_flags. >>> This reduces the size of

Re: TupleTableSlot abstraction

2018-10-01 Thread Kyotaro HORIGUCHI
At Tue, 25 Sep 2018 16:45:09 -0700, Andres Freund wrote in <20180925234509.3hrrf6tmvy5tf...@alap3.anarazel.de> > Hi, > > On 2018-09-04 18:35:34 +0530, Amit Khandekar wrote: > > Subject: [PATCH 05/14] Use tts_flags instead of previous bool members > > > > Pack the boolean members in TupleTableSl

Re: TupleTableSlot abstraction

2018-09-25 Thread Andres Freund
On 2018-09-04 18:35:34 +0530, Amit Khandekar wrote: > The attached v11 tar has the above set of changes. > Subject: [PATCH 07/14] Restructure TupleTableSlot to allow tuples other than > HeapTuple > + > +/* > + * This is a function used by all getattr() callbacks which deal with a heap > + * tup

Re: TupleTableSlot abstraction

2018-09-25 Thread Andres Freund
Hi, On 2018-09-04 18:35:34 +0530, Amit Khandekar wrote: > Subject: [PATCH 05/14] Use tts_flags instead of previous bool members > > Pack the boolean members in TupleTableSlot into a 16 bit tts_flags. > This reduces the size of TupleTableSlot since each bool member takes > at least a byte on the p

Re: TupleTableSlot abstraction

2018-09-25 Thread Andres Freund
On 2018-09-04 18:35:34 +0530, Amit Khandekar wrote: > The attached v11 tar has the above set of changes. - I've pushed 0003 - although that commit seems to have included a lot of things unrelated to the commit message. I guess two patches have accidentally been merged? Could you split out the

Re: TupleTableSlot abstraction

2018-09-04 Thread Amit Khandekar
On 28 August 2018 at 22:43, Ashutosh Bapat wrote: > On Fri, Aug 24, 2018 at 6:46 AM, Andres Freund wrote: > >> >>> @@ -2883,7 +2885,7 @@ CopyFrom(CopyState cstate) >>> if (slot == NULL) /* "do nothing" */ >>> skip_tuple = true; >>>

Re: TupleTableSlot abstraction

2018-08-31 Thread Andres Freund
Hi, On 2018-08-31 10:05:05 +0530, Amit Khandekar wrote: > On 28 August 2018 at 22:43, Ashutosh Bapat > >> I think I was wrong at saying that we should remove this. I think you > >> were right that it should become a callback... > > > We have replaced all slot_getsysattrs() with heap_getsysattr().

Re: TupleTableSlot abstraction

2018-08-30 Thread Amit Khandekar
On 28 August 2018 at 22:43, Ashutosh Bapat wrote: > On Fri, Aug 24, 2018 at 6:46 AM, Andres Freund wrote: >> >>> -/* >>> - * slot_getsysattr >>> - * This function fetches a system attribute of the slot's >>> current tuple. >>> - * Unlike slot_getattr, if the slot does not con

Re: TupleTableSlot abstraction

2018-08-30 Thread Alvaro Herrera
Man, how I dislike patches in tarballs. 0002 says: + * shouldFree is set 'true' since a tuple stored on a disk page should not be + * pfree'd. Surely you mean 'false' :-) -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: TupleTableSlot abstraction

2018-08-28 Thread Ashutosh Bapat
missattnum++) >> - { >> - slot->tts_values[missattnum] = >> attrmiss[missattnum].am_value; >> - slot->tts_isnull[missattnum] = >> !attrmiss[missattnum].am_present; >> - } >> - } >>

Re: TupleTableSlot abstraction

2018-08-23 Thread Andres Freund
Hi, On 2018-08-20 19:51:33 +0530, Ashutosh Bapat wrote: > Sorry, forgot about that. Here's the patch set with that addressed. Btw, you attach files as tar.zip, but they're actually gzip compressed... > From 838a463646a048b3dccff95079a514fdc86effb3 Mon Sep 17 00:00:00 2001 > From: Ashutosh Bapat

Re: TupleTableSlot abstraction

2018-08-20 Thread Andres Freund
On 2018-08-20 17:51:38 +0530, Ashutosh Bapat wrote: > > I also noticed an independent issue in your changes to > > ExecInitScanTupleSlot(): You can't assume that the plan belonging to the > > ScanState have a Scan node in their plan. Look e.g. at Material, Sort > > etc. So currently your scanrelid

Re: TupleTableSlot abstraction

2018-08-18 Thread Andres Freund
On 2018-08-17 01:07:06 -0700, Andres Freund wrote: > Hi, > > On 2018-08-17 12:10:20 +0530, Ashutosh Bapat wrote: > > We need to add LLVM code to fetch tts_flags and > > perform bit operation on it to get or set slow property. I haven't > > found any precedence for LLVM bit operations in postgresql

Re: TupleTableSlot abstraction

2018-08-17 Thread Andres Freund
Hi, On 2018-08-17 12:10:20 +0530, Ashutosh Bapat wrote: > We need to add LLVM code to fetch tts_flags and > perform bit operation on it to get or set slow property. I haven't > found any precedence for LLVM bit operations in postgresql's JIT code. There are several, look for the infomask accesses

Re: TupleTableSlot abstraction

2018-08-09 Thread Ashutosh Bapat
On Wed, Aug 8, 2018 at 5:07 PM, Ashutosh Bapat wrote: Amit Khandekar offlist told me that the previous patch-set doesn't apply cleanly on the latest head. PFA patches rebased on 31380bc7c204e7cfa9c9e1c62947909e2b38577c > > 3. compile with LLVM and fix any compilation and regression erro

Re: TupleTableSlot abstraction

2018-08-06 Thread Ashutosh Bapat
On Mon, Aug 6, 2018 at 10:15 AM, Andres Freund wrote: > Hi, > > On 2018-08-06 10:08:24 +0530, Ashutosh Bapat wrote: >> I think, I explained why getattr() needs to be a separate callback. >> There's a reason why slot_getattr() more code than just calling >> slot_getsomeattrs() and return the requir

Re: TupleTableSlot abstraction

2018-08-05 Thread Andres Freund
Hi, On 2018-08-06 10:08:24 +0530, Ashutosh Bapat wrote: > I think, I explained why getattr() needs to be a separate callback. > There's a reason why slot_getattr() more code than just calling > slot_getsomeattrs() and return the required one - the cases when the > requested attribute is NULL or is

Re: TupleTableSlot abstraction

2018-08-05 Thread Ashutosh Bapat
On Sun, Aug 5, 2018 at 3:49 PM, Andres Freund wrote: > Hi, > > Working on rebasing the pluggable storage patch on the current version > of this. Thanks. Please let me know if you see any issues. > > On 2018-07-26 17:09:08 +0530, Ashutosh Bapat wrote: >> Done. I also noticed that slot_getattr() o

Re: TupleTableSlot abstraction

2018-08-05 Thread Andres Freund
Hi, Working on rebasing the pluggable storage patch on the current version of this. On 2018-07-26 17:09:08 +0530, Ashutosh Bapat wrote: > Done. I also noticed that slot_getattr() optimizes the cases when the > requested attributes is NULL or is missing from a tuple. Given that a > custom TupleTab

Re: TupleTableSlot abstraction

2018-07-05 Thread Ashutosh Bapat
On Mon, Jun 11, 2018 at 6:13 PM, Ashutosh Bapat wrote: > >> >> The slightly bigger issue is that several parts of the code currently >> require heaptuples they can scribble upon. E.g. nodeModifyTable.c >> materializes the tuple - those can be solved by using a local slot to >> store the tuple, rat

Re: TupleTableSlot abstraction

2018-03-13 Thread Andres Freund
On 2018-03-13 15:18:34 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Hi, > > > > I've recently been discussing with Robert how to abstract > > TupleTableSlots in the light of upcoming developments around abstracting > > storage away. Besides that aspect I think we also need to make them

Re: TupleTableSlot abstraction

2018-03-13 Thread Alvaro Herrera
Andres Freund wrote: > Hi, > > I've recently been discussing with Robert how to abstract > TupleTableSlots in the light of upcoming developments around abstracting > storage away. Besides that aspect I think we also need to make them > more abstract to later deal with vectorized excution, but I'm

Re: TupleTableSlot abstraction

2018-02-25 Thread Alexander Korotkov
Hi! Thank you for working on this subject. See some my comments below. On Wed, Feb 21, 2018 at 1:43 AM, Andres Freund wrote: > - TupleTableSlots have to contain all the necessary information for each > type of slot. Some implementations might require a buffer pin to be > hold (our heap), o