Allow to collect statistics on virtual generated columns

2025-04-22 Thread Yugo Nagata
statistics on virtual generated columns ANALYZE expands virtual generated columns' expression, and collects statistics on evaluated values. In this approach, the statistics on virtual generated columns are collected in default, but ANALYZE on the table would become a bit expensive. (2) All

Re: Virtual generated columns

2025-02-24 Thread Richard Guo
On Mon, Feb 24, 2025 at 9:07 PM Dean Rasheed wrote: > On Mon, 24 Feb 2025 at 09:21, Richard Guo wrote: > > Here are the updated patches with revised comments and some tweaks to > > the commit messages. I plan to push them in one or two days. > LGTM. Pushed. Thanks all for working on this issu

Re: Virtual generated columns

2025-02-24 Thread Dean Rasheed
On Mon, 24 Feb 2025 at 09:21, Richard Guo wrote: > > Here are the updated patches with revised comments and some tweaks to > the commit messages. I plan to push them in one or two days. > LGTM. Regards, Dean

Re: Virtual generated columns

2025-02-24 Thread Richard Guo
though, so maybe we're good > > here. Still, I'll fix this later. > > > i also noticed this issue... > > some minor comments about v7. Thanks for reviewing. Here are the updated patches with revised comments and some tweaks to the commit messages. I plan to push

Re: Virtual generated columns

2025-02-23 Thread jian he
On Sat, Feb 22, 2025 at 11:12 PM Richard Guo wrote: > > On Sat, Feb 22, 2025 at 11:55 PM Richard Guo wrote: > > Attached are the updated patches to fix all the mentioned issues. I > > plan to push them early next week after staring at the code for a bit > > longer, barring any objections. > > Si

Re: Virtual generated columns

2025-02-22 Thread Richard Guo
On Sat, Feb 22, 2025 at 11:55 PM Richard Guo wrote: > Attached are the updated patches to fix all the mentioned issues. I > plan to push them early next week after staring at the code for a bit > longer, barring any objections. Sign... I neglected to make the change in 0001 that a Var newnode co

Re: Virtual generated columns

2025-02-22 Thread Richard Guo
On Sat, Feb 22, 2025 at 2:35 AM Dean Rasheed wrote: > On Fri, 21 Feb 2025 at 06:16, Richard Guo wrote: > > * The expansion of virtual generated columns occurs after subquery > > pullup, which can lead to issues. This was an oversight on my part. > > Initially, I believed i

Re: Virtual generated columns

2025-02-21 Thread Dean Rasheed
an Vars, considering dropped > columns. I think we need to check for this when generating the fields > for a RowExpr. Yes, good point. > * The expansion of virtual generated columns occurs after subquery > pullup, which can lead to issues. This was an oversight on my part. > Initial

Re: Virtual generated columns

2025-02-20 Thread Richard Guo
ces, it is possible that some fields are expanded as Consts rather than Vars, considering dropped columns. I think we need to check for this when generating the fields for a RowExpr. * The expansion of virtual generated columns occurs after subquery pullup, which can lead to issues. This was an

Re: Virtual generated columns

2025-02-20 Thread jian he
On Wed, Feb 19, 2025 at 11:25 PM Dean Rasheed wrote: > > One of the new regression tests fails, which actually appears to be a > pre-existing grouping sets bug, due to the fact that only non-Vars are > wrapped in PHVs. This bug can be triggered without virtual generated > colu

Re: Virtual generated columns

2025-02-19 Thread jian he
On Wed, Feb 19, 2025 at 11:25 PM Dean Rasheed wrote: > > On Wed, 19 Feb 2025 at 01:42, Dean Rasheed wrote: > > > > One thing I don't like about this is that it's introducing more code > > duplication between pullup_replace_vars() and > > ReplaceVarsFromTargetList(). Those already had a lot of cod

Re: Virtual generated columns

2025-02-19 Thread Dean Rasheed
wrapped in PHVs. This bug can be triggered without virtual generated columns: CREATE TABLE t (a int, b int); INSERT INTO t VALUES (1, 1); SELECT * FROM (SELECT a, a AS b FROM t) AS vt GROUP BY GROUPING SETS (a, b) HAVING b = 1; a | b ---+--- 1 | (1 row) whereas the result should be a | b

Re: Virtual generated columns

2025-02-18 Thread Dean Rasheed
On Tue, 18 Feb 2025 at 13:12, Richard Guo wrote: > > > It seems to me that, for a relation in the rangetable that has virtual > > generated columns, we can consider it a subquery to some extent. For > > instance, suppose we have a query: > > > > select ... fr

Re: Virtual generated columns

2025-02-18 Thread Richard Guo
On Tue, Feb 18, 2025 at 7:09 PM Richard Guo wrote: > It seems to me that, for a relation in the rangetable that has virtual > generated columns, we can consider it a subquery to some extent. For > instance, suppose we have a query: > > select ... from ... join t on ...; > >

Re: Virtual generated columns

2025-02-18 Thread Richard Guo
ed is a rough patch that moves the expansion of virtual > generated columns to the planner. It needs a lot more testing (and > some regression tests), but it does seem to fix all the issues > mentioned in this thread. Yeah, I believe this is the right way to go: virtual generated columns should be

Re: Virtual generated columns

2025-02-16 Thread jian he
ted column expansion in > > > the rewrite stage as is, > > > and wrap the expressions in PHVs if the virtual generated > > > columns come from the nullable side of an outer join. > > > > PlaceHolderVar looks like a fitting mechanism for this. But it's so far

Re: Virtual generated columns

2025-02-15 Thread Dean Rasheed
x27;s used. It seems pretty baked into how PHVs work that they should only be added by the planner, so I think that I agree with Richard -- virtual generated columns probably have to be expanded in the planner rather than the rewriter. > Maybe a short-term fix would be to error out if we find ou

Re: Virtual generated columns

2025-02-14 Thread Peter Eisentraut
On 13.02.25 14:06, jian he wrote: I didn't solve the out join semantic issue. i am wondering, can we do the virtual generated column expansion in the rewrite stage as is, and wrap the expressions in PHVs if the virtual generated columns come from the nullable side of an outer

Re: Virtual generated columns

2025-02-13 Thread jian he
; > NULL | NULL > > NULL | NULL > > (2 rows) > > Yeah, I also feel that the virtual generated columns should adhere to > outer join semantics, rather than being unconditionally replaced by > the generation expressions. But maybe I'm wrong. > > If that's the

Re: Virtual generated columns

2025-02-11 Thread Richard Guo
On Tue, Feb 11, 2025 at 10:34 AM Richard Guo wrote: > Yeah, I also feel that the virtual generated columns should adhere to > outer join semantics, rather than being unconditionally replaced by > the generation expressions. But maybe I'm wrong. > > If that's the case, th

Re: Virtual generated columns

2025-02-10 Thread Richard Guo
s. > I agree with Richard, the result seems incorrect. The right outcome should be: > gpadmin=# SELECT t2.a, t2.b FROM t t1 LEFT JOIN t t2 ON FALSE; > a | b > --+-- > NULL | NULL > NULL | NULL > (2 rows) Yeah, I also feel that the virtual generated columns should adher

Re: Virtual generated columns

2025-02-09 Thread Zhang Mingli
On Feb 10, 2025 at 12:53 +0800, jian he , wrote: > > please check attached. > > > BTW, I was curious about what happens if the replacement expression is > > constant, so I tried running the query below. > > > > CREATE TABLE t (a int, b int GENERATED ALWAYS AS (1 + 1)); > > INSERT INTO t VALUES (1);

Re: Virtual generated columns

2025-02-09 Thread jian he
On Mon, Feb 10, 2025 at 11:54 AM Richard Guo wrote: > > On Sun, Feb 9, 2025 at 7:02 PM Zhang Mingli wrote: > > On Feb 9, 2025 at 16:00 +0800, Alexander Lakhin , > > wrote: > > Please look at a planner error with a virtual generated column triggered > > by the following script: > > CREATE TABLE t

Re: Virtual generated columns

2025-02-09 Thread Richard Guo
On Sun, Feb 9, 2025 at 7:02 PM Zhang Mingli wrote: > On Feb 9, 2025 at 16:00 +0800, Alexander Lakhin , wrote: > Please look at a planner error with a virtual generated column triggered > by the following script: > CREATE TABLE t(a int, b int GENERATED ALWAYS AS (a * 1)); > > SELECT SUM(CASE WHEN t

Re: Virtual generated columns

2025-02-09 Thread Zhang Mingli
On Feb 9, 2025 at 16:00 +0800, Alexander Lakhin , wrote: > > Please look at a planner error with a virtual generated column triggered > by the following script: > CREATE TABLE t(a int, b int GENERATED ALWAYS AS (a * 1)); > > SELECT SUM(CASE WHEN t.b = 1 THEN 1 ELSE 1 END) OVER (PARTITION BY t.a) >

Re: Virtual generated columns

2025-02-09 Thread Alexander Lakhin
Hello Peter, 07.02.2025 14:34, Peter Eisentraut wrote: I've committed it.  Thanks. Please look at a planner error with a virtual generated column triggered by the following script: CREATE TABLE t(a int, b int GENERATED ALWAYS AS (a * 1)); SELECT SUM(CASE WHEN t.b = 1 THEN 1 ELSE 1 END) OVER (

Re: Virtual generated columns

2025-02-07 Thread Peter Eisentraut
On 06.02.25 14:03, vignesh C wrote: One suggestion: for the option where the user specifies publish_generated_columns as virtual (as shown below), could we change the error indicating that virtual generated columns are not currently supported? CREATE PUBLICATION pub1 FOR TABLE t1 WITH

Re: Virtual generated columns

2025-02-07 Thread Peter Eisentraut
On 06.02.25 00:25, Dean Rasheed wrote: On Tue, 4 Feb 2025 at 22:36, Peter Eisentraut wrote: Yeah, this is quite contorted. I have renamed it like you suggested. I looked over this again and I think the patch is in good shape to be committed. I've committed it. Thanks. One thought that

Re: Virtual generated columns

2025-02-06 Thread vignesh C
ri_extraUpdatedCols is populated. > > Yeah, this is quite contorted. I have renamed it like you suggested. One suggestion: for the option where the user specifies publish_generated_columns as virtual (as shown below), could we change the error indicating that virtual generated columns are

Re: Virtual generated columns

2025-02-05 Thread Dean Rasheed
On Tue, 4 Feb 2025 at 22:36, Peter Eisentraut wrote: > > Yeah, this is quite contorted. I have renamed it like you suggested. I looked over this again and I think the patch is in good shape to be committed. One thought that occurred to me was whether it would be better for the psql describe out

Re: Virtual generated columns

2025-02-04 Thread Peter Eisentraut
On 28.01.25 10:40, Shlok Kyal wrote: Test 5: Update publication on non virtual gen with no column list specified CREATE TABLE t1 (a int, b int GENERATED ALWAYS AS (a * 2) VIRTUAL); create publication pub1 for table t1; alter table t1 replica identity full; update t1 set a = 10; No error is thro

Re: Virtual generated columns

2025-01-28 Thread Shlok Kyal
th that fixed and also a few > >>> tweaks suggested by Jian. > >>> > >> > >> I'm hoping to push my RETURNING OLD/NEW patch [1] soon, so I thought > >> that I would check how it works together with this patch. The good > >> news is

Re: Virtual generated columns

2025-01-27 Thread Dean Rasheed
On Mon, 27 Jan 2025 at 09:59, Peter Eisentraut wrote: > > Here is an updated patch that integrates the above changes and also > makes some adjustments now that the logical replication configuration > questions are resolved. I think this is complete now. > In struct ResultRelInfo, the following f

Re: Virtual generated columns

2025-01-23 Thread Shlok Kyal
fixed. Here is a new patch with that fixed and also a few > tweaks suggested by Jian. > > I've also added a patch that addresses logical replication. It > basically adds back some of the prohibitions against including generated > columns in publications that have been lifted,

Re: Virtual generated columns

2025-01-16 Thread vignesh C
basically adds back some of the prohibitions against including generated > >> columns in publications that have been lifted, but this time only for > >> virtual generated columns, and amends the documentation. It doesn't > >> rename the publication option "publi

Re: Virtual generated columns

2025-01-15 Thread Peter Eisentraut
only for virtual generated columns, and amends the documentation. It doesn't rename the publication option "publish_generated_columns", but maybe that should be done. There are two potential approaches we could take to address the "publish_generated_columns" option

Re: Virtual generated columns

2025-01-15 Thread Peter Eisentraut
this patch. The good news is that AFAICS everything just works, and it's possible to return old/new virtual generated columns in DML queries as expected. It did require a minor update, because my patch adds a new "result_relation" argument to ReplaceVarsFromTargetList() -- needed in

Re: Virtual generated columns

2025-01-15 Thread Dean Rasheed
ws is that AFAICS everything just works, and it's possible to return old/new virtual generated columns in DML queries as expected. It did require a minor update, because my patch adds a new "result_relation" argument to ReplaceVarsFromTargetList() -- needed in DML queries because,

Re: Virtual generated columns

2025-01-14 Thread vignesh C
On Tue, 14 Jan 2025 at 19:08, Peter Eisentraut wrote: > > > I've also added a patch that addresses logical replication. It > basically adds back some of the prohibitions against including generated > columns in publications that have been lifted, but this time only for

Re: Virtual generated columns

2025-01-14 Thread Peter Eisentraut
On 09.01.25 13:41, jian he wrote: we can not ALTER COLUMN DROP EXPRESSION for virtual for now. so the following comments in generated_virtual.sql conflict with the output. ``` -- check that dependencies between columns have also been removed ALTER TABLE gtest29 DROP COLUMN a; -- should not drop

Re: Virtual generated columns

2025-01-14 Thread Peter Eisentraut
On 09.01.25 09:38, jian he wrote: create user foo; create user bar; grant create on schema public to foo; \c - foo create table t1 (id int, ccnum text, ccredacted text generated always as (repeat('*', 12) || substr(ccnum, 13, 4)) virtual); grant select (id, ccredacted) on table t1 to bar; insert

Re: Virtual generated columns

2025-01-13 Thread Dean Rasheed
On Wed, 8 Jan 2025 at 16:14, Peter Eisentraut wrote: > > Here is a new patch version In expand_generated_columns_in_expr(): + RangeTblEntry *rte; + + rte = makeNode(RangeTblEntry); + rte->relid = RelationGetRelid(rel); + + node = expand_generated_columns_internal(node, re

Re: Virtual generated columns

2025-01-09 Thread Dean Rasheed
On Wed, 8 Jan 2025 at 16:14, Peter Eisentraut wrote: > > One thing I could use some review on is the access control handling and > security in general. You can create virtual generated columns that have > their own access privileges but which can read columns that the user > does

Re: Virtual generated columns

2025-01-09 Thread jian he
a int, > check (a * 2 > 0) > > in terms of the constraint execution. > > The current patch does not support not-null constraints, but that's > mostly because it's not implemented yet. Maybe that's what Jian was > thinking about. > yes. we have 4 fo

Re: Virtual generated columns

2025-01-09 Thread jian he
ctly to give my response to each item. > > One thing I could use some review on is the access control handling and > security in general. You can create virtual generated columns that have > their own access privileges but which can read columns that the user > does not have access t

Re: Virtual generated columns

2025-01-08 Thread Marcos Pegoraro
Em qua., 8 de jan. de 2025 às 16:23, Vik Fearing escreveu: > This is lying to the planner, and you get to enjoy whatever breaks > because of it. A function that accesses external data is not immutable; > it is stable at best. > I understand that, but it's not documented, so users can think that

Re: Virtual generated columns

2025-01-08 Thread Peter Eisentraut
On 08.01.25 17:38, Tom Lane wrote: Peter Eisentraut writes: On 03.12.24 15:15, jian he wrote: SELECT attrelid, attname, attgenerated FROM pg_attribute WHERE attgenerated IN ('v') and (attnotnull or not atthasdef); I don't understand what the purpose of testing attnotnull is. That is indepe

Re: Virtual generated columns

2025-01-08 Thread Vik Fearing
On 08/01/2025 20:19, Marcos Pegoraro wrote: Em qua., 8 de jan. de 2025 às 13:14, Peter Eisentraut escreveu: Here is a new patch version where I have gathered various pieces of feedback and improvement suggestions that are scattered over this thread.  I hope I got them all.  I wil

Re: Virtual generated columns

2025-01-08 Thread Marcos Pegoraro
Em qua., 8 de jan. de 2025 às 13:14, Peter Eisentraut escreveu: > Here is a new patch version where I have gathered various pieces of > feedback and improvement suggestions that are scattered over this > thread. I hope I got them all. I will respond to the respective > messages directly to give

Re: Virtual generated columns

2025-01-08 Thread Tom Lane
Peter Eisentraut writes: > On 03.12.24 15:15, jian he wrote: >> SELECT attrelid, attname, attgenerated FROM pg_attribute WHERE >> attgenerated IN ('v') and (attnotnull or not atthasdef); > I don't understand what the purpose of testing attnotnull is. That is > independent of attgenerated, I thi

Re: Virtual generated columns

2025-01-08 Thread Peter Eisentraut
On 03.12.24 15:15, jian he wrote: -- check constraints CREATE TABLE gtest20 (a int PRIMARY KEY, b int GENERATED ALWAYS AS (a * 2) VIRTUAL CHECK (b < 50)); INSERT INTO gtest20 (a) VALUES (10); -- ok INSERT INTO gtest20 (a) VALUES (30); -- violates constraint ALTER TABLE gtest20 ALTER COLUMN b S

Re: Virtual generated columns

2025-01-08 Thread Peter Eisentraut
On 16.12.24 15:34, jian he wrote: hi. some minor issues... SET EXPRESSION AS This form replaces the expression of a generated column. Existing data in the column is rewritten and all the future changes will apply the new generation expression.

Re: Virtual generated columns

2025-01-08 Thread Peter Eisentraut
ssary. please check the attached patch. * remove check_modified_virtual_generated. * using heap_modify_tuple_by_cols in ExecBRInsertTriggers, ExecBRUpdateTriggers to overwrite virtual generated columns value to null. and it's not complicated. so that trigger behavior for stored and virtual wi

Re: Virtual generated columns

2025-01-08 Thread Peter Eisentraut
On 04.12.24 05:55, jian he wrote: On Fri, Nov 29, 2024 at 6:13 PM Peter Eisentraut wrote: - Added support for virtual columns in trigger column lists. (For that, I renamed ExecInitStoredGenerated() to ExecInitGenerated(), which handles the computation of ri_extraUpdatedCols.) why not dupli

Re: Virtual generated columns

2025-01-08 Thread Peter Eisentraut
On 08.01.25 09:22, Richard Guo wrote: - Added support for ALTER TABLE ... SET EXPRESSION. When using ALTER TABLE to set expression for virtual generated columns, we don't enforce a rewrite, which means we don't have the opportunity to check whether the new values for these columns c

Re: Virtual generated columns

2025-01-08 Thread Richard Guo
On Fri, Nov 29, 2024 at 7:14 PM Peter Eisentraut wrote: > Here is a new patch version, with several updates. > - Added support for ALTER TABLE ... SET EXPRESSION. When using ALTER TABLE to set expression for virtual generated columns, we don't enforce a rewrite, which means we don

Re: Virtual generated columns

2024-12-16 Thread jian he
hi. some minor issues... SET EXPRESSION AS This form replaces the expression of a generated column. Existing data in the column is rewritten and all the future changes will apply the new generation expression. the second sentence seems not to ap

Re: Virtual generated columns

2024-12-10 Thread jian he
be necessary. please check the attached patch. * remove check_modified_virtual_generated. * using heap_modify_tuple_by_cols in ExecBRInsertTriggers, ExecBRUpdateTriggers to overwrite virtual generated columns value to null. and it's not complicated. so that trigger behavior for st

Re: Virtual generated columns

2024-12-03 Thread jian he
On Fri, Nov 29, 2024 at 6:13 PM Peter Eisentraut wrote: > > - Added support for virtual columns in trigger column lists. (For that, > I renamed ExecInitStoredGenerated() to ExecInitGenerated(), which > handles the computation of ri_extraUpdatedCols.) > why not duplicate some code from ExecInitSt

Re: Virtual generated columns

2024-12-03 Thread jian he
-- check constraints CREATE TABLE gtest20 (a int PRIMARY KEY, b int GENERATED ALWAYS AS (a * 2) VIRTUAL CHECK (b < 50)); INSERT INTO gtest20 (a) VALUES (10); -- ok INSERT INTO gtest20 (a) VALUES (30); -- violates constraint ALTER TABLE gtest20 ALTER COLUMN b SET EXPRESSION AS (a * 100); -- viol

Re: Virtual generated columns

2024-12-03 Thread Peter Eisentraut
On 28.11.24 10:35, Peter Eisentraut wrote: On 12.11.24 17:08, Peter Eisentraut wrote: On 11.11.24 12:37, jian he wrote: On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut wrote: New patch version.  I've gone through the whole thread again and looked at all the feedback and various bug reports

Re: Virtual generated columns

2024-12-02 Thread Amit Kapila
able x to publication because it contains a virtual generated column > >> with a non-simple expression"? With row filters, this is less of a > >> problem, because the row filter a property of the publication. > >> > > Because virtual generated columns work in

Re: Virtual generated columns

2024-11-29 Thread Peter Eisentraut
On 12.11.24 17:10, Peter Eisentraut wrote: On 11.11.24 06:51, vignesh C wrote: The patch needs to be rebased due to a recent commit 14e87ffa5c5. done in v9 I have verified the behavior of logical replication of row filters on the virtual generated column, and everything appears to be functio

Re: Virtual generated columns

2024-11-29 Thread Peter Eisentraut
On 14.11.24 09:48, jian he wrote: inspired by not-null and check check_modified_virtual_generated again. in plpgsql_exec_trigger, we can: /* * In BEFORE trigger, stored generated columns are not computed yet, * so make them null in the NEW row. (Only needed in UPDAT

Re: Virtual generated columns

2024-11-29 Thread Peter Eisentraut
_NOT_SUPPORTED), errmsg("not-null constraints are not supported on virtual generated columns"), parser_errposition(cxt->pstate, constraint->location))); sometimes, it points to the word "generated", sometimes "not". I guess thi

Re: Virtual generated columns

2024-11-29 Thread Peter Eisentraut
On 12.11.24 17:50, Alvaro Herrera wrote: On 12.11.24 09:49, jian he wrote: On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut wrote: check_modified_virtual_generated, we can replace fastgetattr to heap_attisnull? like: // boolisnull; // fastgetattr(tuple, i

Re: Virtual generated columns

2024-11-29 Thread Peter Eisentraut
ith row filters, this is less of a problem, because the row filter a property of the publication. Because virtual generated columns work in row filters, so I thought it could follow the rules for column lists as well. If the virtual column doesn't adhere to the rules of the row filter then it

Re: Virtual generated columns

2024-11-28 Thread Peter Eisentraut
On 12.11.24 17:08, Peter Eisentraut wrote: On 11.11.24 12:37, jian he wrote: On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut wrote: New patch version.  I've gone through the whole thread again and looked at all the feedback and various bug reports and test cases and made sure they are all ad

Re: Virtual generated columns

2024-11-14 Thread Amit Kapila
On Tue, Nov 12, 2024 at 9:47 PM Peter Eisentraut wrote: > > On 10.11.24 04:16, Amit Kapila wrote: > > The possible idea to replicate virtual generated columns is to compute > > the corresponding expression before sending the data to the client. If > > we can allow it in

Re: Virtual generated columns

2024-11-14 Thread jian he
On Wed, Nov 13, 2024 at 11:30 AM jian he wrote: > > These 3 functions will call StoreRelNotNull to store the not-null constraint. > StoreConstraints > AddRelationNotNullConstraints > AddRelationNewConstraints > > we can disallow not-null on virtual generated columns via t

Re: Virtual generated columns

2024-11-12 Thread jian he
rmsg("not-null constraints are not supported on virtual generated columns"), parser_errposition(cxt->pstate, constraint->location))); sometimes, it points to the word "generated", sometimes "not". I guess this should be fine. example: create t

Re: Virtual generated columns

2024-11-12 Thread Alvaro Herrera
On 2024-Nov-12, Peter Eisentraut wrote: > On 12.11.24 09:49, jian he wrote: > > > On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut > > > wrote: > > check_modified_virtual_generated, we can replace fastgetattr to > > heap_attisnull? like: > > // boolisnull; > >

Re: Virtual generated columns

2024-11-12 Thread Peter Eisentraut
On 10.11.24 04:16, Amit Kapila wrote: The possible idea to replicate virtual generated columns is to compute the corresponding expression before sending the data to the client. If we can allow it in the row filter than why not to publish it as well. Row filters have pretty strong restrictions

Re: Virtual generated columns

2024-11-12 Thread Peter Eisentraut
On 11.11.24 06:51, vignesh C wrote: The patch needs to be rebased due to a recent commit 14e87ffa5c5. done in v9 I have verified the behavior of logical replication of row filters on the virtual generated column, and everything appears to be functioning as expected. One suggestion would be to

Re: Virtual generated columns

2024-11-12 Thread Peter Eisentraut
On 12.11.24 09:49, jian he wrote: On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut wrote: RelationBuildPartitionKey if (!isnull) { char *exprString; Node *expr; exprString = TextDatumGetCString(datum); expr = stringToNode(exprString); pfr

Re: Virtual generated columns

2024-11-12 Thread Peter Eisentraut
On 11.11.24 12:37, jian he wrote: On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut wrote: New patch version. I've gone through the whole thread again and looked at all the feedback and various bug reports and test cases and made sure they are all addressed in the latest patch version. (I'll

Re: Virtual generated columns

2024-11-12 Thread Peter Eisentraut
On 07.11.24 11:02, Dean Rasheed wrote: On Tue, 5 Nov 2024 at 16:17, Peter Eisentraut wrote: New patch version. In expand_generated_columns_in_expr(): +/* + * XXX For the benefit of triggers, make two passes, so it covers + * PRS2_OLD_VARNO and PRS2_NEW_VARNO. +

Re: Virtual generated columns

2024-11-12 Thread jian he
> On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut wrote: > > > > New patch version. I've gone through the whole thread again and looked > > at all the feedback and various bug reports and test cases and made sure > > they are all addressed in the latest patch version. (I'll send some > > separa

Re: Virtual generated columns

2024-11-11 Thread jian he
On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut wrote: > > New patch version. I've gone through the whole thread again and looked > at all the feedback and various bug reports and test cases and made sure > they are all addressed in the latest patch version. (I'll send some > separate messages

Re: Virtual generated columns

2024-11-10 Thread vignesh C
On Tue, 5 Nov 2024 at 21:48, Peter Eisentraut wrote: > > On 30.09.24 04:09, Peter Eisentraut wrote: > > I'm attaching a consolidated patch here, so we have something up to date > > on the record. I haven't worked through all the other recent feedback > > from Jian He yet; I'll do that next. > > N

Re: Virtual generated columns

2024-11-09 Thread Amit Kapila
ssages to respond to some individual messages, but I'm > keeping the latest patch here.) > I have tried to analyze this patch's interaction with logical replication. The patch allows virtual generated columns in row filters and column lists. But for the column list, it doesn't se

Re: Virtual generated columns

2024-11-07 Thread Dean Rasheed
On Tue, 5 Nov 2024 at 16:17, Peter Eisentraut wrote: > > New patch version. In expand_generated_columns_in_expr(): +/* + * XXX For the benefit of triggers, make two passes, so it covers + * PRS2_OLD_VARNO and PRS2_NEW_VARNO. + */ +node = expand_generated_c

Re: Virtual generated columns

2024-11-07 Thread Dean Rasheed
On Tue, 5 Nov 2024 at 16:17, Peter Eisentraut wrote: > > New patch version. What happened with the RLS support? It looks like you moved the code to expand virtual generated columns back to the first loop in fireRIRrules(), which doesn't work because RLS policies might contain re

Re: Virtual generated columns

2024-11-05 Thread Peter Eisentraut
: foreign key constraints on virtual generated columns are not supported I think the existing behavior is fine. The first message is about something that is invalid anyway. The second message is just that something is not supported yet. If we end up implementing, then users will get the first

Re: Virtual generated columns

2024-11-05 Thread Peter Eisentraut
onstr->has_generated_virtual" for the following code? if (constr->has_not_null || constr->has_generated_stored || ndef > 0 || attrmiss || relation->rd_rel->relchecks > 0) fixed in v8 also seems there will be table_rewrite for adding

Re: Virtual generated columns

2024-09-17 Thread jian he
raints on virtual generated columns are not supported change contrib/pageinspect/sql/page.sql expand information on t_infomask, t_bits information. change RelationBuildLocalRelation make the transient TupleDesc->TupleConstr three bool flags more accurate. v6-0001-virtual-generated-columns-misc-chan

Re: Virtual generated columns

2024-09-16 Thread jian he
>has_not_null || constr->has_generated_stored || ndef > 0 || attrmiss || relation->rd_rel->relchecks > 0) also seems there will be table_rewrite for adding virtual generated columns, but we can avoid that. The attached patch is the change and th

Re: Virtual generated columns

2024-09-08 Thread Peter Eisentraut
On 05.09.24 10:27, jian he wrote: On Wed, Sep 4, 2024 at 4:40 PM Peter Eisentraut wrote: Here is an implementation of this. It's much nicer! It also appears to fix all the additional test cases that have been presented. (I haven't integrated them into the patch set yet.) I left the 0001 p

Re: Virtual generated columns

2024-09-08 Thread Peter Eisentraut
fireRIRrules()? Yes, that's what I had in mind except that it has to be called from the second loop in fireRIRrules(), after any RLS policies have been added, because it's possible for a RLS policy expression to refer to virtual generated columns. It's OK to do it in the same loop t

Re: Virtual generated columns

2024-09-05 Thread jian he
On Wed, Sep 4, 2024 at 4:40 PM Peter Eisentraut wrote: > > > Here is an implementation of this. It's much nicer! It also appears to > fix all the additional test cases that have been presented. (I haven't > integrated them into the patch set yet.) > > I left the 0001 patch alone for now and put

Re: Virtual generated columns

2024-09-04 Thread Dean Rasheed
On Wed, 4 Sept 2024 at 09:40, Peter Eisentraut wrote: > > On 21.08.24 12:51, Dean Rasheed wrote: > >> > > Well what I was thinking was that (in fireRIRrules()'s final loop over > > relations in the rtable), if the relation had any virtual generated > > column

Re: Virtual generated columns

2024-09-02 Thread jian he
On Thu, Aug 29, 2024 at 9:35 PM jian he wrote: > > On Thu, Aug 29, 2024 at 8:15 PM Peter Eisentraut wrote: > > > > > > The new patch does some rebasing and contains various fixes to the > > issues you presented. As I mentioned, I'll look into improving the > > rewriting. > > > based on your la

Re: Virtual generated columns

2024-09-02 Thread Nazir Bilal Yavuz
Hi, On Thu, 29 Aug 2024 at 15:16, Peter Eisentraut wrote: > > I also committed the two patches that renamed the existing test files, > so those are not included here anymore. > > The new patch does some rebasing and contains various fixes to the > issues you presented. As I mentioned, I'll look

Re: Virtual generated columns

2024-09-02 Thread jian he
mply using ReplaceVarsFromTargetList() for each > > > RTE with virtual generated columns. That function already has the > > > required wholerow handling code, so there'd be less code duplication. > > > > Hmm, I don't quite see how ReplaceVarsFromTargetList() c

Re: Virtual generated columns

2024-08-29 Thread jian he
On Thu, Aug 29, 2024 at 8:15 PM Peter Eisentraut wrote: > > > > drop table if exists comment_test cascade; > > CREATE TABLE comment_test ( > >id int, > >positive_col int GENERATED ALWAYS AS (22) CHECK (positive_col > 0), > >positive_col1 int GENERATED ALWAYS AS (22) stored CHECK (pos

Re: Virtual generated columns

2024-08-23 Thread jian he
notnull1 (); ALTER TABLE atnotnull1 ADD COLUMN c INT GENERATED ALWAYS AS (22), ADD PRIMARY KEY (c); ERROR: not-null constraints are not supported on virtual generated columns DETAIL: Column "c" of relation "atnotnull1" is a virtual generated column.

Re: Virtual generated columns

2024-08-21 Thread Dean Rasheed
On Wed, 21 Aug 2024 at 08:00, Peter Eisentraut wrote: > > On 08.08.24 20:22, Dean Rasheed wrote: > > Looking at the rewriter changes, it occurred to me that it could > > perhaps be done more simply using ReplaceVarsFromTargetList() for each > > RTE with virtual generate

Re: Virtual generated columns

2024-08-21 Thread Peter Eisentraut
On 08.08.24 20:22, Dean Rasheed wrote: Looking at the rewriter changes, it occurred to me that it could perhaps be done more simply using ReplaceVarsFromTargetList() for each RTE with virtual generated columns. That function already has the required wholerow handling code, so there'd be

Re: Virtual generated columns

2024-08-13 Thread jian he
ot; \d gtest_parent \d gtest_child \d gtest_child2 \d gtest_child3 SELECT tableoid::regclass, * FROM gtest_parent ORDER BY 1, 2, 3; """ Similarly the following tests for gtest29 may aslo need change -- ALTER TABLE ... ALTER COLUMN ... DROP EXPRESSION since we cannot do ALTER TABLE SE

Re: Virtual generated columns

2024-08-08 Thread Dean Rasheed
n't properly deal with virtual generated columns in wholerow attributes: CREATE TABLE foo(a int, a2 int GENERATED ALWAYS AS (a*2) VIRTUAL); INSERT INTO foo VALUES (1); SELECT foo FROM foo; foo -- (1,) (1 row) Looking at the rewriter changes, it occurred to me that it could perhaps be

  1   2   >