numeric datatype for current release not available

2023-07-12 Thread Ashutosh Bapat
't look at what's going wrong. -- Best Wishes, Ashutosh Bapat

Re: numeric datatype for current release not available

2023-07-12 Thread Ashutosh Bapat
On Wed, Jul 12, 2023 at 8:55 PM Julien Rouhaud wrote: > > it's working here. probably a transient error, it happens from time to time. Thanks Julien for looking into it. It's working now. -- Best Wishes, Ashutosh Bapat

Re: logical decoding and replication of sequences, take 2

2023-07-13 Thread Ashutosh Bapat
That also makes a case for not adding a new field to WAL which may not be used. [1] https://www.postgresql.org/message-id/CAExHW5v_vVqkhF4ehST9EzpX1L3bemD1S%2BkTk_-ZVu_ir-nKDw%40mail.gmail.com [2] https://www.postgresql.org/message-id/CAExHW5vHRgjWzi6zZbgCs97eW9U7xMtzXEQK%2BaepuzoGDsDNtg%40mail.gmail.com -- Best Wishes, Ashutosh Bapat

Re: logical decoding and replication of sequences, take 2

2023-07-13 Thread Ashutosh Bapat
On Thu, Jul 13, 2023 at 8:29 PM Tomas Vondra wrote: > > On 6/23/23 15:18, Ashutosh Bapat wrote: > > ... > > > > I reviewed 0001 and related parts of 0004 and 0008 in detail. > > > > I have only one major change request, about > > typedef struct x

Re: logical decoding and replication of sequences, take 2

2023-07-14 Thread Ashutosh Bapat
; case we still need to handle the transactional case, to support sequence > replication without DDL replication enabled. As I said before, I don't think this patchset needs to wait for DDL replication patch. Let's hope that the later lands in the same release and straightens protocol instead of carrying it forever. [1] https://www.postgresql.org/message-id/CAExHW5vScYKKb0RZoiNEPfbaQ60hihfuWeLuZF4JKrwPJXPcUw%40mail.gmail.com -- Best Wishes, Ashutosh Bapat

Re: logical decoding and replication of sequences, take 2

2023-07-14 Thread Ashutosh Bapat
41.325 IST] [916213] [] [] [:0] LOCATION: LogChildExit, postmaster.c:3737 ``` When the subscriber subscribes to the publication without sequence, things work normally. The cross-version replication is working as expected then. -- Best Wishes, Ashutosh Bapat

Re: logical decoding and replication of sequences, take 2

2023-07-14 Thread Ashutosh Bapat
nly add relfilenodes for the sequence. > > > > Will try. > Actually, adding all relfilenodes to hash may not be that bad. There shouldn't be many of those. So the extra step to lookup reltype may not be necessary. What's your reason for uneasiness? But yeah, there's a way to avoid that as well. Should I wait for this before the second round of review? -- Best Wishes, Ashutosh Bapat

Re: logicalrep_message_type throws an error

2023-07-17 Thread Ashutosh Bapat
r occurs in psprintf(), it will throw an error which will override the original error. I think we should avoid any stuff that throws further errors. -- Best Wishes, Ashutosh Bapat

Re: logical decoding and replication of sequences, take 2

2023-07-18 Thread Ashutosh Bapat
bscriber: subscribe to pub_tab subscriber: replicates the data but rows which were deleted on publisher remain on the subscriber -- Best Wishes, Ashutosh Bapat

Re: logicalrep_message_type throws an error

2023-07-18 Thread Ashutosh Bapat
h do something similar by using statically allocated buffers like static char xya[SIZE]. We could do that here. The caller may decide whether to pstrdup this buffer further or just use it one time e.g. as an elog or printf argument. As I said before, we should not even print message type in the error context because it's unknown. Repeating that twice is useless. That will need some changes to apply_error_callback() though. But I am fine with "???" as well. -- Best Wishes, Ashutosh Bapat

Re: logical decoding and replication of sequences, take 2

2023-07-18 Thread Ashutosh Bapat
t before ALTER PUBLICATION can not be reverted, but that's expected. Coming back to TRUNCATE, I don't think it's possible to know whether a publication will send a truncate downstream or not. So we can't throw an error before TRUNCATE change is decoded. Anyway, I think this behaviour should be documented. I didn't see this mentioned in PUBLICATION or SUBSCRIPTION documentation. [1] https://www.postgresql.org/docs/current/sql-alterpublication.html -- Best Wishes, Ashutosh Bapat

Re: unrecognized node type while displaying a Path due to dangling pointer

2023-07-19 Thread Ashutosh Bapat
de in reparameterize_path_by_childrel() but that's very specific to the purpose there and doesn't consider Custom or Foreign paths. -- Best Wishes, Ashutosh Bapat

Re: Let's make PostgreSQL multi-threaded

2023-07-19 Thread Ashutosh Bapat
rather than shared memory. On Sat, Jun 10, 2023 at 5:25 AM Bruce Momjian wrote: > > On Wed, Jun 7, 2023 at 06:38:38PM +0530, Ashutosh Bapat wrote: > > With multiple processes, we can use all the available cores (at least > > theoretically if all those processes are indepen

Re: logical decoding and replication of sequences, take 2

2023-07-20 Thread Ashutosh Bapat
not sequences are replicated is decided by the protocol version, do we really need LogicalDecodingContext::sequences? Drawing parallel with WAL messages, I don't think it's needed. [1] https://www.postgresql.org/message-id/CAExHW5vScYKKb0RZoiNEPfbaQ60hihfuWeLuZF4JKrwPJXPcUw%40mail.gmail.com -

Re: logicalrep_message_type throws an error

2023-07-20 Thread Ashutosh Bapat
e that's what is visible in the code and also the message types are communicated as characters not integers. Makes debugging easier. Context may report integer as an additional data point. -- Best Wishes, Ashutosh Bapat

Re: logicalrep_message_type throws an error

2023-07-24 Thread Ashutosh Bapat
gt; think otherwise. WFM. -- Best Wishes, Ashutosh Bapat

Re: logical decoding and replication of sequences, take 2

2023-07-24 Thread Ashutosh Bapat
; Do you intend to keep these XXX's as is? My previous comments on this > > comment > > block are in [1]. This comment remains unanswered. > > > > In fact, given that whether or not sequences are replicated is decided by > > the > > protocol version, do we really need LogicalDecodingContext::sequences? > > Drawing > > parallel with WAL messages, I don't think it's needed. > > > > Right. We do that for two_phase because you can override that when > creating the subscription - sequences allowed that too initially, but > then we ditched that. So I don't think we need this. Then we should just remove that member and its references. -- Best Wishes, Ashutosh Bapat

Re: logical decoding and replication of sequences, take 2

2023-07-24 Thread Ashutosh Bapat
leanup on abort. > > What do you think? Hash table per transaction seems saner design. Adding it to the top level transaction should be fine. The entry will contain an XID anyway. If we add it to every subtransaction we will need to search hash table in each of the subtransactions when deciding

Re: logical decoding and replication of sequences, take 2

2023-07-25 Thread Ashutosh Bapat
w.postgresql.org/message-id/flat/21c87ea8-86c9-80d6-bc78-9b95033ca00b%40enterprisedb.com#36bb9c7968b7af577dc080950761290d -- Best Wishes, Ashutosh Bapat

Memory consumption during partitionwise join planning

2023-07-27 Thread Ashutosh Bapat
[2] https://www.postgresql.org/message-id/flat/CAJ2pMkZNCgoUKSE%2B_5LthD%2BKbXKvq6h2hQN8Esxpxd%2Bcxmgomg%40mail.gmail.com -- Best Wishes, Ashutosh Bapat setup.sql Description: application/sql queries.sql Description: application/sql From a2ff55d53033a3bc00d587d1ad4354b54bc2a998 Mon Sep 17 00

Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-07-27 Thread Ashutosh Bapat
quired RestrictInfos may not be available in RelOptInfo::joininfo. Let me know your thoughts on this. Comments/suggestions welcome. references [1] https://www.postgresql.org/message-id/caexhw5stmouobe55pmt83r8uxvfcph+pvo5dnpdrvcsbgxe...@mail.gmail.com -- Best Wishes, Ashutosh Bapat setup.sql D

Memory consumed by paths during partitionwise join planning

2023-07-27 Thread Ashutosh Bapat
mail.gmail.com -- Best Wishes, Ashutosh Bapat setup.sql Description: application/sql queries.sql Description: application/sql From 3d24fca1b861741949225b40f6df8892409e8d00 Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Mon, 17 Jul 2023 15:10:45 +0530 Subject: [PATCH 4/4] Local variables p

Memory consumed by child SpecialJoinInfo in partitionwise join planning

2023-07-27 Thread Ashutosh Bapat
ve is more suitable or if there are other approaches References [1] https://www.postgresql.org/message-id/caexhw5stmouobe55pmt83r8uxvfcph+pvo5dnpdrvcsbgxe...@mail.gmail.com [2] https://www.postgresql.org/message-id/CAExHW5s=bclmmq8n_bn6iu+pjau0ds3z_6dn6ile69esmsp...@mail.gmail.com -- Best Wishes, Ashu

Re: [May be a bug] double free or corruption

2023-07-27 Thread Ashutosh Bapat
pt > connections > [postgres@fedora postgres]$ psql > psql (17devel) > Type "help" for help. > > postgres=# \q > [postgres@fedora postgres]$ I guess gdb hit this bug after detaching from the backend you were debugging. So it continued running. In case it crashed because some

Re: unrecognized node type while displaying a Path due to dangling pointer

2023-07-27 Thread Ashutosh Bapat
patch set uses references to free memory consumed by paths which remain unused. The memory consumed is substantial when partitionwise join is used and there are thousands of partitions. [1] https://www.postgresql.org/message-id/CAExHW5tUcVsBkq9qT%3DL5vYz4e-cwQNw%3DKAGJrtSyzOp3F%3DXacA%40mail.gmail.com -- Best Wishes, Ashutosh Bapat

Re: Assert failure on bms_equal(child_joinrel->relids, child_joinrelids)

2023-07-28 Thread Ashutosh Bapat
ught I would mention it where I found it. [1] https://www.postgresql.org/message-id/caexhw5stmouobe55pmt83r8uxvfcph+pvo5dnpdrvcsbgxe...@mail.gmail.com -- Best Wishes, Ashutosh Bapat From 0f00fc48b9ebf73b54724ba82cec4a69d5f63846 Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Wed, 12 Jul 2023 14:

Re: [PoC] Reducing planning time when tables have many partitions

2023-07-28 Thread Ashutosh Bapat
5pmt83r8uxvfcph+pvo5dnpdrvcsbgxe...@mail.gmail.com [2] https://www.postgresql.org/message-id/CAExHW5s4EqY43oB%3Dne6B2%3D-xLgrs9ZGeTr1NXwkGFt2j-OmaQQ%40mail.gmail.com -- Best Wishes, Ashutosh Bapat planning time measurement.ods Description: application/vnd.oasis.opendocument.spreadsheet

Re: logical decoding and replication of sequences, take 2

2023-07-28 Thread Ashutosh Bapat
On Tue, Jul 25, 2023 at 10:02 PM Tomas Vondra wrote: > > On 7/24/23 14:57, Ashutosh Bapat wrote: > > ... > > > >> > >> > >> 2) Currently, the sequences hash table is in reorderbuffer, i.e. global. > >> I was thinking maybe we should have it

Re: logical decoding and replication of sequences, take 2

2023-07-28 Thread Ashutosh Bapat
ame of the function doesn't leave much space for expanding its functionality. So we are good with a new one. Probably some code deduplication. -- Best Wishes, Ashutosh Bapat

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2023-07-28 Thread Ashutosh Bapat
y parent relids of the child rels. 2. Do not free those. 3. Add a comment about keeping the build and free functions in sync. I will work on those next. -- Best Wishes, Ashutosh Bapat

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-02 Thread Ashutosh Bapat
ed of in nearby threads. > As long as the translation happens only once, it should be fine. It's the repeated translations which should be avoided. -- Best Wishes, Ashutosh Bapat

Re: Inquiry about Functionality Availability in PostgreSQL

2023-08-03 Thread Ashutosh Bapat
eSQL database, could you kindly let me know and provide any relevant insights about their potential implementation in future versions? > Core PostgreSQL may not provide some of the functionalities by itself but extensions like pg_audit may provide those functionalities. Maybe you want to lo

Re: Adding a LogicalRepWorker type field

2023-08-03 Thread Ashutosh Bapat
quired by all RelOptKinds crammed under the same structure. If we can avoid that here at the beginning itself, that will be great. May be we should create a union of type specific members while we are introducing the type. This will also provide some protection against unrelated members being (mis)used in the future. -- Best Wishes, Ashutosh Bapat

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-03 Thread Ashutosh Bapat
On Thu, Aug 3, 2023 at 4:14 PM Richard Guo wrote: > > On Thu, Aug 3, 2023 at 12:38 PM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > > Sometimes it would help to avoid the translation at all if we postpone > the reparameterization until createplan.c, such as

Re: [PoC] Reducing planning time when tables have many partitions

2023-08-03 Thread Ashutosh Bapat
On Wed, Aug 2, 2023 at 12:11 PM Yuya Watari wrote: > Hello, > > I really appreciate sharing very useful scripts and benchmarking results. > > On Fri, Jul 28, 2023 at 6:51 PM Ashutosh Bapat > wrote: > > Given that most of the developers run assert enabled builds it wo

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2023-08-04 Thread Ashutosh Bapat
On Fri, Jul 28, 2023 at 7:28 PM Ashutosh Bapat wrote: > > > > + bms_free(child_sjinfo->commute_above_l); > > + bms_free(child_sjinfo->commute_above_r); > > + bms_free(child_sjinfo->commute_below_l); > > + bms_free(child_sjinfo->commu

Re: [PoC] Reducing planning time when tables have many partitions

2023-08-07 Thread Ashutosh Bapat
ilar will be done to the planning memory. I will propose it as a separate patch in the next commitfest and will seek opinions from other hackers. -- Best Wishes, Ashutosh Bapat

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-07 Thread Ashutosh Bapat
On Fri, Aug 4, 2023 at 6:08 AM Richard Guo wrote: > > On Thu, Aug 3, 2023 at 7:20 PM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> However, if reparameterization can *not* happen at the planning time, we >> have chosen a plan which can not be real

Report planning memory in EXPLAIN ANALYZE

2023-08-07 Thread Ashutosh Bapat
=-xlgrs9zgetr1nxwkgft2j-om...@mail.gmail.com -- Best Wishes, Ashutosh Bapat From 1dcadcc9cb38702f9e8882bf16173715982e36da Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Wed, 12 Jul 2023 14:34:14 +0530 Subject: [PATCH] Report memory used for planning a query in EXPLAIN ANALYZE The memory used in the

Re: [PoC] Reducing planning time when tables have many partitions

2023-08-07 Thread Ashutosh Bapat
resql.org/message-id/CAExHW5sZA=5lj_zppro-w09ck8z9p7eayaqq3ks9gdfhrxe...@mail.gmail.com -- Best Wishes, Ashutosh Bapat

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-08 Thread Ashutosh Bapat
hat the > path must be reparameterizable. Works for me. > > Attaching v3 patch to address all the reviews above. The patch looks good to me. Please add this to the next commitfest. -- Best Wishes, Ashutosh Bapat

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-08-08 Thread Ashutosh Bapat
lso figure out answers to 1, 2, 3 there. Please let me know your opinions if any. -- Best Wishes, Ashutosh Bapat [1] https://www.postgresql.org/message-id/CAExHW5s=bclmmq8n_bn6iu+pjau0ds3z_6dn6ile69esmsp...@mail.gmail.com From 4abdfd676a31d885ccdbe1803c60c1df1d0c1df2 Mon Sep 17 00:00:00 2001 Fro

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-09 Thread Ashutosh Bapat
nt of outer rel. > Your changes in create_nestloop_path() only affect the check for parameterized path not the unparameterized paths. So I think we are good there. My worry was misplaced. -- Best Wishes, Ashutosh Bapat

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-08-09 Thread Ashutosh Bapat
On Wed, Aug 9, 2023 at 10:12 AM David Rowley wrote: > > On Fri, 28 Jul 2023 at 02:06, Ashutosh Bapat > wrote: > > 0001 - to measure memory consumption during planning. This is the same > > one as attached to [1]. > I have started a separate thread to discuss this

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-09 Thread Ashutosh Bapat
Hi David, Replying to your comments on this thread. > On Tue, Aug 8, 2023 at 11:40 AM Ashutosh Bapat > wrote: >> >> Hi All, >> I have been looking at memory consumed when planning a partitionwise join >> [1], [2] and [3]. I am using the attached patch to measure t

Re: [BackendXidGetPid] only access allProcs when xid matches

2023-08-09 Thread Ashutosh Bapat
can optimize this kind of inefficiency, I > believe we should ship with better code. > Looks good to me. However, I would just move the variable declaration with their assignments inside the if () rather than combing the expressions. It more readable that way. -- Best Wishes, Ashutosh Bapat

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-09 Thread Ashutosh Bapat
On Wed, Aug 9, 2023 at 8:09 PM Ashutosh Bapat wrote: > > Hi David, > Replying to your comments on this thread. > > > On Tue, Aug 8, 2023 at 11:40 AM Ashutosh Bapat > > wrote: > >> > >> Hi All, > >> I have been looking at memory consumed when

Re: [BackendXidGetPid] only access allProcs when xid matches

2023-08-10 Thread Ashutosh Bapat
Please add this to commitfest so that it's not forgotten. On Wed, Aug 9, 2023 at 8:37 PM Junwang Zhao wrote: > > On Wed, Aug 9, 2023 at 10:46 PM Ashutosh Bapat > wrote: > > > > On Wed, Aug 9, 2023 at 9:30 AM Junwang Zhao wrote: > > > > > > In fun

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-10 Thread Ashutosh Bapat
On Wed, Aug 9, 2023 at 8:56 PM David Rowley wrote: > > On Thu, 10 Aug 2023 at 03:12, Ashutosh Bapat > wrote: > > Thinking more about it, I think memory used is the only right metrics. > > It's an optimization in MemoryContext implementation that malloc'ed >

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-10 Thread Ashutosh Bapat
Hi David, On Wed, Aug 9, 2023 at 8:09 PM Ashutosh Bapat wrote: > > I need to just make sure that the Planning Memory is reported with SUMMARY ON. > The patch reports planning memory in EXPLAIN without ANALYZE when SUMMARY = ON. #explain (summary on) select * from a, b where a.c1 = b.c1

Re: logical decoding and replication of sequences, take 2

2023-08-10 Thread Ashutosh Bapat
downstream as a principle. I think a way to achieve this is to update the replorigin_session_origin_lsn so that a sequence change applied once is not requested (and hence sent) again. [1] https://en.wikipedia.org/wiki/Rubber_banding -- Best Wishes, Ashutosh Bapat

AssertLog instead of Assert in some places

2023-08-11 Thread Ashutosh Bapat
r terminated abnormally before or while processing the request. The connection to the server was lost. Attempting reset: Failed. The connection to the server was lost. Attempting reset: Failed. @!#\q ``` If there is an interest in accepting the patch, I will add it to the commitfest an

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-08-11 Thread Ashutosh Bapat
I spent some time on 4th point below but also looked at other points. Here's what I have found so far On Thu, Jul 27, 2023 at 7:35 PM Ashutosh Bapat wrote: > > 1. The patch uses RestrictInfo::required_relids as the key for > searching child RelOptInfos. I am not sure which

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-11 Thread Ashutosh Bapat
other hand, developers can watch it easily in different > ways, if needed. So, I vote for the 'memory used' metric only. > > The patch looks good, passes the tests. Thanks Andrey. David, are you against reporting "memory used"? If you are not against reporting used me

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-14 Thread Ashutosh Bapat
On Mon, Aug 14, 2023 at 5:23 AM David Rowley wrote: > > On Thu, 10 Aug 2023 at 20:33, Ashutosh Bapat > wrote: > > My point is what's relevant here is how much net memory planner asked > > for. > > But that's not what your patch is reporting. All you'

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-14 Thread Ashutosh Bapat
racking itself might make the planning in-efficient and we might still miss the spikes in memory allocations, if they are very short lived. If the planner runs for more than a few minutes, maybe we could add some tracking. -- Best Wishes, Ashutosh Bapat

Re: AssertLog instead of Assert in some places

2023-08-14 Thread Ashutosh Bapat
und > process that just restarts. Fair point. Our regression doesn't check server error logs for unwanted errors. How about restricting it to only client backends? I don't know how to identify those from others but there must be a way. -- Best Wishes, Ashutosh Bapat

Re: AssertLog instead of Assert in some places

2023-08-14 Thread Ashutosh Bapat
't help. Those might make things worse. We need two separate incantations for non-recoverable and recoverable Asserts respectively. I like Peter's idea of having a new elevel, however it still requires adding conditional USE_ASSERT, an if testing the condition and then writing an error message. AssertLog() in the patch uses just a few more letters. It won't help to expand the scope of the problem since that will reduce the chances of getting anything done. -- Best Wishes, Ashutosh Bapat

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2023-08-15 Thread Ashutosh Bapat
On Fri, Aug 4, 2023 at 2:11 PM Ashutosh Bapat wrote: > > Attached patchset fixing those. > 0001 - patch to report planning memory, with to explain.out regression output > fix. We may consider committing this as well. > 0002 - with your comment addressed above. 0003 - Added

Re: logical decoding and replication of sequences, take 2

2023-08-17 Thread Ashutosh Bapat
o review the impact of not setting replorigin_session_origin_timestamp. What fake transaction experiment are you talking about? -- Best Wishes, Ashutosh Bapat

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-21 Thread Ashutosh Bapat
Essentially combing reparameterize_path_by_child() and path_is_reparameterizable_by_child(). The name of such a function can be chosen appropriately. The mode will be controlled by a fourth argument to the function. When assessing a path no translations happen and no extra memory is allocated. -- Best Wishes, Ashutosh Bapat

Re: persist logical slots to disk during shutdown checkpoint

2023-08-21 Thread Ashutosh Bapat
s 1. to mark a slot dirty and persist if the last confirm_flush_lsn when slot was persisted was too far from the current confirm_flush_lsn of the slot. 2. at shutdown checkpoint, persist all the slots which have these two confirm_flush_lsns different. -- Best Wishes, Ashutosh Bapat

Re: persist logical slots to disk during shutdown checkpoint

2023-08-22 Thread Ashutosh Bapat
(1) right now, we might just forget to do that small incremental change in future. My preference is 1. Do both (1) and (2) together 2. Do (2) first and then (1) as a separate commit. 3. Just implement (2) if we don't have time at all for first two options. -- Best Wishes, Ashutosh Bapat

Re: persist logical slots to disk during shutdown checkpoint

2023-08-22 Thread Ashutosh Bapat
better to do that > optimization (persist confirm_flush_lsn at a regular interval) as a > separate patch as we need to test and prove its value separately. Fine with me. -- Best Wishes, Ashutosh Bapat

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-22 Thread Ashutosh Bapat
o which need to be outside the macros and actually look better to me. Let me know what you think of this. FWIW I ran make check and all the tests pass. I agree that we can not consider this for backbranches but we are anyway thinking of disabling reparameterization for tablesa

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-08-23 Thread Ashutosh Bapat
1b = s.a; > count > --- > 5 > (1 row) > > > For another query it would error out during planning. > > regression=# explain (verbose, costs off) > select count(*) from prt1 t1 left join lateral > (select t1.b as t1b, t2.* from prt2 t2) s > on t1.a = s.b where s.t1b = s.b; > ERROR: variable not found in subplan target list > > To fix it we may need to modify RelOptInfos for Path, BitmapHeapPath, > ForeignPath and CustomPath, and modify IndexOptInfos for IndexPath. It > seems that that is not easily done without postponing reparameterization > of paths until createplan.c. Maybe I am missing something here but why aren't we copying these restrictinfos in the path somewhere? I have a similar question for tablesample clause as well. -- Best Wishes, Ashutosh Bapat

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-24 Thread Ashutosh Bapat
ture, we will need a way to create a new memory context with the same properties as the CurrentMemoryContext, a functionality missing right now. Once we agree upon the approach, the patch will need to be merged into 0002 and in turn 0001. -- Best Wishes, Ashutosh Bapat explain_planning_memory.o

apply_scanjoin_target_to_paths and partitionwise join

2024-04-10 Thread Ashutosh Bapat
ry. I have tried a few variations but without success. Suggestions welcome. The problem is reproducible on PG 15. The patch is based on 15_STABLE branch. But the problem exists in recent branches as well. -- Best Wishes, Ashutosh Bapat diff --git a/src/backend/optimizer/plan/planner.c b/src/back

Re: apply_scanjoin_target_to_paths and partitionwise join

2024-04-10 Thread Ashutosh Bapat
On Thu, Apr 11, 2024 at 12:07 PM Ashutosh Bapat < ashutosh.bapat@gmail.com> wrote: > Hi All, > Per below code and comment in apply_scanjoin_target_to_paths(), the > function zaps all the paths of a partitioned relation. > /* > * If the rel is partitioned, we want to dr

Re: apply_scanjoin_target_to_paths and partitionwise join

2024-04-15 Thread Ashutosh Bapat
Here's patch with On Thu, Apr 11, 2024 at 12:24 PM Ashutosh Bapat < ashutosh.bapat@gmail.com> wrote: > > > On Thu, Apr 11, 2024 at 12:07 PM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> Hi All, >> Per below code and comment in apply

Re: hash_destroy on the hash table allocated with TopMemoryContext

2024-04-17 Thread Ashutosh Bapat
id you create hash table in TopMemoryContext? -- Best Wishes, Ashutosh Bapat

Re: tablecmds.c/MergeAttributes() cleanup

2024-04-19 Thread Ashutosh Bapat
On Sat, Apr 20, 2024 at 9:30 AM Alexander Lakhin wrote: > Hello, > > 30.01.2024 09:22, Ashutosh Bapat wrote: > > > >> Please look at the segmentation fault triggered by the following query > since > >> 4d969b2f8: > >> CREATE TABLE t1(a text C

Re: Short-circuit sort_inner_and_outer if there are no mergejoin clauses

2024-04-25 Thread Ashutosh Bapat
the function is a noop when mergeclause_list is empty. I haven't looked closely to see if creating unique path nonetheless is useful somewhere else. Please add to the next commitfest. If the patch shows some measurable performance improvement, it would become more attractive. -- Best Wishes, Ashutosh Bapat

Re: partitioning and identity column

2024-04-26 Thread Ashutosh Bapat
Thanks Alexander for the report. On Fri, Apr 26, 2024 at 5:30 PM Alexander Lakhin wrote: > Hello Ashutosh and Peter, > > 16.01.2024 21:59, Peter Eisentraut wrote: > > On 09.01.24 15:10, Ashutosh Bapat wrote: > >> Here's complete patch-set. > > > > Look

Re: Test to dump and restore objects left behind by regression

2024-04-26 Thread Ashutosh Bapat
On Fri, Feb 23, 2024 at 10:46 AM Ashutosh Bapat < ashutosh.bapat@gmail.com> wrote: > On Thu, Feb 22, 2024 at 8:35 PM Tom Lane wrote: > > > > Peter Eisentraut writes: > > > The problem is, we don't really have any end-to-end coverage of > >

Re: New committers: Melanie Plageman, Richard Guo

2024-04-26 Thread Ashutosh Bapat
ease join us in wishing them much success and few reverts! > > Thanks, > > Jonathan > -- Best Wishes, Ashutosh Bapat

Re: Background Processes in Postgres Extension

2024-04-28 Thread Ashutosh Bapat
from core dump is useful. if you can reproduce the crash, running reproduction with gdb attached to a process intended to crash is more useful. -- Best Wishes, Ashutosh Bapat

Re: tablecmds.c/MergeAttributes() cleanup

2024-04-29 Thread Ashutosh Bapat
On Mon, Apr 29, 2024 at 6:46 PM Robert Haas wrote: > On Sat, Apr 20, 2024 at 12:17 AM Ashutosh Bapat > wrote: > > Yes please. Probably this issue surfaced again after we reverted > compression and storage fix? Please If that's the case, please add it to > the open item

Re: partitioning and identity column

2024-04-30 Thread Ashutosh Bapat
n. I have examined all the callers of getIdentitySequence() and they seem to be fine. The code related to SetIdentity, DropIdentity is not called for partitions, errors for which are thrown elsewhere earlier. -- Best Wishes, Ashutosh Bapat From bce2e8d573040901b18c0c9f6884f0fd44f50724 Mon Sep 17

Re: Control flow in logical replication walsender

2024-05-01 Thread Ashutosh Bapat
PostgreSQL (v11), so an > upgrade may fix it, but at the moment, I'm trying to find a cause and a > mitigation. > > Is there a large transaction which is failing to be replicated repeatedly - timeouts, crashes on upstream or downstream? -- Best Wishes, Ashutosh Bapat

Re: tablecmds.c/MergeAttributes() cleanup

2024-05-03 Thread Ashutosh Bapat
ttable, but I can't do it now because I > > won't be around for the next few hours in case the buildfarm blows up. > > I can do it tomorrow, or perhaps Peter would like to handle it since > > it seems to have been his commit that introduced the issue. > > I have committed it now. > > Thanks Peter. -- Best Wishes, Ashutosh Bapat

Re: apply_scanjoin_target_to_paths and partitionwise join

2024-05-06 Thread Ashutosh Bapat
On Mon, May 6, 2024 at 4:26 PM Jakub Wartak wrote: > Hi Ashutosh & hackers, > > On Mon, Apr 15, 2024 at 9:00 AM Ashutosh Bapat > wrote: > > > > Here's patch with > > > [..] > > Adding to the next commitfest but better to consider this for the nex

Re: partitioning and identity column

2024-05-06 Thread Ashutosh Bapat
On Sun, May 5, 2024 at 1:43 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Tue, Apr 30, 2024 at 04:29:11PM +0530, Ashutosh Bapat wrote: > > On Sun, Apr 28, 2024 at 12:29 PM Alexander Lakhin > > wrote: > > > > > 27.04.2024 18:00, Alexander Lakhin wro

Re: Control flow in logical replication walsender

2024-05-06 Thread Ashutosh Bapat
On Tue, May 7, 2024 at 12:00 AM Christophe Pettus wrote: > Thank you for the reply! > > > On May 1, 2024, at 02:18, Ashutosh Bapat > wrote: > > Is there a large transaction which is failing to be replicated > repeatedly - timeouts, crashes on upstream or downstream? >

Re: partitioning and identity column

2024-05-09 Thread Ashutosh Bapat
Thanks a lot Peter. On Wed, May 8, 2024 at 2:34 AM Peter Eisentraut wrote: > On 30.04.24 12:59, Ashutosh Bapat wrote: > > PFA patch which fixes all the three problems. > > I have committed this patch. Thanks all. > -- Best Wishes, Ashutosh Bapat

Re: apply_scanjoin_target_to_paths and partitionwise join

2024-05-22 Thread Ashutosh Bapat
On Mon, May 6, 2024 at 6:28 PM Ashutosh Bapat wrote: > > > On Mon, May 6, 2024 at 4:26 PM Jakub Wartak > wrote: > >> Hi Ashutosh & hackers, >> >> On Mon, Apr 15, 2024 at 9:00 AM Ashutosh Bapat >> wrote: >> > >> > Here's patch

Re: Avoid possible dereference null pointer (src/backend/catalog/pg_depend.c)

2024-05-23 Thread Ashutosh Bapat
have to invoke relation_open() with new relid, in order to use rel in the error message. I don't think all that is worth it, unless we find a scenario when SearchSysCacheAttName() returns NULL. -- Best Wishes, Ashutosh Bapat

Re: struct RelOptInfo member relid comments

2024-05-23 Thread Ashutosh Bapat
ase+OJ", but I cannot find the > explanation of "OJ" or the "OJ" Acronyms. > > > OJ is an outer join, AFAIU. OJ's have their own relids. If you are wondering why not all joins - I think inner joins need not be tracked as separated relations in parse tree, but OJ's need to be. -- Best Wishes, Ashutosh Bapat

Re: Avoid possible dereference null pointer (src/backend/catalog/pg_depend.c)

2024-05-23 Thread Ashutosh Bapat
oesn't throw an error and returns InvalidAttnum which won't return any valid identity sequence, and thus return a NIL sequence list which is handled in that function already. Using these two functions will avoid the clutter as well as segfault. If that's acceptable, I will provide a patch. -- Best Wishes, Ashutosh Bapat

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-05-24 Thread Ashutosh Bapat
the function. There was some discussion about setting a search path for a function at [1]. But the last message there is non-conclusive. We may want to extend it to extensions such that all the object references in a given extension are resolved using extension specific search_path. [1] https://www.postgresql.org/message-id/2710f56add351a1ed553efb677408e51b060e67c.camel%40j-davis.com -- Best Wishes, Ashutosh Bapat

Re: Avoid possible dereference null pointer (src/backend/catalog/pg_depend.c)

2024-05-24 Thread Ashutosh Bapat
On Fri, May 24, 2024 at 12:16 PM Michael Paquier wrote: > On Fri, May 24, 2024 at 11:58:51AM +0530, Ashutosh Bapat wrote: > > If we are looking for avoiding a segfault and get a message which helps > > debugging, using get_attname and get_attnum might be better options. > >

Re: Avoid possible dereference null pointer (src/backend/catalog/pg_depend.c)

2024-05-27 Thread Ashutosh Bapat
e > tempted to introduce a missing_ok to this routine after looking at the > callers in all the tree, as some of them want to fail still would not > expect it, so that would reduce a bit the elog churn. That's a story > for a different day, though. > -- > Michael > -- Best Wishes, Ashutosh Bapat

Re: apply_scanjoin_target_to_paths and partitionwise join

2024-05-27 Thread Ashutosh Bapat
ty for you. Am I right? Plans with lower costs being slower is not new for optimizer. Partitionwise join just adds another case. > > Maybe we can discuss that in person next week? > Sure. > > On 2024-05-22 07:57, Ashutosh Bapat wrote: > > > > The test was added by 6b

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2024-05-28 Thread Ashutosh Bapat
On Mon, Feb 19, 2024 at 5:17 AM Ashutosh Bapat wrote: > On Mon, Feb 19, 2024 at 4:35 AM Tomas Vondra > wrote: > > > > Hi, > > > > After taking a look at the patch optimizing SpecialJoinInfo allocations, > > I decided to take a quick look at this one too. I

meson and check-tests

2024-05-31 Thread Ashutosh Bapat
ssing pg_regress --schedule argument and instead pass the list of tests. Any idea how to do that? -- Best Wishes, Ashutosh Bapat

Re: Logical Replication of sequences

2024-06-04 Thread Ashutosh Bapat
f the node. > > Any other ideas? > > In case of primary crash the sequence won't get replicated. That is true even with the previous approach in case walsender is shut down because of a crash, but it is more serious with this approach. How about periodically sending this information? -- Best Wishes, Ashutosh Bapat

Re: Reuse child_relids in try_partitionwise_join was Re: Assert failure on bms_equal(child_joinrel->relids, child_joinrelids)

2024-06-05 Thread Ashutosh Bapat
7;planning time' and p.variant = 'pwj' order by p.vari ant, num_joins; [1] https://www.postgresql.org/message-id/CAExHW5snUW7pD2RdtaBa1T_TqJYaY6W_YPVjWDrgSf33i-0uqA%40mail.gmail.com -- Best Wishes, Ashutosh Bapat setup.sql Description: application/sql queries.sql Description: app

Re: apply_scanjoin_target_to_paths and partitionwise join

2024-06-05 Thread Ashutosh Bapat
s column statistics and this bug I am fixing. Can you please elaborate? > 3. I dislike, that this patch makes much harder to debug, why no > partitionwise join is chosen. > Can you please elaborate more? How does my change make debugging harder? -- Best Wishes, Ashutosh Bapat

Re: meson and check-tests

2024-06-05 Thread Ashutosh Bapat
On Mon, Jun 3, 2024 at 10:04 PM Tristan Partin wrote: > On Sun Jun 2, 2024 at 12:25 AM CDT, Tom Lane wrote: > > "Tristan Partin" writes: > > > On Fri May 31, 2024 at 12:02 PM CDT, Ashutosh Bapat wrote: > > >> We talked this off-list at the conference.

Re: Test to dump and restore objects left behind by regression

2024-06-05 Thread Ashutosh Bapat
On Tue, Jun 4, 2024 at 4:28 AM Michael Paquier wrote: > On Fri, Apr 26, 2024 at 06:38:22PM +0530, Ashutosh Bapat wrote: > > Some points for discussion: > > 1. The test still hardcodes the diffs between two dumps. Haven't found a > > better way to do it. I did conside

<    2   3   4   5   6   7   8   9   10   11   >