Re: vac_update_datfrozenxid will raise "wrong tuple length" if pg_database tuple contains toast attribute.

2020-11-30 Thread Ashwin Agrawal
On Sun, Nov 29, 2020 at 5:27 PM Michael Paquier wrote: > > One thing that strikes me as unwise is that we could run into similar > > problems with vac_update_relstats() in the future, and there have been > > recent talks about having more toast tables like for pg_class. That > > is not worth car

Re: [PoC] Non-volatile WAL buffer

2020-11-24 Thread Ashwin Agrawal
On Sun, Nov 22, 2020 at 5:23 PM Tomas Vondra wrote: > I'm not entirely sure whether the "pmemdax" (i.e. unpatched instance > with WAL on PMEM DAX device) is actually safe, but I included it anyway > to see what difference is. I am curious to learn more on this aspect. Kernels have provided supp

Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes

2022-01-03 Thread Ashwin Agrawal
On Wed, Dec 22, 2021 at 4:23 PM SATYANARAYANA NARLAPURAM < satyanarlapu...@gmail.com> wrote: > Hi Hackers, > > I am considering implementing RPO (recovery point objective) enforcement > feature for Postgres where the WAL writes on the primary are stalled when > the WAL distance between the primary

For standby pg_ctl doesn't wait for PM_STATUS_READY in presence of promote_trigger_file

2020-08-04 Thread Ashwin Agrawal
y) + if (!EnableHotStandby && !promote_trigger_file_exist) { AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY); #ifdef USE_SYSTEMD -- *Ashwin Agrawal (VMware)*

Re: language cleanups in code and docs

2020-08-20 Thread Ashwin Agrawal
On Wed, Jun 17, 2020 at 9:27 AM David Steele wrote: > On 6/17/20 12:08 PM, Magnus Hagander wrote: > > On Wed, Jun 17, 2020 at 4:15 PM Andrew Dunstan > > mailto:andrew.duns...@2ndquadrant.com>> > > > > > I'm not sure I like doing s/Black/Block/ here. It reads oddly. There > are > > too man

CREATE RULE may generate duplicate entries in pg_depend

2020-08-26 Thread Ashwin Agrawal
1 | n pg_rewrite | rtest_t5 |2618 | 16457 |0 | 1259 |16445 | 2 | n (3 rows) -- *Ashwin Agrawal (VMware)*

Re: Syntax highlighting for Postgres spec files

2019-09-19 Thread Ashwin Agrawal
I didn't try as waiting to see if for emacs as well shows up :-) Do we want to get these in src/tools/editors? On Thu, Sep 19, 2019 at 10:15 AM Ekin Dursun wrote: > Hi people, > > I have written language plugins for .spec files used in isolation tests. > They are available for Vim and Visual Stu

Re: heapam_index_build_range_scan's anyvisible

2019-09-25 Thread Ashwin Agrawal
ll warnings being treated as errors > : recipe for target 'verify_nbtree.o' failed > make[4]: *** [verify_nbtree.o] Error 1 > Thanks for reporting, I did indeed missed out contrib. Please find attached the v2 of the patch which includes the change required in contrib as well.

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-04 Thread Ashwin Agrawal
On Fri, Oct 4, 2019 at 8:49 AM Tom Lane wrote: > Jacob Champion writes: > > On Fri, Oct 4, 2019 at 7:51 AM Tom Lane wrote: > >> I concur with Joe here. The reason why some of the existing > >> memset's use "false" is for symmetry with other places where we use > >> "memset(p, true, n)" to set

Missing error_context_stack = NULL in AutoVacWorkerMain()

2019-10-18 Thread Ashwin Agrawal
I am not sure if this causes any potential problems or not, but for consistency of code seems we are missing below. All other places in code where sigsetjmp() exists for top level handling has error_context_stack set to NULL. diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaste

Re: Pluggable Storage - Andres's take

2019-03-05 Thread Ashwin Agrawal
Hi, While playing with the tableam, usage of which starts with commit v12-0023-tableam-Introduce-and-use-begin-endscan-and-do-i.patch, should we check for NULL function pointer before actually calling the same and ERROR out instead as NOT_SUPPORTED or something on those lines. Understand its kind

Re: Make drop database safer

2019-03-06 Thread Ashwin Agrawal
On Mon, Feb 11, 2019 at 3:55 PM Ashwin Agrawal wrote: > > Thanks for the response and inputs. > > On Sat, Feb 9, 2019 at 4:51 AM Andres Freund wrote: > >> Hi, >> >> On 2019-02-08 16:36:13 -0800, Alexandra Wang wrote: >> > Current sequence of operations

Re: Make drop database safer

2019-03-06 Thread Ashwin Agrawal
On Wed, Mar 6, 2019 at 7:56 AM Tomas Vondra wrote: > > > On 2/12/19 12:55 AM, Ashwin Agrawal wrote: > > > > Thanks for the response and inputs. > > > > On Sat, Feb 9, 2019 at 4:51 AM Andres Freund > <mailto:and...@anarazel.de>> wrote: > > &

Re: Using the return value of strlcpy() and strlcat()

2019-03-13 Thread Ashwin Agrawal
On Wed, Mar 13, 2019 at 9:51 AM Tom Lane wrote: > ilm...@ilmari.org (Dagfinn Ilmari =?utf-8?Q?Manns=C3=A5ker?=) writes: > > [ let's convert > > + strlcpy(buf + buflen, name, NAMEDATALEN); > > + buflen += strlen(buf + buflen); > > to > > + buflen += strlcpy(buf

Re: heapam_index_build_range_scan's anyvisible

2019-07-30 Thread Ashwin Agrawal
On Tue, Jul 16, 2019 at 10:22 AM Andres Freund wrote: > Hi, > > On 2019-07-11 17:27:46 -0700, Ashwin Agrawal wrote: > > Please find attached the patch to remove IndexBuildCallback's dependency > on > > HeapTuple, as discussed. Changed to have the argument as ItemPoi

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-07-31 Thread Ashwin Agrawal
On Tue, Jul 30, 2019 at 2:58 PM Thomas Munro wrote: > On Tue, Jun 25, 2019 at 6:02 AM Andres Freund wrote: > > > - CheckForSerializableConflictOut() no more takes HeapTuple nor > > > buffer, instead just takes xid. Push heap specific parts from > > > CheckForSerializableConflictOut() into it

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-07-31 Thread Ashwin Agrawal
On Wed, Jul 31, 2019 at 10:55 AM Andres Freund wrote: > Hi, > > On 2019-07-31 10:42:50 -0700, Ashwin Agrawal wrote: > > On Tue, Jul 30, 2019 at 2:58 PM Thomas Munro > wrote: > > > > > On Tue, Jun 25, 2019 at 6:02 AM Andres Freund > wrote: > > > &g

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-07-31 Thread Ashwin Agrawal
On Wed, Jul 31, 2019 at 12:37 PM Ashwin Agrawal wrote: > > On Wed, Jul 31, 2019 at 10:55 AM Andres Freund wrote: > >> Hi, >> >> On 2019-07-31 10:42:50 -0700, Ashwin Agrawal wrote: >> > On Tue, Jul 30, 2019 at 2:58 PM Thomas Munro >> wrote: >>

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-02 Thread Ashwin Agrawal
ead. So, the flow would be AM calling CheckForSerializableConflictOut() as today and only if serializable in use will invoke the callback to check with AM if more work should be performed or not. Essentially HeapCheckForSerializableConflictOut() will become callback function instead. Due to void* callback argument a

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-07 Thread Ashwin Agrawal
On Fri, Aug 2, 2019 at 4:56 PM Ashwin Agrawal wrote: > > On Wed, Jul 31, 2019 at 2:06 PM Andres Freund wrote: > >> Looking at the code as of master, we currently have: >> > > Super awesome feedback and insights, thank you! > > - PredicateLockTuple() calls S

Re: Zedstore - compressed in-core columnar storage

2019-08-14 Thread Ashwin Agrawal
On Wed, Aug 14, 2019 at 2:51 AM Ashutosh Sharma wrote: > Hi Ashwin, > > I tried playing around with the zedstore code a bit today and there > are couple questions that came into my mind. > Great! Thank You. > > 1) Can zedstore tables be vacuumed? If yes, does VACUUM on zedstore > table set th

Re: Comment in ginpostinglist.c doesn't match code

2019-08-22 Thread Ashwin Agrawal
On Thu, Aug 22, 2019 at 1:14 AM Heikki Linnakangas wrote: > > The patch also includes a little unit test module to test this without > creating a 16 TB table. A whole new test module seems a bit like > overkill just for this, but clearly we were missing test coverage here. > And it will come hand

Re: Zedstore - compressed in-core columnar storage

2019-08-26 Thread Ashwin Agrawal
On Mon, Aug 26, 2019 at 5:36 AM Ashutosh Sharma wrote: > Thanks Ashwin and Heikki for your responses. I've one more query here, > > If BTree index is created on a zedstore table, the t_tid field of > Index tuple contains the physical tid that is not actually pointing to > the data block instead i

make_etags: avoid recursive symbolic creation.

2018-01-18 Thread Ashwin Agrawal
src/tools/make_etags creates link to TAGS file from each and every directory in source tree. It works fine without all those links in each and every directory in source code so remove the extra work and annoyance. diff --git a/src/tools/make_etags b/src/tools/make_etags index 3ce96bc3ca..2bfd1314f

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-11-08 Thread Ashwin Agrawal
On Thu, Nov 7, 2019 at 8:44 PM Thomas Munro wrote: > On Thu, Aug 8, 2019 at 6:53 AM Ashwin Agrawal wrote: > >>> - I wonder if CheckForSerializableConflictOutNeeded() shouldn't have a > >>> portion of it's code as a static inline. In particular, it's

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-11-12 Thread Ashwin Agrawal
On Sun, Nov 10, 2019 at 8:21 PM Thomas Munro wrote: > I pushed the first two, Thank You! but on another read-through of the main patch > I didn't like the comments for CheckForSerializableConflictOut() or > the fact that it checks SerializationNeededForRead() again, after I > thought a bit abo

Re: Start Walreceiver completely before shut down it on standby server.

2019-12-10 Thread Ashwin Agrawal
On Tue, Dec 10, 2019 at 3:06 AM jiankang liu wrote: > Start Walreceiver completely before shut down it on standby server. > > The walreceiver will be shut down, when read an invalid record in the > WAL streaming from master.And then, we retry from archive/pg_wal again. > > After that, we start wa

Re: Should we rename amapi.h and amapi.c?

2019-12-23 Thread Ashwin Agrawal
On Sun, Dec 22, 2019 at 9:34 PM Michael Paquier wrote: > Hi all, > > I was working on some stuff for table AMs, and I got to wonder it we > had better rename amapi.h to indexam.h and amapi.c to indexam.c, so as > things are more consistent with table AM. It is a bit annoying to > name the files

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2020-01-27 Thread Ashwin Agrawal
On Mon, Jan 27, 2020 at 4:47 PM Thomas Munro wrote: > OK, I kept only the small comment change from that little fixup patch, > and pushed this. > > > I had proposed as alternative way in initial email and also later, > > didn't receive comment on that, so re-posting. > > > typedef bool (*AMCheckF

Re: Zedstore - compressed in-core columnar storage

2019-04-08 Thread Ashwin Agrawal
On Mon, Apr 8, 2019 at 6:04 PM Andres Freund wrote: > Hi, > > On 2019-04-08 17:27:05 -0700, Ashwin Agrawal wrote: > > Heikki and I have been hacking recently for few weeks to implement > > in-core columnar storage for PostgreSQL. Here's the design and initial >

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Ashwin Agrawal
On Tue, Apr 9, 2019 at 11:29 AM Robert Haas wrote: > On Tue, Apr 9, 2019 at 11:51 AM Alvaro Herrera > wrote: > > This is not surprising, considering that columnar store is precisely the > > reason for starting the work on table AMs. > > > > We should certainly look into integrating some sort of

Re: Zedstore - compressed in-core columnar storage

2019-04-09 Thread Ashwin Agrawal
On Tue, Apr 9, 2019 at 9:13 AM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > > On 09.04.2019 18:51, Alvaro Herrera wrote: > > On 2019-Apr-09, Konstantin Knizhnik wrote: > > > >> On 09.04.2019 3:27, Ashwin Agrawal wrote: > >>> Heikki a

Re: block-level incremental backup

2019-04-10 Thread Ashwin Agrawal
On Wed, Apr 10, 2019 at 9:21 AM Robert Haas wrote: > I have a related idea, though. Suppose that, as Peter says upthread, > you have a replication slot that prevents old WAL from being removed. > You also have a background worker that is connected to that slot. It > decodes WAL and produces sum

Re: Zedstore - compressed in-core columnar storage

2019-04-10 Thread Ashwin Agrawal
> On Apr 10, 2019, at 9:08 PM, Mark Kirkwood > wrote: > > >> On 11/04/19 4:01 PM, Mark Kirkwood wrote: >>> On 9/04/19 12:27 PM, Ashwin Agrawal wrote: >>> >>> Heikki and I have been hacking recently for few weeks to implement >>> in-core

Re: finding changed blocks using WAL scanning

2019-04-11 Thread Ashwin Agrawal
On Wed, Apr 10, 2019 at 2:50 PM Robert Haas wrote: > Over at > https://www.postgresql.org/message-id/CA%2BTgmobFVe4J4AA7z9OMUzKnm09Tt%2BsybhxeL_Ddst3q3wqpzQ%40mail.gmail.com > I mentioned parsing the WAL to extract block references so that > incremental backup could efficiently determine which bl

Re: finding changed blocks using WAL scanning

2019-04-11 Thread Ashwin Agrawal
On Thu, Apr 11, 2019 at 6:27 AM Robert Haas wrote: > On Thu, Apr 11, 2019 at 3:52 AM Peter Eisentraut > wrote: > > I had in mind that you could have different overlapping incremental > > backup jobs in existence at the same time. Maybe a daily one to a > > nearby disk and a weekly one to a fara

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Sat, Apr 13, 2019 at 4:22 PM Peter Geoghegan wrote: > On Thu, Apr 11, 2019 at 6:06 AM Rafia Sabih > wrote: > > Reading about it reminds me of this work -- TAG column storage( > https://urldefense.proofpoint.com/v2/url?u=http-3A__www09.sigmod.org_sigmod_record_issues_0703_03.article-2Dgraefe.p

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Sun, Apr 14, 2019 at 9:40 AM Tomas Vondra wrote: > On Thu, Apr 11, 2019 at 06:20:47PM +0300, Heikki Linnakangas wrote: > >On 11/04/2019 17:54, Tom Lane wrote: > >>Ashwin Agrawal writes: > >>>Thank you for trying it out. Yes, noticed for certain patterns >

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Mon, Apr 15, 2019 at 10:33 AM Tomas Vondra wrote: > On Mon, Apr 15, 2019 at 09:29:37AM -0700, Ashwin Agrawal wrote: > > On Sun, Apr 14, 2019 at 9:40 AM Tomas Vondra > >wrote: > > > > On Thu, Apr 11, 2019 at 06:20:47PM +0300, Heikki Linnakangas wrote: &

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Mon, Apr 15, 2019 at 11:18 AM Tomas Vondra wrote: > > Maybe. I'm not going to pretend I fully understand the internals. Does > that mean the container contains ZSUncompressedBtreeItem as elements? Or > just the plain Datum values? > First, your reading of code and all the comments/questions s

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Mon, Apr 15, 2019 at 12:50 PM Peter Geoghegan wrote: > On Mon, Apr 15, 2019 at 9:16 AM Ashwin Agrawal > wrote: > > Would like to know more specifics on this Peter. We may be having > different context on hybrid row/column design. > > I'm confused about how close yo

Re: Regression test PANICs with master-standby setup on same machine

2019-04-24 Thread Ashwin Agrawal
On Wed, Apr 24, 2019 at 9:25 AM Andres Freund wrote: > The inability > to reasonably test master/standby setups on a single machine is pretty > jarring (yes, one can use basebackup tablespace maps - but that doesn't > work well for new tablespaces). +1 agree. Feature which can't be easily test

Re: Zedstore - compressed in-core columnar storage

2019-04-24 Thread Ashwin Agrawal
On Tue, Apr 16, 2019 at 9:15 AM Tomas Vondra wrote: > > I'm not sure it's that clear cut, actually. Sure, it's not the usual > (block,item) pair so it's not possible to jump to the exact location, so > it's not the raw physical identifier as regular TID. But the data are > organized in a btree, w

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-25 Thread Ashwin Agrawal
On Wed, Apr 24, 2019 at 5:48 PM Peter Geoghegan wrote: > On Wed, Apr 24, 2019 at 12:17 PM Tom Lane wrote: > > After a bit more thought it seemed like another answer would be to > > make all three of those functions assert that the caller did the > > right thing, as per attached. This addresses

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-25 Thread Ashwin Agrawal
On Thu, Apr 25, 2019 at 9:45 AM Tom Lane wrote: > However, by that argument we should change all 3 of these functions to > set up the data. If we're eating the layering violation to the extent > of letting OpenTemporaryFile call into commands/tablespace, then there's > little reason for the othe

Re: What is an item pointer, anyway?

2019-04-26 Thread Ashwin Agrawal
On Fri, Apr 26, 2019 at 2:19 PM Peter Geoghegan wrote: > itemid.h introduces the struct ItemIdData as follows: > > /* > * An item pointer (also called line pointer) on a buffer page > > Meanwhile, itemptr.h introduces the struct ItemPointerData as follows: > > /* > * ItemPointer: > * > * This

Re: Race conditions with checkpointer and shutdown

2019-04-29 Thread Ashwin Agrawal
On Sat, Apr 27, 2019 at 5:57 PM Tom Lane wrote: > > I have spent a fair amount of time trying to replicate these failures > locally, with little success. I now think that the most promising theory > is Munro's idea in [1] that the walreceiver is hanging up during its > unsafe attempt to do erepor

Re: Race conditions with checkpointer and shutdown

2019-04-29 Thread Ashwin Agrawal
On Mon, Apr 29, 2019 at 10:36 AM Tom Lane wrote: > > Can you try applying a1a789eb5ac894b4ca4b7742f2dc2d9602116e46 > to see if it fixes the problem for you? Yes, will give it a try on greenplum and report back the result. Have we decided if this will be applied to back branches?

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-29 Thread Ashwin Agrawal
On Thu, Apr 25, 2019 at 11:53 PM Michael Paquier wrote: > > On Thu, Apr 25, 2019 at 12:45:03PM -0400, Tom Lane wrote: > > I still remain concerned that invoking catalog lookups from fd.c is a darn > > bad idea, even if we have a fallback for it to work (for some value of > > "work") in non-transac

Re: Pluggable Storage - Andres's take

2019-04-29 Thread Ashwin Agrawal
On Thu, Apr 25, 2019 at 3:43 PM Andres Freund wrote: > Hm. I think some of those changes would be a bit bigger than I initially > though. Attached is a more minimal fix that'd route > RelationGetNumberOfBlocksForFork() through tableam if necessary. I > think it's definitely the right answer for 1

Match table_complete_speculative() code to comment

2019-04-30 Thread Ashwin Agrawal
Comment for table_complete_speculative() says /* * Complete "speculative insertion" started in the same transaction. If * succeeded is true, the tuple is fully inserted, if false, it's removed. */ static inline void table_complete_speculative(Relation rel, TupleTableSlot *slot,

Re: Pluggable Storage - Andres's take

2019-05-06 Thread Ashwin Agrawal
On Mon, May 6, 2019 at 7:14 AM Andres Freund wrote: > > Hi, > > On May 6, 2019 3:40:55 AM PDT, Rafia Sabih wrote: > >I was trying the toyam patch and on make check it failed with > >segmentation fault at > > > >static void > >toyam_relation_set_new_filenode(Relation rel, > > char persistence, > >

Re: Pluggable Storage - Andres's take

2019-05-07 Thread Ashwin Agrawal
On Mon, May 6, 2019 at 1:39 PM Ashwin Agrawal wrote: > > Also wish to point out, while working on Zedstore, we realized that > TupleDesc from Relation object can be trusted at AM layer for > scan_begin() API. As for ALTER TABLE rewrite case (ATRewriteTables()), > catalog is up

Inconsistency between table am callback and table function names

2019-05-08 Thread Ashwin Agrawal
The general theme for table function names seem to be "table_". For example table_scan_getnextslot() and its corresponding callback scan_getnextslot(). Most of the table functions and callbacks follow mentioned convention except following ones table_beginscan table_endscan table_rescan table_f

Re: Inconsistency between table am callback and table function names

2019-05-08 Thread Ashwin Agrawal
On Wed, May 8, 2019 at 2:51 PM Andres Freund wrote: > > Hi, > > On 2019-05-08 00:32:22 -0700, Ashwin Agrawal wrote: > > The general theme for table function names seem to be > > "table_". For example table_scan_getnextslot() and its > > corresponding callba

Re: Pluggable Storage - Andres's take

2019-05-09 Thread Ashwin Agrawal
On Wed, May 8, 2019 at 2:46 PM Andres Freund wrote: > > Hi, > > On 2019-05-07 23:18:39 -0700, Ashwin Agrawal wrote: > > On Mon, May 6, 2019 at 1:39 PM Ashwin Agrawal wrote: > > > Also wish to point out, while working on Zedstore, we realized that > > > T

Re: Inconsistency between table am callback and table function names

2019-05-10 Thread Ashwin Agrawal
On Thu, May 9, 2019 at 8:52 AM Andres Freund wrote: > The changes necessary for tableam were already huge. Changing naming > schemes for functions that are used all over the backend (e.g. ~80 calls > to table_beginscan), and where there's other wrapper functions that also > widely used (237 calls

Re: Inconsistency between table am callback and table function names

2019-05-10 Thread Ashwin Agrawal
On Fri, May 10, 2019 at 10:51 AM Andres Freund wrote: > > Hi, > > On 2019-05-10 10:43:44 -0700, Ashwin Agrawal wrote: > > On Thu, May 9, 2019 at 8:52 AM Andres Freund wrote: > > > The changes necessary for tableam were already huge. Changing naming > > > sch

Re: Inconsistency between table am callback and table function names

2019-05-14 Thread Ashwin Agrawal
On Mon, May 13, 2019 at 12:51 PM Robert Haas wrote: > On Fri, May 10, 2019 at 3:43 PM Ashwin Agrawal > wrote: > > Meant to stick the question mark in that email, somehow missed. Yes > > not planning to spend any time on it if objections. Here is the list > > of r

Re: Inconsistency between table am callback and table function names

2019-05-14 Thread Ashwin Agrawal
On Wed, May 8, 2019 at 5:05 PM Ashwin Agrawal wrote: > Not having consistency is the main aspect I wish to bring to > attention. Like for some callback functions the comment is > > /* see table_insert() for reference about parameters */ > void(*tuple_insert

Re: Adding a test for speculative insert abort case

2019-05-15 Thread Ashwin Agrawal
sion after inserting the tuple in unique index but before completing the speculative insert. Hence, helps to create the condition easily. I believe order of index insertion is helping here that unique index is inserted and then non-unique index is inserted too. Attaching patch with the test

Re: Pluggable Storage - Andres's take

2019-05-15 Thread Ashwin Agrawal
On Wed, May 15, 2019 at 11:54 AM Andres Freund wrote: > Hi, > > On 2019-04-25 15:43:15 -0700, Andres Freund wrote: > > > > 3) nodeTidscan, skipping over too large tids > > >I think this should just be moved into the AMs, there's no need to > > >have this in nodeTidscan.c > > > > I think h

Create TOAST table only if AM needs

2019-05-17 Thread Ashwin Agrawal
tion to AM. From fcbf6b8187ef8389cf38cd185b709bcb6ef1152a Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal Date: Fri, 10 May 2019 17:42:06 -0700 Subject: [PATCH 1/1] Create toast table only if AM needs it. All AM may not need toast table, hence add property to TableAmRoutine to define if uses toas

Re: Create TOAST table only if AM needs

2019-05-17 Thread Ashwin Agrawal
On Fri, May 17, 2019 at 1:51 PM Andres Freund wrote: > > > /* > > > * Planner related callbacks for the heap AM > > @@ -2558,6 +2615,8 @@ static const TableAmRoutine heapam_methods = { > > > > .relation_estimate_siz

Re: Create TOAST table only if AM needs

2019-05-17 Thread Ashwin Agrawal
On Fri, May 17, 2019 at 11:34 AM Andres Freund wrote: > Hi, > > On 2019-05-17 11:26:29 -0700, Ashwin Agrawal wrote: > > Currently TOAST table is always created (if needed based on data type > > properties) independent of table AM. How toasting is handled seems > >

Re: Pluggable Storage - Andres's take

2019-05-17 Thread Ashwin Agrawal
On Fri, May 17, 2019 at 12:54 PM Andres Freund wrote: > Hi, > > On 2019-05-15 23:00:38 -0700, Ashwin Agrawal wrote: > > Highlevel this looks good to me. Will look into full details tomorrow. > > Ping? > > I'll push the first of the patches soon, and unless you

Re: Create TOAST table only if AM needs

2019-05-17 Thread Ashwin Agrawal
On Fri, May 17, 2019 at 2:42 PM Robert Haas wrote: > In any case, it bears saying that > tableam is a remarkable accomplishment regardless of whatever > shortcomings it has in this area or elsewhere. > Big +1 to this.

Re: Pluggable Storage - Andres's take

2019-05-17 Thread Ashwin Agrawal
On Tue, Apr 9, 2019 at 6:17 AM Heikki Linnakangas wrote: > On 08/04/2019 20:37, Andres Freund wrote: > > On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: > >> There's a little bug in index-only scan executor node, where it mixes > up the > >> slots to hold a tuple from the index, and from

Re: Pluggable Storage - Andres's take

2019-05-17 Thread Ashwin Agrawal
On Wed, May 15, 2019 at 11:54 AM Andres Freund wrote: > Attached is a prototype of a variation of this. I added a > table_tuple_tid_valid(TableScanDesc sscan, ItemPointer tid) > callback / wrapper. Currently it just takes a "plain" scan, but we could > add a separate table_beginscan variant too.

Re: Inconsistency between table am callback and table function names

2019-05-24 Thread Ashwin Agrawal
On Thu, May 23, 2019 at 4:32 PM Andres Freund wrote: > Hi, > > On 2019-05-14 12:11:46 -0700, Ashwin Agrawal wrote: > > Thank you. Please find the patch to rename the agreed functions. It would > > be good to make all consistent instead of applying exception to three >

Re: Zedstore - compressed in-core columnar storage

2019-05-24 Thread Ashwin Agrawal
On Thu, May 23, 2019 at 7:30 PM Ajin Cherian wrote: > Hi Ashwin, > > - how to pass the "column projection list" to table AM? (as stated in > initial email, currently we have modified table am API to pass the > projection to AM) > > We were working on a similar columnar storage using pluggable

Confusing error message for REINDEX TABLE CONCURRENTLY

2019-05-24 Thread Ashwin Agrawal
ent case, make it consistent, less confusing and leave it there. Attaching the patch to just do that. Thoughts? From dc8119abe180cc14b0720c4de79495de4c62bf12 Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal Date: Fri, 24 May 2019 16:34:48 -0700 Subject: [PATCH v1] Avoid confusing error message for R

Re: Confusing error message for REINDEX TABLE CONCURRENTLY

2019-05-27 Thread Ashwin Agrawal
On Sun, May 26, 2019 at 6:43 PM Michael Paquier wrote: > As you mention for reindex_relation() no indexes <=> nothing to do, > still let's not rely on that. Instead of making the error message > specific to concurrent operations, I would suggest to change it to > "table foo has no indexes to rei

Re: Confusing error message for REINDEX TABLE CONCURRENTLY

2019-06-03 Thread Ashwin Agrawal
On Tue, May 28, 2019 at 12:05 PM David Rowley wrote: > On Tue, 28 May 2019 at 01:23, Ashwin Agrawal wrote: > > I think we will need to separate out the NOTICE message for concurrent > and regular case. > > > > For example this doesn't sound correct > >

Re: Comment typo in tableam.h

2019-06-03 Thread Ashwin Agrawal
There were few more minor typos I had collected for table am, passing them along here. Some of the required callback functions are missing Assert checking (minor thing), adding them in separate patch. From f32bdf5d0d3af5fd6ee6bf6430905f9c4bf5fefa Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal

Re: Comment typo in tableam.h

2019-06-03 Thread Ashwin Agrawal
On Mon, Jun 3, 2019 at 5:26 PM Andres Freund wrote: > Hi, > > Thanks for these! > > On 2019-06-03 17:24:15 -0700, Ashwin Agrawal wrote: > > /* > >* Estimate the size of shared memory needed for a parallel scan > of this > > - * relati

Re: Comment typo in tableam.h

2019-06-03 Thread Ashwin Agrawal
On Mon, Jun 3, 2019 at 6:24 PM Andres Freund wrote: > Hi, > > On 2019-06-03 18:21:56 -0700, Ashwin Agrawal wrote: > > On Mon, Jun 3, 2019 at 5:26 PM Andres Freund wrote: > > > > > Hi, > > > > > > Thanks for these! > > >

Re: Function to promote standby servers

2018-10-15 Thread Ashwin Agrawal
Just curious to know, is promotion via function only allowed for hot-standby or works for any standby? On Mon, Oct 15, 2018 at 7:16 AM Laurenz Albe wrote: > Michael Paquier wrote: > > > diff --git a/src/backend/access/transam/xlog.c > b/src/backend/access/transam/xlog.c > > > index 7375a78ffc..3

Re: Backends stalled in 'startup' state

2023-01-17 Thread Ashwin Agrawal
On Tue, Jan 17, 2023 at 4:52 PM Ashwin Agrawal wrote: > > We recently saw many backends (close to max_connection limit) get stalled > in 'startup' in one of the production environments for Greenplum (fork of > PostgreSQL). Tracing the reason, it was found all the tuples

Re: brininsert optimization opportunity

2023-11-25 Thread Ashwin Agrawal
On Sat, Nov 25, 2023 at 12:06 PM Tomas Vondra wrote: > I've done a bit more cleanup on the last version of the patch (renamed > the fields to start with bis_ as agreed, rephrased the comments / docs / > commit message a bit) and pushed. Thanks a lot Tomas for helping to drive the patch to compl

Re: Make mesage at end-of-recovery less scary.

2020-03-23 Thread Ashwin Agrawal
On Mon, Mar 23, 2020 at 2:37 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2020-03-05 08:06, Kyotaro Horiguchi wrote: > > | [20866] LOG: replication terminated by primary server > > | [20866] DETAIL: End of WAL reached on timeline 1 at 0/30001C8. > > | [20866] FATAL: could

Re: SyncRepLock acquired exclusively in default configuration

2020-04-06 Thread Ashwin Agrawal
es visible. > > Remove the race by using more appropriate test in syncrep.c > > > > Author: Asim Rama Praveen and Ashwin Agrawal > > Reported-by: Xin Zhang, Ashwin Agrawal, and Asim Rama Praveen > > Reviewed-by: Michael Paquier, Masahiko Sawada > > > &

Re: Backends stalled in 'startup' state

2022-09-27 Thread Ashwin Agrawal
On Thu, Sep 15, 2022 at 4:42 PM Ashwin Agrawal wrote: > > We recently saw many backends (close to max_connection limit) get stalled > in 'startup' in one of the production environments for Greenplum (fork of > PostgreSQL). Tracing the reason, it was found all the tuples

Re: storing an explicit nonce

2021-10-07 Thread Ashwin Agrawal
On Thu, Oct 7, 2021 at 12:12 PM Robert Haas wrote: > On Thu, Oct 7, 2021 at 2:52 PM Stephen Frost wrote: > > Assuming that's correct, and I don't see any reason to doubt it, then > > perhaps it would make sense to have the LSN be unencrypted and include > > it in the tweak as that would limit th

Backends stalled in 'startup' state

2022-09-15 Thread Ashwin Agrawal
); + else + tableScan = table_beginscan(OldHeap, SnapshotAny, 0, (ScanKey) NULL); heapScan = (HeapScanDesc) tableScan; indexScan = NULL; -- 1] https://www.post

pg_basebackup --create-slot-if-not-exists?

2022-09-21 Thread Ashwin Agrawal
EXISTS kind of semantic. (Seems the limitation most likely is coming from CREATE REPLICATION SLOT protocol itself), Thoughts? -- *Ashwin Agrawal (VMware)*

Re: pg_basebackup --create-slot-if-not-exists?

2022-09-22 Thread Ashwin Agrawal
On Wed, Sep 21, 2022 at 5:34 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Wednesday, September 21, 2022, Ashwin Agrawal > wrote: > >> Currently, pg_basebackup has >> --create-slot option to create slot if not already exists or >> --slot to use e

Re: SyncRepLock acquired exclusively in default configuration

2020-04-07 Thread Ashwin Agrawal
On Mon, Apr 6, 2020 at 2:14 PM Andres Freund wrote: > > How about we change it to this ? > > Hm. Better. But I think it might need at least a compiler barrier / > volatile memory load? Unlikely here, but otherwise the compiler could > theoretically just stash the variable somewhere locally (it's

pg_basebackup misses to report checksum error

2020-05-06 Thread Ashwin Agrawal
If pg_basebackup is not able to read BLCKSZ content from file, then it just emits a warning "could not verify checksum in file "" block X: read buffer size X and page size 8192 differ" currently but misses to error with "checksum error occurred". Only if it can read 8192 and checksum mismatch h

Re: pg_basebackup misses to report checksum error

2020-05-07 Thread Ashwin Agrawal
On Wed, May 6, 2020 at 3:02 PM Robert Haas wrote: > On Wed, May 6, 2020 at 5:48 PM Ashwin Agrawal wrote: > > If pg_basebackup is not able to read BLCKSZ content from file, then it > > just emits a warning "could not verify checksum in file "" block > >

Re: pg_basebackup misses to report checksum error

2020-05-07 Thread Ashwin Agrawal
On Thu, May 7, 2020 at 10:25 AM Stephen Frost wrote: > Greetings, > > * Ashwin Agrawal (aagra...@pivotal.io) wrote: > > On Wed, May 6, 2020 at 3:02 PM Robert Haas > wrote: > > > On Wed, May 6, 2020 at 5:48 PM Ashwin Agrawal > wrote: > > > > If pg_bas

Re: SyncRepLock acquired exclusively in default configuration

2020-05-19 Thread Ashwin Agrawal
On Mon, May 18, 2020 at 7:41 PM Masahiko Sawada < masahiko.saw...@2ndquadrant.com> wrote: > This item is for PG14, right? If so I'd like to add this item to the > next commit fest. > Sure, add it to commit fest. Seems though it should be backpatched to relevant branches as well.

Re: Make drop database safer

2019-02-11 Thread Ashwin Agrawal
Thanks for the response and inputs. On Sat, Feb 9, 2019 at 4:51 AM Andres Freund wrote: > Hi, > > On 2019-02-08 16:36:13 -0800, Alexandra Wang wrote: > > Current sequence of operations for drop database (dropdb()) > > 1. Start Transaction > > 2. Make catalog changes > > 3. Drop database buffers

Re: Extracting only the columns needed for a query

2020-01-31 Thread Ashwin Agrawal
On Wed, Jan 15, 2020 at 7:54 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > For UPDATE, we need all of the columns in the table because of the > > table_lock() API's current expectation that the slot has all of the > > columns populated. If we want UPDATE to only need to insert the column > >

Use LN_S instead of "ln -s" in Makefile

2020-02-14 Thread Ashwin Agrawal
re for src/backend/Makefile. Greping for 'ln -s' revealed three places it's used. Attaching the patch to fix the same. From 60b3374de1e4d0fd143c2a7b3bf78893e32579af Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal Date: Fri, 14 Feb 2020 16:16:37 -0800 Subject: [PATCH v1] Use LN_S in

Re: Use LN_S instead of "ln -s" in Makefile

2020-02-14 Thread Ashwin Agrawal
On Fri, Feb 14, 2020 at 4:57 PM Tom Lane wrote: > Ashwin Agrawal writes: > > In general, the variable LN_S is 'ln -s', however, LN_S changes to 'cp > > -pR' if configure finds the file system does not support symbolic > > links. > > I was playing w

Re: Confusing error message for REINDEX TABLE CONCURRENTLY

2019-06-04 Thread Ashwin Agrawal
On Mon, Jun 3, 2019 at 6:27 PM Michael Paquier wrote: > On Mon, Jun 03, 2019 at 04:53:48PM -0700, Ashwin Agrawal wrote: > > Please check if the attached patch addresses and satisfies all the points > > discussed so far in this thread. > > It looks to be so, please see be

Re: Removing a few more lseek() calls

2019-06-06 Thread Ashwin Agrawal
On Sat, Mar 30, 2019 at 2:14 AM Thomas Munro wrote: > Hello, > > Patch 0001 gets rid of the unconditional lseek() calls for SLRU I/O, > as a small follow-up to commit c24dcd0c. Patch 0002 gets rid of a few > places that usually do a good job of avoiding lseek() calls while > reading and writing

Re: heapam_index_build_range_scan's anyvisible

2019-06-10 Thread Ashwin Agrawal
On Fri, Jun 7, 2019 at 1:19 PM Robert Haas wrote: > I spent some time today studying heapam_index_build_range_scan and > quickly reached the conclusion that it's kind of a mess. > While at it might be helpful and better to also decouple HeapTuple dependency for IndexBuildCallback. Currently, all

Re: heapam_index_build_range_scan's anyvisible

2019-06-10 Thread Ashwin Agrawal
On Mon, Jun 10, 2019 at 2:56 PM Andres Freund wrote: > > Currently, all AM needs to build HeapTuple in > > index_build_range_scan function. I looked into all the callback functions > > and only htup->t_self is used from heaptuple in all the functions > (unless I > > missed something). So, if seem

  1   2   >