Re: A modest proposal: make parser/rewriter/planner inputs read-only

2025-04-15 Thread Ashwin Agrawal
On Sat, Apr 5, 2025 at 7:31 PM Andres Freund wrote: > Hi, > > On 2025-04-05 12:46:37 -0400, Tom Lane wrote: > > 1. Invent a way to make a particular memory context read-only > > after putting some data into it. > > > > 2. In debug builds, after we've built a tree that should be considered > > rea

Re: brininsert optimization opportunity

2023-11-25 Thread Ashwin Agrawal
On Sat, Nov 25, 2023 at 12:06 PM Tomas Vondra wrote: > I've done a bit more cleanup on the last version of the patch (renamed > the fields to start with bis_ as agreed, rephrased the comments / docs / > commit message a bit) and pushed. Thanks a lot Tomas for helping to drive the patch to compl

Re: Backends stalled in 'startup' state

2023-01-17 Thread Ashwin Agrawal
On Tue, Jan 17, 2023 at 4:52 PM Ashwin Agrawal wrote: > > We recently saw many backends (close to max_connection limit) get stalled > in 'startup' in one of the production environments for Greenplum (fork of > PostgreSQL). Tracing the reason, it was found all the tuples

Re: Backends stalled in 'startup' state

2022-09-27 Thread Ashwin Agrawal
On Thu, Sep 15, 2022 at 4:42 PM Ashwin Agrawal wrote: > > We recently saw many backends (close to max_connection limit) get stalled > in 'startup' in one of the production environments for Greenplum (fork of > PostgreSQL). Tracing the reason, it was found all the tuples

Re: pg_basebackup --create-slot-if-not-exists?

2022-09-22 Thread Ashwin Agrawal
On Wed, Sep 21, 2022 at 5:34 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Wednesday, September 21, 2022, Ashwin Agrawal > wrote: > >> Currently, pg_basebackup has >> --create-slot option to create slot if not already exists or >> --slot to use e

pg_basebackup --create-slot-if-not-exists?

2022-09-21 Thread Ashwin Agrawal
EXISTS kind of semantic. (Seems the limitation most likely is coming from CREATE REPLICATION SLOT protocol itself), Thoughts? -- *Ashwin Agrawal (VMware)*

Backends stalled in 'startup' state

2022-09-15 Thread Ashwin Agrawal
); + else + tableScan = table_beginscan(OldHeap, SnapshotAny, 0, (ScanKey) NULL); heapScan = (HeapScanDesc) tableScan; indexScan = NULL; -- 1] https://www.post

Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes

2022-01-03 Thread Ashwin Agrawal
On Wed, Dec 22, 2021 at 4:23 PM SATYANARAYANA NARLAPURAM < satyanarlapu...@gmail.com> wrote: > Hi Hackers, > > I am considering implementing RPO (recovery point objective) enforcement > feature for Postgres where the WAL writes on the primary are stalled when > the WAL distance between the primary

Re: storing an explicit nonce

2021-10-07 Thread Ashwin Agrawal
On Thu, Oct 7, 2021 at 12:12 PM Robert Haas wrote: > On Thu, Oct 7, 2021 at 2:52 PM Stephen Frost wrote: > > Assuming that's correct, and I don't see any reason to doubt it, then > > perhaps it would make sense to have the LSN be unencrypted and include > > it in the tweak as that would limit th

Re: vac_update_datfrozenxid will raise "wrong tuple length" if pg_database tuple contains toast attribute.

2020-11-30 Thread Ashwin Agrawal
On Sun, Nov 29, 2020 at 5:27 PM Michael Paquier wrote: > > One thing that strikes me as unwise is that we could run into similar > > problems with vac_update_relstats() in the future, and there have been > > recent talks about having more toast tables like for pg_class. That > > is not worth car

Re: [PoC] Non-volatile WAL buffer

2020-11-24 Thread Ashwin Agrawal
On Sun, Nov 22, 2020 at 5:23 PM Tomas Vondra wrote: > I'm not entirely sure whether the "pmemdax" (i.e. unpatched instance > with WAL on PMEM DAX device) is actually safe, but I included it anyway > to see what difference is. I am curious to learn more on this aspect. Kernels have provided supp

CREATE RULE may generate duplicate entries in pg_depend

2020-08-26 Thread Ashwin Agrawal
1 | n pg_rewrite | rtest_t5 |2618 | 16457 |0 | 1259 |16445 | 2 | n (3 rows) -- *Ashwin Agrawal (VMware)*

Re: language cleanups in code and docs

2020-08-20 Thread Ashwin Agrawal
On Wed, Jun 17, 2020 at 9:27 AM David Steele wrote: > On 6/17/20 12:08 PM, Magnus Hagander wrote: > > On Wed, Jun 17, 2020 at 4:15 PM Andrew Dunstan > > mailto:andrew.duns...@2ndquadrant.com>> > > > > > I'm not sure I like doing s/Black/Block/ here. It reads oddly. There > are > > too man

For standby pg_ctl doesn't wait for PM_STATUS_READY in presence of promote_trigger_file

2020-08-04 Thread Ashwin Agrawal
y) + if (!EnableHotStandby && !promote_trigger_file_exist) { AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY); #ifdef USE_SYSTEMD -- *Ashwin Agrawal (VMware)*

Re: SyncRepLock acquired exclusively in default configuration

2020-05-19 Thread Ashwin Agrawal
On Mon, May 18, 2020 at 7:41 PM Masahiko Sawada < masahiko.saw...@2ndquadrant.com> wrote: > This item is for PG14, right? If so I'd like to add this item to the > next commit fest. > Sure, add it to commit fest. Seems though it should be backpatched to relevant branches as well.

Re: pg_basebackup misses to report checksum error

2020-05-07 Thread Ashwin Agrawal
On Thu, May 7, 2020 at 10:25 AM Stephen Frost wrote: > Greetings, > > * Ashwin Agrawal (aagra...@pivotal.io) wrote: > > On Wed, May 6, 2020 at 3:02 PM Robert Haas > wrote: > > > On Wed, May 6, 2020 at 5:48 PM Ashwin Agrawal > wrote: > > > > If pg_bas

Re: pg_basebackup misses to report checksum error

2020-05-07 Thread Ashwin Agrawal
On Wed, May 6, 2020 at 3:02 PM Robert Haas wrote: > On Wed, May 6, 2020 at 5:48 PM Ashwin Agrawal wrote: > > If pg_basebackup is not able to read BLCKSZ content from file, then it > > just emits a warning "could not verify checksum in file "" block > >

pg_basebackup misses to report checksum error

2020-05-06 Thread Ashwin Agrawal
If pg_basebackup is not able to read BLCKSZ content from file, then it just emits a warning "could not verify checksum in file "" block X: read buffer size X and page size 8192 differ" currently but misses to error with "checksum error occurred". Only if it can read 8192 and checksum mismatch h

Re: SyncRepLock acquired exclusively in default configuration

2020-04-07 Thread Ashwin Agrawal
On Mon, Apr 6, 2020 at 2:14 PM Andres Freund wrote: > > How about we change it to this ? > > Hm. Better. But I think it might need at least a compiler barrier / > volatile memory load? Unlikely here, but otherwise the compiler could > theoretically just stash the variable somewhere locally (it's

Re: SyncRepLock acquired exclusively in default configuration

2020-04-06 Thread Ashwin Agrawal
es visible. > > Remove the race by using more appropriate test in syncrep.c > > > > Author: Asim Rama Praveen and Ashwin Agrawal > > Reported-by: Xin Zhang, Ashwin Agrawal, and Asim Rama Praveen > > Reviewed-by: Michael Paquier, Masahiko Sawada > > > &

Re: Make mesage at end-of-recovery less scary.

2020-03-23 Thread Ashwin Agrawal
On Mon, Mar 23, 2020 at 2:37 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2020-03-05 08:06, Kyotaro Horiguchi wrote: > > | [20866] LOG: replication terminated by primary server > > | [20866] DETAIL: End of WAL reached on timeline 1 at 0/30001C8. > > | [20866] FATAL: could

Re: Use LN_S instead of "ln -s" in Makefile

2020-02-14 Thread Ashwin Agrawal
On Fri, Feb 14, 2020 at 4:57 PM Tom Lane wrote: > Ashwin Agrawal writes: > > In general, the variable LN_S is 'ln -s', however, LN_S changes to 'cp > > -pR' if configure finds the file system does not support symbolic > > links. > > I was playing w

Use LN_S instead of "ln -s" in Makefile

2020-02-14 Thread Ashwin Agrawal
re for src/backend/Makefile. Greping for 'ln -s' revealed three places it's used. Attaching the patch to fix the same. From 60b3374de1e4d0fd143c2a7b3bf78893e32579af Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal Date: Fri, 14 Feb 2020 16:16:37 -0800 Subject: [PATCH v1] Use LN_S in

Re: Extracting only the columns needed for a query

2020-01-31 Thread Ashwin Agrawal
On Wed, Jan 15, 2020 at 7:54 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > For UPDATE, we need all of the columns in the table because of the > > table_lock() API's current expectation that the slot has all of the > > columns populated. If we want UPDATE to only need to insert the column > >

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2020-01-27 Thread Ashwin Agrawal
On Mon, Jan 27, 2020 at 4:47 PM Thomas Munro wrote: > OK, I kept only the small comment change from that little fixup patch, > and pushed this. > > > I had proposed as alternative way in initial email and also later, > > didn't receive comment on that, so re-posting. > > > typedef bool (*AMCheckF

Re: Should we rename amapi.h and amapi.c?

2019-12-23 Thread Ashwin Agrawal
On Sun, Dec 22, 2019 at 9:34 PM Michael Paquier wrote: > Hi all, > > I was working on some stuff for table AMs, and I got to wonder it we > had better rename amapi.h to indexam.h and amapi.c to indexam.c, so as > things are more consistent with table AM. It is a bit annoying to > name the files

Re: Start Walreceiver completely before shut down it on standby server.

2019-12-10 Thread Ashwin Agrawal
On Tue, Dec 10, 2019 at 3:06 AM jiankang liu wrote: > Start Walreceiver completely before shut down it on standby server. > > The walreceiver will be shut down, when read an invalid record in the > WAL streaming from master.And then, we retry from archive/pg_wal again. > > After that, we start wa

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-11-12 Thread Ashwin Agrawal
On Sun, Nov 10, 2019 at 8:21 PM Thomas Munro wrote: > I pushed the first two, Thank You! but on another read-through of the main patch > I didn't like the comments for CheckForSerializableConflictOut() or > the fact that it checks SerializationNeededForRead() again, after I > thought a bit abo

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-11-08 Thread Ashwin Agrawal
On Thu, Nov 7, 2019 at 8:44 PM Thomas Munro wrote: > On Thu, Aug 8, 2019 at 6:53 AM Ashwin Agrawal wrote: > >>> - I wonder if CheckForSerializableConflictOutNeeded() shouldn't have a > >>> portion of it's code as a static inline. In particular, it's

Missing error_context_stack = NULL in AutoVacWorkerMain()

2019-10-18 Thread Ashwin Agrawal
I am not sure if this causes any potential problems or not, but for consistency of code seems we are missing below. All other places in code where sigsetjmp() exists for top level handling has error_context_stack set to NULL. diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaste

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-04 Thread Ashwin Agrawal
On Fri, Oct 4, 2019 at 8:49 AM Tom Lane wrote: > Jacob Champion writes: > > On Fri, Oct 4, 2019 at 7:51 AM Tom Lane wrote: > >> I concur with Joe here. The reason why some of the existing > >> memset's use "false" is for symmetry with other places where we use > >> "memset(p, true, n)" to set

Re: heapam_index_build_range_scan's anyvisible

2019-09-25 Thread Ashwin Agrawal
ll warnings being treated as errors > : recipe for target 'verify_nbtree.o' failed > make[4]: *** [verify_nbtree.o] Error 1 > Thanks for reporting, I did indeed missed out contrib. Please find attached the v2 of the patch which includes the change required in contrib as well.

Re: Syntax highlighting for Postgres spec files

2019-09-19 Thread Ashwin Agrawal
I didn't try as waiting to see if for emacs as well shows up :-) Do we want to get these in src/tools/editors? On Thu, Sep 19, 2019 at 10:15 AM Ekin Dursun wrote: > Hi people, > > I have written language plugins for .spec files used in isolation tests. > They are available for Vim and Visual Stu

Re: Zedstore - compressed in-core columnar storage

2019-08-26 Thread Ashwin Agrawal
On Mon, Aug 26, 2019 at 5:36 AM Ashutosh Sharma wrote: > Thanks Ashwin and Heikki for your responses. I've one more query here, > > If BTree index is created on a zedstore table, the t_tid field of > Index tuple contains the physical tid that is not actually pointing to > the data block instead i

Re: Comment in ginpostinglist.c doesn't match code

2019-08-22 Thread Ashwin Agrawal
On Thu, Aug 22, 2019 at 1:14 AM Heikki Linnakangas wrote: > > The patch also includes a little unit test module to test this without > creating a 16 TB table. A whole new test module seems a bit like > overkill just for this, but clearly we were missing test coverage here. > And it will come hand

Re: Zedstore - compressed in-core columnar storage

2019-08-14 Thread Ashwin Agrawal
On Wed, Aug 14, 2019 at 2:51 AM Ashutosh Sharma wrote: > Hi Ashwin, > > I tried playing around with the zedstore code a bit today and there > are couple questions that came into my mind. > Great! Thank You. > > 1) Can zedstore tables be vacuumed? If yes, does VACUUM on zedstore > table set th

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-07 Thread Ashwin Agrawal
On Fri, Aug 2, 2019 at 4:56 PM Ashwin Agrawal wrote: > > On Wed, Jul 31, 2019 at 2:06 PM Andres Freund wrote: > >> Looking at the code as of master, we currently have: >> > > Super awesome feedback and insights, thank you! > > - PredicateLockTuple() calls S

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-02 Thread Ashwin Agrawal
ead. So, the flow would be AM calling CheckForSerializableConflictOut() as today and only if serializable in use will invoke the callback to check with AM if more work should be performed or not. Essentially HeapCheckForSerializableConflictOut() will become callback function instead. Due to void* callback argument a

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-07-31 Thread Ashwin Agrawal
On Wed, Jul 31, 2019 at 12:37 PM Ashwin Agrawal wrote: > > On Wed, Jul 31, 2019 at 10:55 AM Andres Freund wrote: > >> Hi, >> >> On 2019-07-31 10:42:50 -0700, Ashwin Agrawal wrote: >> > On Tue, Jul 30, 2019 at 2:58 PM Thomas Munro >> wrote: >>

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-07-31 Thread Ashwin Agrawal
On Wed, Jul 31, 2019 at 10:55 AM Andres Freund wrote: > Hi, > > On 2019-07-31 10:42:50 -0700, Ashwin Agrawal wrote: > > On Tue, Jul 30, 2019 at 2:58 PM Thomas Munro > wrote: > > > > > On Tue, Jun 25, 2019 at 6:02 AM Andres Freund > wrote: > > > &g

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-07-31 Thread Ashwin Agrawal
On Tue, Jul 30, 2019 at 2:58 PM Thomas Munro wrote: > On Tue, Jun 25, 2019 at 6:02 AM Andres Freund wrote: > > > - CheckForSerializableConflictOut() no more takes HeapTuple nor > > > buffer, instead just takes xid. Push heap specific parts from > > > CheckForSerializableConflictOut() into it

Re: heapam_index_build_range_scan's anyvisible

2019-07-30 Thread Ashwin Agrawal
On Tue, Jul 16, 2019 at 10:22 AM Andres Freund wrote: > Hi, > > On 2019-07-11 17:27:46 -0700, Ashwin Agrawal wrote: > > Please find attached the patch to remove IndexBuildCallback's dependency > on > > HeapTuple, as discussed. Changed to have the argument as ItemPoi

Re: heapam_index_build_range_scan's anyvisible

2019-07-11 Thread Ashwin Agrawal
On Mon, Jun 10, 2019 at 5:38 PM Ashwin Agrawal wrote: > > On Mon, Jun 10, 2019 at 2:56 PM Andres Freund wrote: > >> > Currently, all AM needs to build HeapTuple in >> > index_build_range_scan function. I looked into all the callback >> functions >> > an

Re: PGOPTIONS="-fh" make check gets stuck since Postgres 11

2019-07-10 Thread Ashwin Agrawal
On Wed, Jul 10, 2019 at 6:46 AM Tom Lane wrote: > Michael Paquier writes: > > On Wed, Jul 10, 2019 at 12:51:28PM +0530, Amit Kapila wrote: > >> It would be good if we can come up with something like that. It will > >> be helpful for zheap, where in some cases we get different row > >> ordering

Re: Declared but no defined functions

2019-07-08 Thread Ashwin Agrawal
On Sat, Jul 6, 2019 at 4:32 PM Masahiko Sawada wrote: > Indeed. I've tried to search again with the following script and got > more such functions. > > for func in `git ls-files | egrep "\w+\.h$" | xargs cat | egrep -v > "(^typedef)|(DECLARE)|(BKI)" | egrep "^(extern )*[\_0-9A-Za-z]+ > [\_\*0-9a-

Re: Comment typo in tableam.h

2019-07-08 Thread Ashwin Agrawal
On Sat, Jul 6, 2019 at 12:05 AM Amit Kapila wrote: > On Tue, Jul 2, 2019 at 1:00 AM Ashwin Agrawal wrote: > > Please find attached v2 of patch 1 without objectionable comment change. > v1 of patch 2 attaching here just for convenience, no modifications made to > it. > >

Re: C testing for Postgres

2019-07-02 Thread Ashwin Agrawal
On Mon, Jul 1, 2019 at 11:26 PM Michael Paquier wrote: > On Fri, Jun 28, 2019 at 09:42:54AM -0400, Adam Berlin wrote: > > If we were to use this tool, would the community want to vendor the > > framework in the Postgres repository, or keep it in a separate repository > > that produces a versioned

Re: Comment typo in tableam.h

2019-07-01 Thread Ashwin Agrawal
On Fri, Jun 28, 2019 at 1:47 PM Amit Kapila wrote: > On Mon, Jun 24, 2019 at 11:26 PM Ashwin Agrawal > wrote: > > > > On Mon, Jun 3, 2019 at 5:24 PM Ashwin Agrawal > wrote: > >> > >> There were few more minor typos I had collected for table am, passing &

Re: Zedstore - compressed in-core columnar storage

2019-07-01 Thread Ashwin Agrawal
On Thu, May 30, 2019 at 8:07 AM DEV_OPS wrote: > > it's really cool and very good progress, > > I'm interesting if SIDM/JIT will be supported > That's something outside of Zedstore work directly at least now. The intent is to work with current executor code or enhance it only wherever needed. If

Re: Zedstore - compressed in-core columnar storage

2019-07-01 Thread Ashwin Agrawal
On Sun, Jun 30, 2019 at 7:59 PM Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > From: Ashwin Agrawal [mailto:aagra...@pivotal.io] > > The objective is to gather feedback on design and approach to the same. > > The implementation has core basic pieces wor

Re: An out-of-date comment in nodeIndexonlyscan.c

2019-06-27 Thread Ashwin Agrawal
On Thu, Jun 27, 2019 at 4:33 PM Thomas Munro wrote: > Here's a patch I'd like to commit to fix the comment. We could look > at improving the actual code after > https://commitfest.postgresql.org/23/2169/ is done. > Change looks good to me. > I wonder if it might be possible to avoid page lock

Re: errbacktrace

2019-06-25 Thread Ashwin Agrawal
On Tue, Jun 25, 2019 at 4:08 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > New thread continuing from > < > https://www.postgresql.org/message-id/d4903af2-e7b7-b551-71f8-3e4a6bdc2...@2ndquadrant.com > >. > > Here is a extended version of Álvaro's patch that adds an errbacktrace(

Re: Comment typo in tableam.h

2019-06-24 Thread Ashwin Agrawal
On Mon, Jun 3, 2019 at 5:24 PM Ashwin Agrawal wrote: > There were few more minor typos I had collected for table am, passing them > along here. > > Some of the required callback functions are missing Assert checking (minor > thing), adding them in separate patch. > Curious

Remove HeapTuple and Buffer dependency for predicate locking functions

2019-06-24 Thread Ashwin Agrawal
ge-id/CAEepm%3D2QbqQ_%2BKQQCnhKukF6NEAeq4SqiO3Qxe%2BfHza5-H-jKA%40mail.gmail.com From aac57c17f078f869bf360677556842d58d5d33ea Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal Date: Fri, 21 Jun 2019 13:42:00 -0700 Subject: [PATCH v1] Remove HeapTuple dependency for predicate locking functions. Foll

Re: Extracting only the columns needed for a query

2019-06-16 Thread Ashwin Agrawal
On Sat, Jun 15, 2019 at 10:02 AM Tom Lane wrote: > > Approach B: after parsing and/or after planning > > If we wanted to do something about this, making the planner record > the set of used columns seems like the thing to do. We could avoid > the expense of doing it when it's not needed by setti

Re: heapam_index_build_range_scan's anyvisible

2019-06-10 Thread Ashwin Agrawal
On Mon, Jun 10, 2019 at 2:56 PM Andres Freund wrote: > > Currently, all AM needs to build HeapTuple in > > index_build_range_scan function. I looked into all the callback functions > > and only htup->t_self is used from heaptuple in all the functions > (unless I > > missed something). So, if seem

Re: heapam_index_build_range_scan's anyvisible

2019-06-10 Thread Ashwin Agrawal
On Fri, Jun 7, 2019 at 1:19 PM Robert Haas wrote: > I spent some time today studying heapam_index_build_range_scan and > quickly reached the conclusion that it's kind of a mess. > While at it might be helpful and better to also decouple HeapTuple dependency for IndexBuildCallback. Currently, all

Re: Removing a few more lseek() calls

2019-06-06 Thread Ashwin Agrawal
On Sat, Mar 30, 2019 at 2:14 AM Thomas Munro wrote: > Hello, > > Patch 0001 gets rid of the unconditional lseek() calls for SLRU I/O, > as a small follow-up to commit c24dcd0c. Patch 0002 gets rid of a few > places that usually do a good job of avoiding lseek() calls while > reading and writing

Re: Confusing error message for REINDEX TABLE CONCURRENTLY

2019-06-04 Thread Ashwin Agrawal
On Mon, Jun 3, 2019 at 6:27 PM Michael Paquier wrote: > On Mon, Jun 03, 2019 at 04:53:48PM -0700, Ashwin Agrawal wrote: > > Please check if the attached patch addresses and satisfies all the points > > discussed so far in this thread. > > It looks to be so, please see be

Re: Comment typo in tableam.h

2019-06-03 Thread Ashwin Agrawal
On Mon, Jun 3, 2019 at 6:24 PM Andres Freund wrote: > Hi, > > On 2019-06-03 18:21:56 -0700, Ashwin Agrawal wrote: > > On Mon, Jun 3, 2019 at 5:26 PM Andres Freund wrote: > > > > > Hi, > > > > > > Thanks for these! > > >

Re: Comment typo in tableam.h

2019-06-03 Thread Ashwin Agrawal
On Mon, Jun 3, 2019 at 5:26 PM Andres Freund wrote: > Hi, > > Thanks for these! > > On 2019-06-03 17:24:15 -0700, Ashwin Agrawal wrote: > > /* > >* Estimate the size of shared memory needed for a parallel scan > of this > > - * relati

Re: Comment typo in tableam.h

2019-06-03 Thread Ashwin Agrawal
There were few more minor typos I had collected for table am, passing them along here. Some of the required callback functions are missing Assert checking (minor thing), adding them in separate patch. From f32bdf5d0d3af5fd6ee6bf6430905f9c4bf5fefa Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal

Re: Confusing error message for REINDEX TABLE CONCURRENTLY

2019-06-03 Thread Ashwin Agrawal
On Tue, May 28, 2019 at 12:05 PM David Rowley wrote: > On Tue, 28 May 2019 at 01:23, Ashwin Agrawal wrote: > > I think we will need to separate out the NOTICE message for concurrent > and regular case. > > > > For example this doesn't sound correct > >

Re: Confusing error message for REINDEX TABLE CONCURRENTLY

2019-05-27 Thread Ashwin Agrawal
On Sun, May 26, 2019 at 6:43 PM Michael Paquier wrote: > As you mention for reindex_relation() no indexes <=> nothing to do, > still let's not rely on that. Instead of making the error message > specific to concurrent operations, I would suggest to change it to > "table foo has no indexes to rei

Confusing error message for REINDEX TABLE CONCURRENTLY

2019-05-24 Thread Ashwin Agrawal
ent case, make it consistent, less confusing and leave it there. Attaching the patch to just do that. Thoughts? From dc8119abe180cc14b0720c4de79495de4c62bf12 Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal Date: Fri, 24 May 2019 16:34:48 -0700 Subject: [PATCH v1] Avoid confusing error message for R

Re: Zedstore - compressed in-core columnar storage

2019-05-24 Thread Ashwin Agrawal
On Thu, May 23, 2019 at 7:30 PM Ajin Cherian wrote: > Hi Ashwin, > > - how to pass the "column projection list" to table AM? (as stated in > initial email, currently we have modified table am API to pass the > projection to AM) > > We were working on a similar columnar storage using pluggable

Re: Inconsistency between table am callback and table function names

2019-05-24 Thread Ashwin Agrawal
On Thu, May 23, 2019 at 4:32 PM Andres Freund wrote: > Hi, > > On 2019-05-14 12:11:46 -0700, Ashwin Agrawal wrote: > > Thank you. Please find the patch to rename the agreed functions. It would > > be good to make all consistent instead of applying exception to three >

Re: Pluggable Storage - Andres's take

2019-05-17 Thread Ashwin Agrawal
On Wed, May 15, 2019 at 11:54 AM Andres Freund wrote: > Attached is a prototype of a variation of this. I added a > table_tuple_tid_valid(TableScanDesc sscan, ItemPointer tid) > callback / wrapper. Currently it just takes a "plain" scan, but we could > add a separate table_beginscan variant too.

Re: Pluggable Storage - Andres's take

2019-05-17 Thread Ashwin Agrawal
On Tue, Apr 9, 2019 at 6:17 AM Heikki Linnakangas wrote: > On 08/04/2019 20:37, Andres Freund wrote: > > On 2019-04-08 15:34:46 +0300, Heikki Linnakangas wrote: > >> There's a little bug in index-only scan executor node, where it mixes > up the > >> slots to hold a tuple from the index, and from

Re: Create TOAST table only if AM needs

2019-05-17 Thread Ashwin Agrawal
On Fri, May 17, 2019 at 2:42 PM Robert Haas wrote: > In any case, it bears saying that > tableam is a remarkable accomplishment regardless of whatever > shortcomings it has in this area or elsewhere. > Big +1 to this.

Re: Pluggable Storage - Andres's take

2019-05-17 Thread Ashwin Agrawal
On Fri, May 17, 2019 at 12:54 PM Andres Freund wrote: > Hi, > > On 2019-05-15 23:00:38 -0700, Ashwin Agrawal wrote: > > Highlevel this looks good to me. Will look into full details tomorrow. > > Ping? > > I'll push the first of the patches soon, and unless you

Re: Create TOAST table only if AM needs

2019-05-17 Thread Ashwin Agrawal
On Fri, May 17, 2019 at 11:34 AM Andres Freund wrote: > Hi, > > On 2019-05-17 11:26:29 -0700, Ashwin Agrawal wrote: > > Currently TOAST table is always created (if needed based on data type > > properties) independent of table AM. How toasting is handled seems > >

Re: Create TOAST table only if AM needs

2019-05-17 Thread Ashwin Agrawal
On Fri, May 17, 2019 at 1:51 PM Andres Freund wrote: > > > /* > > > * Planner related callbacks for the heap AM > > @@ -2558,6 +2615,8 @@ static const TableAmRoutine heapam_methods = { > > > > .relation_estimate_siz

Create TOAST table only if AM needs

2019-05-17 Thread Ashwin Agrawal
tion to AM. From fcbf6b8187ef8389cf38cd185b709bcb6ef1152a Mon Sep 17 00:00:00 2001 From: Ashwin Agrawal Date: Fri, 10 May 2019 17:42:06 -0700 Subject: [PATCH 1/1] Create toast table only if AM needs it. All AM may not need toast table, hence add property to TableAmRoutine to define if uses toas

Re: Pluggable Storage - Andres's take

2019-05-15 Thread Ashwin Agrawal
On Wed, May 15, 2019 at 11:54 AM Andres Freund wrote: > Hi, > > On 2019-04-25 15:43:15 -0700, Andres Freund wrote: > > > > 3) nodeTidscan, skipping over too large tids > > >I think this should just be moved into the AMs, there's no need to > > >have this in nodeTidscan.c > > > > I think h

Re: Adding a test for speculative insert abort case

2019-05-15 Thread Ashwin Agrawal
sion after inserting the tuple in unique index but before completing the speculative insert. Hence, helps to create the condition easily. I believe order of index insertion is helping here that unique index is inserted and then non-unique index is inserted too. Attaching patch with the test

Re: Inconsistency between table am callback and table function names

2019-05-14 Thread Ashwin Agrawal
On Wed, May 8, 2019 at 5:05 PM Ashwin Agrawal wrote: > Not having consistency is the main aspect I wish to bring to > attention. Like for some callback functions the comment is > > /* see table_insert() for reference about parameters */ > void(*tuple_insert

Re: Inconsistency between table am callback and table function names

2019-05-14 Thread Ashwin Agrawal
On Mon, May 13, 2019 at 12:51 PM Robert Haas wrote: > On Fri, May 10, 2019 at 3:43 PM Ashwin Agrawal > wrote: > > Meant to stick the question mark in that email, somehow missed. Yes > > not planning to spend any time on it if objections. Here is the list > > of r

Re: Inconsistency between table am callback and table function names

2019-05-10 Thread Ashwin Agrawal
On Fri, May 10, 2019 at 10:51 AM Andres Freund wrote: > > Hi, > > On 2019-05-10 10:43:44 -0700, Ashwin Agrawal wrote: > > On Thu, May 9, 2019 at 8:52 AM Andres Freund wrote: > > > The changes necessary for tableam were already huge. Changing naming > > > sch

Re: Inconsistency between table am callback and table function names

2019-05-10 Thread Ashwin Agrawal
On Thu, May 9, 2019 at 8:52 AM Andres Freund wrote: > The changes necessary for tableam were already huge. Changing naming > schemes for functions that are used all over the backend (e.g. ~80 calls > to table_beginscan), and where there's other wrapper functions that also > widely used (237 calls

Re: Pluggable Storage - Andres's take

2019-05-09 Thread Ashwin Agrawal
On Wed, May 8, 2019 at 2:46 PM Andres Freund wrote: > > Hi, > > On 2019-05-07 23:18:39 -0700, Ashwin Agrawal wrote: > > On Mon, May 6, 2019 at 1:39 PM Ashwin Agrawal wrote: > > > Also wish to point out, while working on Zedstore, we realized that > > > T

Re: Inconsistency between table am callback and table function names

2019-05-08 Thread Ashwin Agrawal
On Wed, May 8, 2019 at 2:51 PM Andres Freund wrote: > > Hi, > > On 2019-05-08 00:32:22 -0700, Ashwin Agrawal wrote: > > The general theme for table function names seem to be > > "table_". For example table_scan_getnextslot() and its > > corresponding callba

Inconsistency between table am callback and table function names

2019-05-08 Thread Ashwin Agrawal
The general theme for table function names seem to be "table_". For example table_scan_getnextslot() and its corresponding callback scan_getnextslot(). Most of the table functions and callbacks follow mentioned convention except following ones table_beginscan table_endscan table_rescan table_f

Re: Pluggable Storage - Andres's take

2019-05-07 Thread Ashwin Agrawal
On Mon, May 6, 2019 at 1:39 PM Ashwin Agrawal wrote: > > Also wish to point out, while working on Zedstore, we realized that > TupleDesc from Relation object can be trusted at AM layer for > scan_begin() API. As for ALTER TABLE rewrite case (ATRewriteTables()), > catalog is up

Re: Pluggable Storage - Andres's take

2019-05-06 Thread Ashwin Agrawal
On Mon, May 6, 2019 at 7:14 AM Andres Freund wrote: > > Hi, > > On May 6, 2019 3:40:55 AM PDT, Rafia Sabih wrote: > >I was trying the toyam patch and on make check it failed with > >segmentation fault at > > > >static void > >toyam_relation_set_new_filenode(Relation rel, > > char persistence, > >

Match table_complete_speculative() code to comment

2019-04-30 Thread Ashwin Agrawal
Comment for table_complete_speculative() says /* * Complete "speculative insertion" started in the same transaction. If * succeeded is true, the tuple is fully inserted, if false, it's removed. */ static inline void table_complete_speculative(Relation rel, TupleTableSlot *slot,

Re: Pluggable Storage - Andres's take

2019-04-29 Thread Ashwin Agrawal
On Thu, Apr 25, 2019 at 3:43 PM Andres Freund wrote: > Hm. I think some of those changes would be a bit bigger than I initially > though. Attached is a more minimal fix that'd route > RelationGetNumberOfBlocksForFork() through tableam if necessary. I > think it's definitely the right answer for 1

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-29 Thread Ashwin Agrawal
On Thu, Apr 25, 2019 at 11:53 PM Michael Paquier wrote: > > On Thu, Apr 25, 2019 at 12:45:03PM -0400, Tom Lane wrote: > > I still remain concerned that invoking catalog lookups from fd.c is a darn > > bad idea, even if we have a fallback for it to work (for some value of > > "work") in non-transac

Re: Race conditions with checkpointer and shutdown

2019-04-29 Thread Ashwin Agrawal
On Mon, Apr 29, 2019 at 10:36 AM Tom Lane wrote: > > Can you try applying a1a789eb5ac894b4ca4b7742f2dc2d9602116e46 > to see if it fixes the problem for you? Yes, will give it a try on greenplum and report back the result. Have we decided if this will be applied to back branches?

Re: Race conditions with checkpointer and shutdown

2019-04-29 Thread Ashwin Agrawal
On Sat, Apr 27, 2019 at 5:57 PM Tom Lane wrote: > > I have spent a fair amount of time trying to replicate these failures > locally, with little success. I now think that the most promising theory > is Munro's idea in [1] that the walreceiver is hanging up during its > unsafe attempt to do erepor

Re: What is an item pointer, anyway?

2019-04-26 Thread Ashwin Agrawal
On Fri, Apr 26, 2019 at 2:19 PM Peter Geoghegan wrote: > itemid.h introduces the struct ItemIdData as follows: > > /* > * An item pointer (also called line pointer) on a buffer page > > Meanwhile, itemptr.h introduces the struct ItemPointerData as follows: > > /* > * ItemPointer: > * > * This

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-25 Thread Ashwin Agrawal
On Thu, Apr 25, 2019 at 9:45 AM Tom Lane wrote: > However, by that argument we should change all 3 of these functions to > set up the data. If we're eating the layering violation to the extent > of letting OpenTemporaryFile call into commands/tablespace, then there's > little reason for the othe

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-25 Thread Ashwin Agrawal
On Wed, Apr 24, 2019 at 5:48 PM Peter Geoghegan wrote: > On Wed, Apr 24, 2019 at 12:17 PM Tom Lane wrote: > > After a bit more thought it seemed like another answer would be to > > make all three of those functions assert that the caller did the > > right thing, as per attached. This addresses

Re: Zedstore - compressed in-core columnar storage

2019-04-24 Thread Ashwin Agrawal
On Tue, Apr 16, 2019 at 9:15 AM Tomas Vondra wrote: > > I'm not sure it's that clear cut, actually. Sure, it's not the usual > (block,item) pair so it's not possible to jump to the exact location, so > it's not the raw physical identifier as regular TID. But the data are > organized in a btree, w

Re: Regression test PANICs with master-standby setup on same machine

2019-04-24 Thread Ashwin Agrawal
On Wed, Apr 24, 2019 at 9:25 AM Andres Freund wrote: > The inability > to reasonably test master/standby setups on a single machine is pretty > jarring (yes, one can use basebackup tablespace maps - but that doesn't > work well for new tablespaces). +1 agree. Feature which can't be easily test

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Mon, Apr 15, 2019 at 12:50 PM Peter Geoghegan wrote: > On Mon, Apr 15, 2019 at 9:16 AM Ashwin Agrawal > wrote: > > Would like to know more specifics on this Peter. We may be having > different context on hybrid row/column design. > > I'm confused about how close yo

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Mon, Apr 15, 2019 at 11:18 AM Tomas Vondra wrote: > > Maybe. I'm not going to pretend I fully understand the internals. Does > that mean the container contains ZSUncompressedBtreeItem as elements? Or > just the plain Datum values? > First, your reading of code and all the comments/questions s

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Mon, Apr 15, 2019 at 10:33 AM Tomas Vondra wrote: > On Mon, Apr 15, 2019 at 09:29:37AM -0700, Ashwin Agrawal wrote: > > On Sun, Apr 14, 2019 at 9:40 AM Tomas Vondra > >wrote: > > > > On Thu, Apr 11, 2019 at 06:20:47PM +0300, Heikki Linnakangas wrote: &

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Sun, Apr 14, 2019 at 9:40 AM Tomas Vondra wrote: > On Thu, Apr 11, 2019 at 06:20:47PM +0300, Heikki Linnakangas wrote: > >On 11/04/2019 17:54, Tom Lane wrote: > >>Ashwin Agrawal writes: > >>>Thank you for trying it out. Yes, noticed for certain patterns >

Re: Zedstore - compressed in-core columnar storage

2019-04-15 Thread Ashwin Agrawal
On Sat, Apr 13, 2019 at 4:22 PM Peter Geoghegan wrote: > On Thu, Apr 11, 2019 at 6:06 AM Rafia Sabih > wrote: > > Reading about it reminds me of this work -- TAG column storage( > https://urldefense.proofpoint.com/v2/url?u=http-3A__www09.sigmod.org_sigmod_record_issues_0703_03.article-2Dgraefe.p

  1   2   >