[HACKERS] Parallel query fails on standby server

2016-03-08 Thread Ashutosh Sharma
n read-write mode during recovery");return false;}* *Solution:* Make use of a global variable "*InitializingParallelWorker"* to protect the check for *RecoveryInProgress()* when Parallel Worker is being Initialsed. PFA patch to fix the issue. With Regards, Ashutosh Sharma Enterpri

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-03-23 Thread Ashutosh Sharma
for all the scenarios discussed above and let me know your thoughts. With Regards, Ashutosh Sharma EnterpriseDB: *http://www.enterprisedb.com <http://www.enterprisedb.com>* On Sat, Feb 27, 2016 at 9:26 AM, Andres Freund wrote: > On February 26, 2016 7:55:18 PM PST, Amit Kapila >

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-03-26 Thread Ashutosh Sharma
nzeu9vut7tx6b-n1wyouwwfhd6...@mail.gmail.com>* Note: I am applying this patch on top of commit " *6150a1b08a9fe7ead2b25240be46dddeae9d98e1*". With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprisedb.com On Fri, Mar 25, 2016 at 9:29 AM, Amit Kapila wrote: > On Wed, Mar 23, 201

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-03-26 Thread Ashutosh Sharma
Sharma EnterpriseDB: http://www.enterprisedb.com <http://www.enterprisedb.com/> On Sat, Mar 26, 2016 at 9:31 PM, Ashutosh Sharma wrote: > Hi, > > I am getting some reject files while trying to apply " > *pinunpin-cas-5.patch*" attached with the thread, > > >

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-03-29 Thread Ashutosh Sharma
Hi, I am unable to revert 6150a1b0 on top of recent commit in the master branch. It seems like there has been some commit made recently that has got dependency on 6150a1b0. With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprisedb.com On Sun, Mar 27, 2016 at 5:45 PM, Andres Freund

Re: [HACKERS] "Some tests to cover hash_index"

2016-08-22 Thread Ashutosh Sharma
now no more visible w.r.t hash index after the WAL patch for hash index. Please have a look and let me know your thoughts. [1] - https://www.postgresql.org/message-id/CAA4eK1JOBX%3DYU33631Qh-XivYXtPSALh514%2BjR8XeD7v%2BK3r_Q%40mail.gmail.com With Regards, Ashutosh Sharma EnterpriseDB:http://www.ente

Re: [HACKERS] "Some tests to cover hash_index"

2016-08-22 Thread Ashutosh Sharma
Hi, I missed to attach the patch in my previous mail. Here i attach the patch. With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Tue, Aug 23, 2016 at 11:47 AM, Ashutosh Sharma wrote: > Hi All, > > I have reverified the code coverage for hash index code using

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-08-22 Thread Ashutosh Sharma
ge-id/CAE9k0PkNjryhSiG53mjnKFhi%2BMipJMjSa%3DYkH-UeW3bfr1HPJQ%40mail.gmail.com With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Some tests to cover hash_index"

2016-08-23 Thread Ashutosh Sharma
Hi, > Well, that change should be part of Amit's patch to add WAL logging, > not this patch, whose mission is just to improve test coverage. I have just removed the warning message from expected output file as i have performed the testing on Amit's latest patch that removes this warning message f

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-07 Thread Ashutosh Sharma
> Thanks to Ashutosh Sharma for doing the testing of the patch and > helping me in analyzing some of the above issues. Hi All, I would like to summarize the test-cases that i have executed for validating WAL logging in hash index feature. 1) I have mainly ran the pgbench test with read

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-14 Thread Ashutosh Sharma
%3Dbr9UrxMVn_rhWhKPLaHfEdM5A%40mail.gmail.com Please note that i am performing the test on latest patch for concurrent hash index and WAL log in hash index shared by Amit yesterday. With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprisedb.com On Wed, Sep 14, 2016 at 12:04 AM, Jesper Pedersen wrote

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-14 Thread Ashutosh Sharma
1299 #24 0x006940fe in main (argc=4, argv=0x2ac81f0) at main.c:228 Please let me know for any further inputs. [1]- https://www.postgresql.org/message-id/CAE9k0Pmxh-4NAr4GjzDDFHdBKDrKy2FV-Z%2B2Tp8vb2Kmxu%3D6zg%40mail.gmail.com With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprised

Re: [HACKERS] pageinspect: Hash index support

2016-09-20 Thread Ashutosh Sharma
/dev/null on line 46 Also, i think the USAGE for hash_metap() is refering to hash_metap_bytea(). Please correct it. I have just started reviewing the patch, will keep on posting my comments upon further review. Thanks. With Regards, Ashutosh Sharma. EnterpriseDB: http://www.enterprisedb.com On T

Re: [HACKERS] pageinspect: Hash index support

2016-09-21 Thread Ashutosh Sharma
defined page i guess we should error out instead of reading the page because it is quite possible that the page would be corrupted. Please let me know your thoughts on this. 5). I think we have added enough functions to show the page level statistics but not the index level statistics like the t

Re: [HACKERS] pageinspect: Hash index support

2016-09-21 Thread Ashutosh Sharma
gory shouldn't it error out. With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-25 Thread Ashutosh Sharma
BW9w%40mail.gmail.com [4] - https://www.postgresql.org/message-id/CAMkU%3D1xRt8jBBB7g_7K41W00%3Dbr9UrxMVn_rhWhKPLaHfEdM5A%40mail.gmail.com With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

2016-09-25 Thread Ashutosh Sharma
on discussing this patch further. > Please correct me if this assessment does not match your expectations. Thanks for the update. I am absolutely OK with it. I feel it would be a good idea to review "Exclude additional directories in pg_basebackup" which also addresses the issue r

[HACKERS] dead or outdated URLs found in win32.h

2017-10-06 Thread Ashutosh Sharma
/8fskxacy(v=vs.80).aspx -- outdated http://msdn.microsoft.com/en-us/library/a90k134d(v=vs.80).aspx -- outdated https://msdn.microsoft.com/en-gb/library/aa489609.aspx -- outdated -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql

[HACKERS] Microvacuum support for Hash Index

2016-10-24 Thread Ashutosh Sharma
Hi All, I have added a microvacuum support for hash index access method and attached is the v1 patch for the same. The patch basically takes care of the following things: 1. Firstly, it changes the marking of dead tuples from 'tuple-at-a-time' to 'page-at-a-time' during hash index scan. For this

Re: [HACKERS] Microvacuum support for Hash Index

2016-10-30 Thread Ashutosh Sharma
Hi Amit, Thanks for showing your interest and reviewing my patch. I have started looking into your review comments. I will share the updated patch in a day or two. With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprisedb.com On Fri, Oct 28, 2016 at 4:42 PM, Amit Kapila wrote: >

Re: [HACKERS] Microvacuum support for Hash Index

2016-11-01 Thread Ashutosh Sharma
Hi, > While replaying the delete/vacuum record on standby, it can conflict > with some already running queries. Basically the replay can remove > some row which can be visible on standby. You need to resolve > conflicts similar to what we do in btree delete records (refer > btree_xlog_delete).

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

2016-11-02 Thread Ashutosh Sharma
Hi, I have started with the review for this patch and would like to share some of my initial review comments that requires author's attention. 1) I am getting some trailing whitespace errors when trying to apply this patch using git apply command. Following are the error messages i am getting. [

Re: [HACKERS] Applying XLR_INFO_MASK correctly when looking at WAL record information

2016-11-03 Thread Ashutosh Sharma
Hi, > What about the patch attached to make things more consistent? I have reviewed this patch. It does serve the purpose and looks sane to me. I am marking it as ready for committer. With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mail

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

2016-11-08 Thread Ashutosh Sharma
e an opinion from > committer or others as well before adding this target. What do you > say? Ok. We can do that. I have verified the updated v2 patch. The patch looks good to me. I am marking it as ready for committer. Thanks. With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprisedb.c

Re: [HACKERS] Microvacuum support for Hash Index

2016-11-10 Thread Ashutosh Sharma
to share you a next version of patch for supporting microvacuum in hash index. With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [sqlsmith] Parallel worker crash on seqscan

2016-11-21 Thread Ashutosh Sharma
latest commit in master branch and could not reproduce here as well. Amit (included in this email thread) has also tried it once and he was also not able to reproduce it. Could you please let me know if there is something more that needs to be done in order to reproduce it other than what you have

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-17 Thread Ashutosh Sharma
>> > Good point. >> > Please find graph of mean and errors in attachment. >> >> So ... no difference? > > > Yeah, nothing surprising. It's just another graph based on the same data. > I wonder how pgxact-align-3 would work on machine of Ashutosh Sharma

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-20 Thread Ashutosh Sharma
gt;> point. Should be simple enough in gnuplot ... > >> > > >> > Good point. > >> > Please find graph of mean and errors in attachment. > >> > >> So ... no difference? > > > > > > Yeah, nothing surprising. It's just another

Re: [HACKERS] pageinspect: Hash index support

2017-02-21 Thread Ashutosh Sharma
Thanks for reporting it. This is because of incorrect data typecasting. Attached is the patch that fixes this issue. On Tue, Feb 21, 2017 at 2:58 PM, Mithun Cy wrote: > On Fri, Feb 10, 2017 at 1:06 AM, Robert Haas > wrote: > > > Alright, committed with a little further hacking. > > I did pull t

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-21 Thread Ashutosh Sharma
On Tue, Feb 21, 2017 at 4:21 PM, Alexander Korotkov wrote: > > Hi, Ashutosh! > On Mon, Feb 20, 2017 at 1:20 PM, Ashutosh Sharma > wrote: >> >> Following are the pgbench results for read-write workload, I got with >> pgxact-align-3 patch. The results are fo

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-21 Thread Ashutosh Sharma
On Tue, Feb 21, 2017 at 5:52 PM, Alexander Korotkov wrote: > On Tue, Feb 21, 2017 at 2:37 PM, Andres Freund wrote: >> >> On 2017-02-21 16:57:36 +0530, Ashutosh Sharma wrote: >> > Yes, there is still some regression however it has come down by a >> > small margi

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-23 Thread Ashutosh Sharma
) CPU E7- 8830 @ 2.13GHz Also, Excel sheet (results-readwrite-300-SF) containing the results for all the 3 runs is attached. -- With Regards, Ashutosh Sharma EnterpriseDB:*http://www.enterprisedb.com <http://www.enterprisedb.com/>* On Thu, Feb 23, 2017 at 2:44 AM, Alvaro Herrera wrote

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-23 Thread Ashutosh Sharma
On Fri, Feb 24, 2017 at 10:41 AM, Simon Riggs wrote: > On 24 February 2017 at 04:41, Ashutosh Sharma wrote: >> >> Okay. As suggested by Alexander, I have changed the order of reading and >> doing initdb for each pgbench run. With these changes, I got following >> resul

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-28 Thread Ashutosh Sharma
Hi, On Fri, Feb 24, 2017 at 12:22 PM, Ashutosh Sharma wrote: On Fri, Feb 24, 2017 at 10:41 AM, Simon Riggs wrote: > > On 24 February 2017 at 04:41, Ashutosh Sharma > wrote: > >> > >> Okay. As suggested by Alexander, I have changed the order of reading and > &g

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-28 Thread Ashutosh Sharma
On Tue, Feb 28, 2017 at 11:44 PM, Simon Riggs wrote: > On 28 February 2017 at 11:34, Ashutosh Sharma > wrote: > > >> So, Here are the pgbench results I got with ' >> *reduce_pgxact_access_AtEOXact.v2.patch*' on a read-write workload. >> > > Tha

Re: [HACKERS] Should we cacheline align PGXACT?

2017-03-01 Thread Ashutosh Sharma
On Wed, Mar 1, 2017 at 5:29 PM, Simon Riggs wrote: > > On 1 March 2017 at 04:50, Ashutosh Sharma wrote: >> >> On Tue, Feb 28, 2017 at 11:44 PM, Simon Riggs wrote: >>> >>> On 28 February 2017 at 11:34, Ashutosh Sharma wrote: >>> >>

[HACKERS] Parallel seq. plan is not coming against inheritance or partition table

2017-03-04 Thread Ashutosh Sharma
ving zero heap pages didn't get parallel workers other childRels that was good enough to go for Parallel Seq Scan had to go for normal seq scan which could be costlier. Fix: Attached is the patch that fixes this issue. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterpri

Re: [HACKERS] Parallel seq. plan is not coming against inheritance or partition table

2017-03-05 Thread Ashutosh Sharma
s not aware of Parallel Append. Thanks. With Regards, Ashutosh Sharma EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel seq. plan is not coming against inheritance or partition table

2017-03-06 Thread Ashutosh Sharma
f-condition is satisfied. *if (heap_pages < (BlockNumber) min_parallel_table_scan_size && index_pages < (BlockNumber) min_parallel_index_scan_size && rel->reloptkind == RELOPT_BASEREL)return 0;* -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: [HACKERS] increasing the default WAL segment size

2017-03-06 Thread Ashutosh Sharma
: macro redefinition c:\users\ashu\postgresql\src\include\pg_config_manual.h20 Apart from these, I am not having any comments as of now. I am still validating the patch on Windows. If I find any issues i will update it. -- With Regards, Ashutosh Sharma. EnterpriseDB: http://www.enterprisedb.co

Re: [HACKERS] Parallel seq. plan is not coming against inheritance or partition table

2017-03-06 Thread Ashutosh Sharma
On Tue, Mar 7, 2017 at 11:18 AM, Amit Kapila wrote: > On Tue, Mar 7, 2017 at 10:22 AM, Ashutosh Sharma > wrote: >>> I also think that commit didn't intend to change the behavior, >>> however, the point is how sensible is it to keep such behavior after >>> P

Re: [HACKERS] Supporting huge pages on Windows

2017-03-08 Thread Ashutosh Sharma
20606.288995 [1] - https://msdn.microsoft.com/en-IN/library/ms190730.aspx -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Thu, Feb 23, 2017 at 12:59 PM, Tsunakawa, Takayuki wrote: > From: Amit Kapila [mailto:amit.kapil...@gmail.com] >> > Hmm, the large-page require

Re: [HACKERS] Should we cacheline align PGXACT?

2017-03-10 Thread Ashutosh Sharma
commit. Amit Langote, reviewed by David Fetter -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-12 Thread Ashutosh Sharma
ving so many INSERT statements in test.sql file. I think it would be better to replace it with single SQL statement. Thanks. [1]- https://www.postgresql.org/message-id/CAA4eK1KibVzgVETVay0%2BsiVEgzaXnP5R21BdWiK9kg9wx2E40Q%40mail.gmail.com [2]- https://www.postgresql.org/message-id/CAE9k0PkRSyzx8d

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-12 Thread Ashutosh Sharma
btree_index" postgres=# SELECT * FROM bt_page_items('btree_index', 1024) LIMIT 1; ERROR: block number out of range 5) Code duplication in bt_page_items() and bt_page_items_bytea() needs to be handled. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: [HACKERS] WAL Consistency checking for hash indexes

2017-03-13 Thread Ashutosh Sharma
Q87rKYzmYQ%40mail.gmail.com -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Wed, Mar 8, 2017 at 2:32 PM, Kuntal Ghosh wrote: > On Fri, Mar 3, 2017 at 9:44 AM, Amit Kapila wrote: >> On Tue, Feb 28, 2017 at 11:06 AM, Kuntal Ghosh >> wrote: >>> Hel

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-13 Thread Ashutosh Sharma
(0/304EDE0,-25462,1,220,7432,8192,8192,4,0) (1 row) I think pd_checksum in PageHeaderData is defined as uint16 (0 to 65,535) whereas in SQL function for page_header it is defined as smallint (-32768 to +32767). -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-14 Thread Ashutosh Sharma
AD_TUPLES' flag which got added as a part of Microvacuum patch is attached with this mail. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Wed, Feb 1, 2017 at 10:30 AM, Michael Paquier wrote: > On Sat, Jan 28, 2017 at 8:02 PM, Amit Kapila wrote: >> On

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-14 Thread Ashutosh Sharma
o share you the updated > patch asap. > >> >> >> On Tue, Feb 14, 2017 at 8:27 AM, Ashutosh Sharma >> wrote: >>> >>> 1) 0001-Rewrite-hash-index-scans-to-work-a-page-at-a-time.patch: this >>> patch rewrites the hash index scan module to work in

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-14 Thread Ashutosh Sharma
On Mar 14, 2017 5:37 PM, "Alvaro Herrera" wrote: Ashutosh Sharma wrote: > Yes. But, as i said earlier I am getting negative checksum value for > page_header as well. Isn't that wrong. For eg. When I debug the > following query, i could pd_checksum value as '4007

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Ashutosh Sharma
the comments needs some work. Thanks for your that suggestion... I spent a lot of time thinking on this and also had a small discussion with Amit but could not find any issue with taking cleanup lock on modified page instead of primary bucket page.I had to do some decent code changes for this. Atta

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Ashutosh Sharma
> > I think one possibility is to get it using > indexrel->rd_index->indrelid in _hash_doinsert(). > Thanks. I have tried the same in the v7 patch shared upthread. > >> >> But they're not called delete records in a hash index. The function >> is called hash_xlog_vacuum_one_page. The correspondi

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Ashutosh Sharma
On Wed, Mar 15, 2017 at 9:28 PM, Robert Haas wrote: > On Wed, Mar 15, 2017 at 11:37 AM, Ashutosh Sharma > wrote: >>> +/* Get RelfileNode from relation OID */ >>> +rel = relation_open(htup->t_tableOid, NoLock); >>> +rnode = rel->r

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Ashutosh Sharma
t; this needs reformatting, but it's oddly narrow. Corrected. > > I suggest changing the header comment of > hash_xlog_vacuum_get_latestRemovedXid like this: > > + * Get the latestRemovedXid from the heap pages pointed at by the index > + * tuples being deleted. See also btree_xlog_del

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Ashutosh Sharma
On Mar 16, 2017 7:49 AM, "Robert Haas" wrote: On Wed, Mar 15, 2017 at 4:31 PM, Robert Haas wrote: > On Wed, Mar 15, 2017 at 3:54 PM, Ashutosh Sharma wrote: >> Changed as per suggestions. Attached v9 patch. Thanks. > > Wow, when do you sleep? Will have a lo

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-16 Thread Ashutosh Sharma
On Thu, Mar 16, 2017 at 11:11 AM, Amit Kapila wrote: > On Wed, Mar 15, 2017 at 9:23 PM, Ashutosh Sharma > wrote: >> >>> >>> Few other comments: >>> 1. >>> + if (ndeletable > 0) >>> + { >>> + /* N

Re: [HACKERS] WAL Consistency checking for hash indexes

2017-03-16 Thread Ashutosh Sharma
-- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Wed, Mar 15, 2017 at 11:27 AM, Kuntal Ghosh wrote: > On Wed, Mar 15, 2017 at 12:32 AM, Robert Haas wrote: >> On Mon, Mar 13, 2017 at 10:36 AM, Ashutosh Sharma >> wrote: >>> Couple of review c

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-16 Thread Ashutosh Sharma
DELETE? We might want to log the clear of flag along with > WAL record for XLOG_HASH_DELETE. > Yes, it should be cleared. I completely missed this part in a hurry. Thanks for informing. I have taken care of it in the attached v2 patch. -- With Regards, Ashutosh Sharma EnterpriseDB:http://w

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-16 Thread Ashutosh Sharma
On Fri, Mar 17, 2017 at 8:20 AM, Amit Kapila wrote: > On Thu, Mar 16, 2017 at 9:39 PM, Ashutosh Sharma > wrote: >>>> >>> >>> Don't you think, we should also clear it during the replay of >>> XLOG_HASH_DELETE? We might want to log the clear of f

Re: [HACKERS] WAL Consistency checking for hash indexes

2017-03-17 Thread Ashutosh Sharma
On Fri, Mar 17, 2017 at 9:03 AM, Amit Kapila wrote: > On Thu, Mar 16, 2017 at 1:15 PM, Ashutosh Sharma > wrote: >> Hi, >> >> Attached is the patch that allows WAL consistency tool to mask >> 'LH_PAGE_HAS_DEAD_TUPLES' flag in hash index. The flag got ad

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-17 Thread Ashutosh Sharma
On Fri, Mar 17, 2017 at 6:13 PM, Amit Kapila wrote: > On Fri, Mar 17, 2017 at 12:27 PM, Ashutosh Sharma > wrote: >> On Fri, Mar 17, 2017 at 8:20 AM, Amit Kapila wrote: >> >> As I said in my previous e-mail, I think you need >>> to record clearing of this flag

Re: [HACKERS] pageinspect and hash indexes

2017-03-17 Thread Ashutosh Sharma
sn't it unhelpful to have the > pageinspect module throw errors, rather than returning a dummy value to > indicate there was an error? Well, this is something not specific to hash index. So, I have no answer :) -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: [HACKERS] pageinspect and hash indexes

2017-03-18 Thread Ashutosh Sharma
On Sat, Mar 18, 2017 at 1:34 PM, Amit Kapila wrote: > On Sat, Mar 18, 2017 at 12:12 AM, Ashutosh Sharma > wrote: >> On Fri, Mar 17, 2017 at 10:54 PM, Jeff Janes wrote: >>> While trying to figure out some bloating in the newly logged hash indexes, >>> I'm lo

Re: [HACKERS] ANALYZE command progress checker

2017-03-18 Thread Ashutosh Sharma
ECT viewname, definition FROM pg_views WHERE schemaname <> 'information_schema' ORDER BY viewname; I am still reviewing your patch and doing some testing. Will update if i find any issues. Thanks. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Fri, Mar 17, 2017 at 3:16 P

Re: [HACKERS] ANALYZE command progress checker

2017-03-19 Thread Ashutosh Sharma
and ANALYZE. This may be expanded in the future. 3) I think above needs to be rephrased. Something like...Currently, the supported progress reporting commands are 'VACUUM' and 'ANALYZE'. Moreover, I also ran your patch on Windows platform and didn't find a

Re: [HACKERS] pageinspect and hash indexes

2017-03-20 Thread Ashutosh Sharma
On Mon, Mar 20, 2017 at 9:31 AM, Amit Kapila wrote: > On Sat, Mar 18, 2017 at 5:13 PM, Ashutosh Sharma > wrote: >> On Sat, Mar 18, 2017 at 1:34 PM, Amit Kapila wrote: >>> On Sat, Mar 18, 2017 at 12:12 AM, Ashutosh Sharma >>> wrote: >>>> On Fri, M

[HACKERS] comments in hash_alloc_buckets

2017-03-20 Thread Ashutosh Sharma
LCKSZ); Attached is the patch that corrects above comment. Thanks. [1] - https://www.postgresql.org/message-id/CAMkU%3D1y6NjKmqbJ8wLMhr%3DF74WzcMALYWcVFhEpm7i%3DmV%3DXsOg%40mail.gmail.com -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com corrected_comments_hash_alloc_buckets.

Re: [HACKERS] pageinspect and hash indexes

2017-03-20 Thread Ashutosh Sharma
On Mon, Mar 20, 2017 at 6:53 PM, Ashutosh Sharma wrote: > On Mon, Mar 20, 2017 at 9:31 AM, Amit Kapila wrote: >> On Sat, Mar 18, 2017 at 5:13 PM, Ashutosh Sharma >> wrote: >>> On Sat, Mar 18, 2017 at 1:34 PM, Amit Kapila >>> wrote: >>>> On S

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-21 Thread Ashutosh Sharma
roduce the issue on my local machine using the test script you shared. Could you please check with the attached patch if you are still seeing the issue. Thanks in advance. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com corrected_block_id_reference_in_hash_vac

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-21 Thread Ashutosh Sharma
rdAssemble, it first adds all the data assciated with registered buffers into the WAL record followed by the main data (). Hence, the WAL record in btree and hash are organised differently. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com -- Sent via pgsql-hackers

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-22 Thread Ashutosh Sharma
Hi, On Wed, Mar 22, 2017 at 8:41 AM, Amit Kapila wrote: > On Tue, Mar 21, 2017 at 11:49 PM, Ashutosh Sharma > wrote: >>> >>> I can confirm that that fixes the seg faults for me. >> >> Thanks for confirmation. >> >>> >>> Did you me

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-22 Thread Ashutosh Sharma
experimentation. The >> detail of non-default GUC params and pgbench command are mentioned in >> the result sheet. I also did the benchmarking with unique values at >> 300 and 1000 scale factor and its results are provided in >> 'results-unique-values-default-ff'. >

Re: [HACKERS] pageinspect and hash indexes

2017-03-22 Thread Ashutosh Sharma
RRUPTED), > + errmsg("index table contains empty page"))); > > > Do we want to give a separate message for EMPTY and NEW pages? Isn't > it better that the same error message can be given for both of them as > from user perspective there is not much difference between b

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-03-23 Thread Ashutosh Sharma
tly better than HEAD, with 7 and 10 SP(s) we do see regression with patch. Therefore, I think the threshold value of 4 for number of subtransactions considered in the patch looks fine to me. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Tue, Mar 21, 2017 at 6:1

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-23 Thread Ashutosh Sharma
On Thu, Mar 23, 2017 at 9:17 AM, Amit Kapila wrote: > > On Thu, Mar 23, 2017 at 8:43 AM, Amit Kapila wrote: > > On Wed, Mar 22, 2017 at 3:39 PM, Ashutosh Sharma > > wrote: > >> Hi, > >> > >> On Wed, Mar 22, 2017 at 8:41 AM, Amit Kapila > &g

Re: [HACKERS] pageinspect and hash indexes

2017-03-23 Thread Ashutosh Sharma
e if it makes any difference to user. > okay, I have now anyways removed the check for PageIsEmpty. Please refer to the attached '0002 allow_pageinspect_handle_UNUSED_hash_pages.patch' Also, I have attached '0001-Mark-freed-overflow-page-as-UNUSED-pagetype-v2.patch' that handles y

Re: [HACKERS] Add pgstathashindex() to get hash index table statistics.

2017-03-23 Thread Ashutosh Sharma
ow the number of unused pages in hash index. Please find the attached patch for the same. Thanks. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com From e3b59fa85f16d6d15be5360e85b7faf63e8683a9 Mon Sep 17 00:00:00 2001 From: ashu Date: Thu, 23 Mar 2017 23:02:26 +0530 S

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-23 Thread Ashutosh Sharma
On Thu, Mar 23, 2017 at 8:29 PM, Jesper Pedersen wrote: > Hi, > > On 03/22/2017 09:32 AM, Ashutosh Sharma wrote: >> >> Done. Please refer to the attached v2 version of patch. >> > > Thanks. > >>>> 1) 0001-Rewrite-hash-index-scans-to-work-a-page-at

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-23 Thread Ashutosh Sharma
> Hi, > > On 03/23/2017 02:11 PM, Ashutosh Sharma wrote: >> >> On Thu, Mar 23, 2017 at 8:29 PM, Jesper Pedersen >> wrote: >>> >>> 0001v2: >>> >>> In hashgettuple() you can remove the 'currItem' and 'offnum' fro

Re: [HACKERS] pageinspect and hash indexes

2017-03-23 Thread Ashutosh Sharma
On Fri, Mar 24, 2017 at 9:21 AM, Amit Kapila wrote: > On Thu, Mar 23, 2017 at 6:46 PM, Ashutosh Sharma > wrote: >>> >>> Oh, okay, but my main objection was that we should not check hash page >>> type (hasho_flag) without ensuring whether it is a hash page. Ca

Re: [HACKERS] pageinspect and hash indexes

2017-03-23 Thread Ashutosh Sharma
On Fri, Mar 24, 2017 at 9:46 AM, Ashutosh Sharma wrote: > On Fri, Mar 24, 2017 at 9:21 AM, Amit Kapila wrote: >> On Thu, Mar 23, 2017 at 6:46 PM, Ashutosh Sharma >> wrote: >>>> >>>> Oh, okay, but my main objection was that we should not check hash page

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-23 Thread Ashutosh Sharma
gt; _hash_vacuum_one_page() >>> { >>> .. >>> deletable[ndeletable++] = offnum; >>> tuples_removed += 1;-- >>> >> >> Yes, I think 'ndeletable' alone should be fine. >> > > I think it would have been probably okay to use *in

Re: [HACKERS] Supporting huge pages on Windows

2017-03-24 Thread Ashutosh Sharma
this patch. Do you know >>> when you'll have a chance to take a look? >>> >> >> I have provided my feedback and I could not test it on my machine. I >> think Ashutosh Sharma has tested it. I can give it another look, but >> not sure if it helps. >

Re: [HACKERS] pageinspect and hash indexes

2017-03-24 Thread Ashutosh Sharma
Thanks for reviewing my patch. I have removed the extra white space. Attached are both the patches. >>> >>> Sorry, I have mistakenly attached wrong patch. Here are the correct >>> set of patches. >> >> The latest version of patches looks fine to me. > > I don't really like 0002. What abo

Re: [HACKERS] Add pgstathashindex() to get hash index table statistics.

2017-03-25 Thread Ashutosh Sharma
On Sat, Mar 25, 2017 at 11:02 AM, Amit Kapila wrote: > On Thu, Mar 23, 2017 at 11:24 PM, Ashutosh Sharma > wrote: >> Hi, >> >> On Tue, Feb 7, 2017 at 9:23 AM, Robert Haas wrote: >>> On Mon, Feb 6, 2017 at 10:40 PM, Amit Kapila >>> wrote: >

Re: [HACKERS] comments in hash_alloc_buckets

2017-03-25 Thread Ashutosh Sharma
>> While working on - [1], I realised that the following comments in >> _hash_alloc_buckets() needs to be corrected. >> >> /* >> * Initialize the freed overflow page. Just zeroing the page won't work, >> * See _hash_freeovflpage for similar usage. >> */ >> _hash_pageinit(pag

Re: [HACKERS] Add pgstathashindex() to get hash index table statistics.

2017-03-25 Thread Ashutosh Sharma
illustrates this point, >> > > oh, is it a page where all the items have been deleted and no new > items have been inserted? Yes, it is a page from where items have been removed and no new insertion has happened. The reason why I have told that place is > not appropriate is

Re: [HACKERS] [sqlsmith] Failed assertion in _hash_kill_items/MarkBufferDirtyHint

2017-03-27 Thread Ashutosh Sharma
first deal with all the killed items but we do this without releasing lock and pin on the current page. Hence, with SELECT queries this crash is not visible. The attached patch fixes this. But, please note that all these changes will get removed with the patch for page scan mode - [1]. [1] - ht

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-27 Thread Ashutosh Sharma
afeguards against similar cases. I have added similar check for hash_kill_items() as well. > > This is not a full review, but I'm out of time for the moment. No worries. I will be ready for your further review comments any time. Thanks for the review. -- With Regards, Ashutosh Sharm

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-27 Thread Ashutosh Sharma
y server. If there is any inconsistent block on standby the tool would probably terminate the recovery process and you would see following message in the server logfile. "inconsistent page found, rel %u/%u/%u, forknum %u, blkno %u" -- With Regards, Ashutosh Sharma EnterpriseDB:http:

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-27 Thread Ashutosh Sharma
On Mar 27, 2017 22:25, "Robert Haas" wrote: On Fri, Mar 24, 2017 at 3:49 AM, Amit Kapila wrote: > On Fri, Mar 24, 2017 at 12:25 PM, Ashutosh Sharma wrote: >>> >>> I think it would have been probably okay to use *int* for ntuples as >>> that matches wit

Re: [HACKERS] [sqlsmith] Failed assertion in _hash_kill_items/MarkBufferDirtyHint

2017-03-27 Thread Ashutosh Sharma
On Mar 28, 2017 00:00, "Andreas Seltenreich" wrote: Ashutosh Sharma writes: >> TRAP: FailedAssertion("!(LWLockHeldByMe(((LWLock*) (&(bufHdr)->content_lock", File: "bufmgr.c", Line: 3397) > Thanks for reporting this problem. Could you please let

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-29 Thread Ashutosh Sharma
I think you should consider refactoring this so that it doesn't need >> to use goto. Maybe move the while (offnum <= maxoff) logic into a >> helper function and have it return itemIndex. If itemIndex == 0, you >> can call it again. >> > > okay, Added a helper function for _hash_readpage(). Please

[HACKERS] inconsistent page found on STANDBY server

2017-03-30 Thread Ashutosh Sharma
while registering data for xlog record 'XLOG_HASH_UPDATE_META_PAGE' we are not passing the correct length of data being registered and therefore, data (xl_hash_update_meta_page) is not completely recorded into the wal record. Fix: === Attached patch fixes this issue. -- With Regards, Ashut

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-31 Thread Ashutosh Sharma
> the tuple-forming part, which is exactly the same in both cases. > > It also adds the P_ISMETA(opaque) check to the original function, which > seems like a useful defense against page written to a different place (which > is essentially the issue I was originally investigating). &

Re: [HACKERS] [sqlsmith] Failed assertion in _hash_kill_items/MarkBufferDirtyHint

2017-03-31 Thread Ashutosh Sharma
On Sat, Apr 1, 2017 at 1:08 AM, Robert Haas wrote: > On Mon, Mar 27, 2017 at 5:39 AM, Ashutosh Sharma > wrote: >> Thanks for reporting this problem. Could you please let me know on for >> how long did you run sqlsmith to get this crash. However, I have found >> the rea

Re: [HACKERS] [sqlsmith] Failed assertion in _hash_kill_items/MarkBufferDirtyHint

2017-03-31 Thread Ashutosh Sharma
On Sat, Apr 1, 2017 at 6:00 AM, Robert Haas wrote: > On Fri, Mar 31, 2017 at 6:09 PM, Ashutosh Sharma > wrote: >> Well, That is another option but the main idea was to be inline with >> the btree code. > > That's not a bad goal in principal, but _bt_killitems

Re: [HACKERS] [sqlsmith] Failed assertion in _hash_kill_items/MarkBufferDirtyHint

2017-03-31 Thread Ashutosh Sharma
Hi, On Sat, Apr 1, 2017 at 7:06 AM, Ashutosh Sharma wrote: > On Sat, Apr 1, 2017 at 6:00 AM, Robert Haas wrote: >> On Fri, Mar 31, 2017 at 6:09 PM, Ashutosh Sharma >> wrote: >>> Well, That is another option but the main idea was to be inline with >>> the b

Re: [HACKERS] Page Scan Mode in Hash Index

2017-04-01 Thread Ashutosh Sharma
Hi, > > On 03/29/2017 09:16 PM, Ashutosh Sharma wrote: >>> >>> This patch needs a rebase. >> >> >> Please try applying these patches on top of [1]. I think you should be >> able >> to apply it cleanly. Sorry, I think I forgot to mention thi

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-04-01 Thread Ashutosh Sharma
Hi Tomas, On Fri, Mar 31, 2017 at 11:05 PM, Tomas Vondra wrote: > On 03/31/2017 06:01 PM, Ashutosh Sharma wrote: >> >> >> It seems like the latest patch(v4) shared by Tomas upthread is an >> empty patch. If I am not wrong, please share the correct patch. >> Thank

  1   2   3   >