Re: [HACKERS] [POC] hash partitioning

2017-10-06 Thread Jesper Pedersen
this problem using your and Beena's work toward faster partition pruning[1] and Runtime Partition Pruning[2]. Will think on this changes if it is required for the pruning feature. Could you rebase on latest master ? Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgs

[HACKERS] initdb w/ restart

2017-09-28 Thread Jesper Pedersen
he attached patch changes the message to "trying to start server anyway" to highlight it is an attempt, not something that will happen. Probably not a good idea to change the logic around pg_ctl.c:688, hence this suggestion. Thoughts ? Best regards, Jesper >From 9e8cdda3173a25f1

Re: [HACKERS] Partitions: \d vs \d+

2017-09-28 Thread Jesper Pedersen
ease cycle to change that output. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Partitions: \d vs \d+

2017-09-28 Thread Jesper Pedersen
quot;No partition constraint" vs. "Partition constraint: satisfies_hash_partition(...)". Current master (7769fc000) with [1] and [2]. [1] https://commitfest.postgresql.org/14/1059/ [2] https://commitfest.postgresql.org/14/1089/ Best regards, Jesper -- Sent via pgsql-hacke

Re: [HACKERS] Multicolumn hash indexes

2017-09-27 Thread Jesper Pedersen
ol1, col3), ... which isn't ideal. Maybe an initial proof-of-concept could store the hash of the first column (col1) plus the hash of all columns (col1, col2, col3) in the index, and see what requirements / design decisions would appear from that. Best regards, Jesper -- Sent via pgsql-hack

Re: [HACKERS] [POC] hash partitioning

2017-09-27 Thread Jesper Pedersen
l.com Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] path toward faster partition pruning

2017-09-26 Thread Jesper Pedersen
On 09/26/2017 10:33 AM, Robert Haas wrote: On Tue, Sep 26, 2017 at 9:00 AM, Jesper Pedersen wrote: Could you share your thoughts on the usage of PartitionAppendInfo's min_datum_idx / max_datum_idx ? Especially in relation to hash partitions. This brings up something that I've ki

Re: [HACKERS] Improve catcache/syscache performance.

2017-09-26 Thread Jesper Pedersen
ine uint32 Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] path toward faster partition pruning

2017-09-26 Thread Jesper Pedersen
self, but won't be able to immediately (see below). Could you share your thoughts on the usage of PartitionAppendInfo's min_datum_idx / max_datum_idx ? Especially in relation to hash partitions. I'm looking at get_partitions_for_keys. Best regards, Jesper -- Sent via pgs

Re: [HACKERS] [POC] hash partitioning

2017-09-18 Thread Jesper Pedersen
dated some of the documentation. V20 patch passes make check-world, and my testing (typical 64 partitions, and various ATTACH/DETACH scenarios). Thanks for working on this ! Best regards, Jesper >From 189a40a5ca6c7a1bc79b750cbc95584b3061fda5 Mon Sep 17 00:00:00 2001 From: jesperpedersen

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Jesper Pedersen
On 09/14/2017 01:52 PM, Robert Haas wrote: On Thu, Sep 14, 2017 at 1:07 PM, Jesper Pedersen wrote: Yeah, it would be nice to have a syntax like ) PARTITION BY HASH (col) WITH (AUTO_CREATE = 64); But then there also needs to be a way to create the 64 associated indexes too for everything to

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Jesper Pedersen
be easy. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Jesper Pedersen
Hi, On 09/14/2017 12:05 PM, Robert Haas wrote: On Thu, Sep 14, 2017 at 11:39 AM, Jesper Pedersen wrote: When I do CREATE TABLE mytab ( a integer NOT NULL, b integer NOT NULL, c integer, d integer ) PARTITION BY HASH (b); and create 64 partitions; CREATE TABLE mytab_p00

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Jesper Pedersen
Hi Amul, On 09/14/2017 04:58 AM, amul sul wrote: On Wed, Sep 13, 2017 at 7:43 PM, Jesper Pedersen This patch needs a rebase. Thanks for your note. ​ ​ Attached is the patch rebased on the latest master head. Also added error on ​creating ​​default partition ​for the hash partitioned table

Re: [HACKERS] Fix performance degradation of contended LWLock on NUMA

2017-09-14 Thread Jesper Pedersen
On 09/11/2017 11:01 AM, Jesper Pedersen wrote: Thanks for working on this ! Moved to "Ready for Committer". Best regards,  Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] hash partitioning

2017-09-13 Thread Jesper Pedersen
Hi Amul, On 09/08/2017 08:40 AM, amul sul wrote: Rebased 0002 against this commit & renamed to 0001, PFA. This patch needs a rebase. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql

Re: [HACKERS] POC: Cache data in GetSnapshotData()

2017-09-13 Thread Jesper Pedersen
Both for -M prepared, and -M prepared -S I'm not seeing any improvements (1 to 375 clients); e.g. +-1%. Although the -M prepared -S case should improve, I'm not sure that the extra overhead in the -M prepared case is worth the added code complexity. Best regards, Jesper -- Sent via

Re: [HACKERS] Fix performance degradation of contended LWLock on NUMA

2017-09-11 Thread Jesper Pedersen
o the cache storms, as shown by perf c2c. Recently I understand it should be stored in array indexed by tranche, but I didn't implement it yet, and therefore didn't measure. Different constants for the LWLock could have an impact, but the constants would also be dependent on machin

Re: [HACKERS] Fix performance degradation of contended LWLock on NUMA

2017-09-08 Thread Jesper Pedersen
n either, so ignore */ + return; Use { } for the if, or move the comment out of the nesting for readability. Open questions: --- * spins_per_delay as extern * Calculation of skip_wait_list You could run the patch through pgindent too. Passes make check-world. S

Re: [HACKERS] Fix performance of generic atomics

2017-09-06 Thread Jesper Pedersen
d5a57e7...@postgrespro.ru Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fix performance of generic atomics

2017-09-05 Thread Jesper Pedersen
don't see any performance change in the various runs. However, there is no regression either in all cases. As such, I have marked the entry "Ready for Committer". Remember to add a version postfix to your patches such that is easy to identify which is the latest version. Best r

Re: [HACKERS] Fix performance of generic atomics

2017-09-05 Thread Jesper Pedersen
On 09/05/2017 02:24 PM, Tom Lane wrote: Jesper Pedersen writes: I have tested this patch on a 2-socket machine, but don't see any performance change in the various runs. However, there is no regression either in all cases. Hm, so if we can't demonstrate a performance win, it

Re: [HACKERS] Fix performance of generic atomics

2017-09-05 Thread Jesper Pedersen
don't see any performance change in the various runs. However, there is no regression either in all cases. As such, I have marked the entry "Ready for Committer". Remember to add a version postfix to your patches such that is easy to identify which is the latest version. Best r

Re: [HACKERS] Page Scan Mode in Hash Index

2017-08-24 Thread Jesper Pedersen
On 08/24/2017 01:21 AM, Ashutosh Sharma wrote: Done. Attached are the patches with above changes. Thanks ! Based on the feedback in this thread, I have moved the patch to "Ready for Committer". Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] Page Scan Mode in Hash Index

2017-08-23 Thread Jesper Pedersen
are other instances of "inline" in the code base, so I guess that this is ok. +* Advance to next tuple on current page; or if there's no more, try to Advance to the next tuple on the current page; or if done, try to Best regards, Jesper -- Sent via pgsql-hackers ma

Re: [HACKERS] dtrace probes

2017-04-21 Thread Jesper Pedersen
On 04/20/2017 10:30 AM, Jesper Pedersen wrote: I think this fix is harmless and has some value in terms of consistency. One minor suggestion is that you should leave a space after typecasting. - TRACE_POSTGRESQL_LWLOCK_WAIT_DONE(T_NAME(lock), mode); + TRACE_POSTGRESQL_LWLOCK_WAIT_DONE(T_NAME

Re: [HACKERS] dtrace probes

2017-04-20 Thread Jesper Pedersen
-10 which has caused this behavior? If not, then I don't think it should be added to open items of PG-10. It is really a bug fix, so it could even be back patched. Thanks for the feedback ! Best regards, Jesper >From 7175dc8e05ff703229bd6cab6b254587ffc076c8 Mon Sep 17 00:00:00 200

[HACKERS] dtrace probes

2017-04-18 Thread Jesper Pedersen
, and therefore PostgreSQL 11 material. Depending on the feedback I can add this patch to the open item list in order to fix it for PostgreSQL 10. Best regards, Jesper >From d6f5c119c057c7ff8c84f88bb4122a1ca245a7d4 Mon Sep 17 00:00:00 2001 From: jesperpedersen Date: Tue, 18 Apr 2017 11:4

Re: [HACKERS] bumping HASH_VERSION to 3

2017-03-31 Thread Jesper Pedersen
ter an upgrade to v10. - Can remove the backward-compatibility code added by 293e24e507838733aba4748b514536af2d39d7f2 instead of keeping it around forever. - No need to worry about doing an in-place upgrade of the metapage for the above-mentioned patch. Thoughts? +1 Best regards, Jesper --

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-30 Thread Jesper Pedersen
-id/CAE9k0P%3DV2LhtyeMXd295fhisp%3DNWUhRVJ9EZQCDowWiY9rSohQ%40mail.gmail.com Thanks, that works ! As you have provided a patch for Robert's comments, and no other review have been posted I'm moving this patch to "Ready for Committer" for additional committer feedback. B

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-29 Thread Jesper Pedersen
base. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] A better way to expand hash indexes.

2017-03-27 Thread Jesper Pedersen
l fixes, mainly to the documentation - on top of v7. Best regards, Jesper >From 5545e48ab7136f17b3d471e0ee679a6db6040865 Mon Sep 17 00:00:00 2001 From: jesperpedersen Date: Mon, 27 Mar 2017 14:15:00 -0400 Subject: [PATCH] Small fixes --- src/backend/access/ha

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-23 Thread Jesper Pedersen
Hi, On 03/23/2017 02:11 PM, Ashutosh Sharma wrote: On Thu, Mar 23, 2017 at 8:29 PM, Jesper Pedersen wrote: 0001v2: In hashgettuple() you can remove the 'currItem' and 'offnum' from the 'else' part, and do the assignment inside if (so->numKilled < MaxI

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-23 Thread Jesper Pedersen
nts for 0002 and 0003. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-21 Thread Jesper Pedersen
ch command are mentioned in the result sheet. I also did the benchmarking with unique values at 300 and 1000 scale factor and its results are provided in 'results-unique-values-default-ff'. I'm seeing similar results, and especially with write heavy scenarios. Best regards, Jesper

Re: [HACKERS] pageinspect: Hash index support

2017-02-03 Thread Jesper Pedersen
On 02/03/2017 11:41 AM, Jesper Pedersen wrote: contrib/pageinspect actually seems to lack *any* infrastructure for sharing functions across modules. It's time to remedy that. I propose inventing "pageinspect.h" to hold commonly visible declarations, and moving get_page_from_raw()

Re: [HACKERS] pageinspect: Hash index support

2017-02-03 Thread Jesper Pedersen
ing get_page_from_raw() into rawpage.c, which seems like a reasonably natural home for it. (Alternatively, we could invent a pageinspect.c file to hold utility functions, but that might be overkill.) No objections. Attached is v1 of this w/ verify_hash_page() using get_page_from_raw(). Sorry

Re: [HACKERS] pageinspect: Hash index support

2017-02-03 Thread Jesper Pedersen
On 02/02/2017 02:28 PM, Jesper Pedersen wrote: On 02/02/2017 02:24 PM, Robert Haas wrote: So, committed. Wow, I wish every patch had this many reviewers. Thanks Robert ! This message should have included a thank you to everybody who provided valuable feedback for this feature, and for

Re: [HACKERS] pageinspect: Hash index support

2017-02-02 Thread Jesper Pedersen
On 02/02/2017 02:24 PM, Robert Haas wrote: So, committed. Wow, I wish every patch had this many reviewers. Thanks Robert ! Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] Microvacuum support for Hash Index

2017-01-26 Thread Jesper Pedersen
On 01/23/2017 02:53 PM, Jesper Pedersen wrote: I have done some more testing with this, and have moved to the patch back to 'Needs Review' pending Amit's comments. Moved to "Ready for Committer". Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] Microvacuum support for Hash Index

2017-01-23 Thread Jesper Pedersen
Hi Ashutosh, On 01/20/2017 03:24 PM, Jesper Pedersen wrote: Yeah, those are the steps; just with a Skylake laptop. However, I restarted with a fresh master, with WAL v8 and MV v5, and can't reproduce the issue. I have done some more testing with this, and have moved to the patch ba

Re: [HACKERS] Microvacuum support for Hash Index

2017-01-20 Thread Jesper Pedersen
d confirm as if you have been getting this issue repeatedly. Thanks. Yeah, those are the steps; just with a Skylake laptop. However, I restarted with a fresh master, with WAL v8 and MV v5, and can't reproduce the issue. Best regards, Jesper -- Sent via pgsql-hackers mailing

Re: [HACKERS] Microvacuum support for Hash Index

2017-01-19 Thread Jesper Pedersen
Hi Ashutosh, On 01/10/2017 08:40 AM, Jesper Pedersen wrote: On 01/10/2017 05:24 AM, Ashutosh Sharma wrote: Thanks for reporting this problem. It is basically coming because i forgot to unpin the bucketbuf in hash_xlog_vacuum_one_page(). Please find the attached v5 patch that fixes the issue

Re: [HACKERS] pageinspect: Hash index support

2017-01-18 Thread Jesper Pedersen
ed patch also handles all the comments from Mithun. Please, include a version number for your patch files in the future. Fixed in this version: * verify_hash_page: Display magic in hex, like hash_metapage_info * Update header for hash_page_type Moving the patch back to 'Needs Revi

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-01-12 Thread Jesper Pedersen
On 12/27/2016 01:58 AM, Amit Kapila wrote: After recent commit's 7819ba1e and 25216c98, this patch requires a rebase. Attached is the rebased patch. This needs a rebase after commit e898437. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] pageinspect: Hash index support

2017-01-12 Thread Jesper Pedersen
Hi, On 01/11/2017 03:16 PM, Ashutosh Sharma wrote: I have rephrased it to make it more clear. Rebased, and removed the compile warn in hashfuncs.c Best regards, Jesper >From 8a07230b89b97280f0f1d645145da1fd140969c6 Mon Sep 17 00:00:00 2001 From: jesperpedersen Date: Thu, 12 Jan 2017

Re: [HACKERS] Retiring from the Core Team

2017-01-12 Thread Jesper Pedersen
son, as of today, I am stepping down from the PostgreSQL Core Team. Thanks for all your work over the years ! Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pageinspect: Hash index support

2017-01-10 Thread Jesper Pedersen
er changes are required. Right, this is a new recommended process since commit 40b449ae8, which see for rationale. Use, eg, commit 11da83a0e as a model for extension update patches. Thanks for the commit ids ! Revised patched attached. Best regards, Jesp

Re: [HACKERS] Microvacuum support for Hash Index

2017-01-10 Thread Jesper Pedersen
. Process 14610 waits for ShareLock on transaction 1444616; blocked by process 14608. HINT: See server log for query details. CONTEXT: while rechecking updated tuple (12,3) in relation "test" ... using pgbench -M prepared -c 10 -j 10 -T 300 -f test.sql test Best regards, Jesper -

Re: [HACKERS] Microvacuum support for Hash Index

2017-01-09 Thread Jesper Pedersen
ocessMain () at startup.c:216 on the slave instance in a master-slave setup. Also, the src/backend/access/README file should be updated with a description of the changes which this patch provides. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@p

Re: [HACKERS] Microvacuum support for Hash Index

2017-01-05 Thread Jesper Pedersen
Hi Ashutosh, On 01/04/2017 06:13 AM, Ashutosh Sharma wrote: Attached is the v3 patch rebased on postgreSQL HEAD and WAL v7 patch. It also takes care of all the previous comments from Jesper - [1]. With an --enable-cassert build (master / WAL v7 / MV v3) and -- ddl.sql -- CREATE TABLE test

Re: [HACKERS] pageinspect: Hash index support

2017-01-05 Thread Jesper Pedersen
sql. Peter has already written a test-case-[1] based on your earlier patch for supporting hash index with pageinspect module. Once the latest patch (v10) becomes stable i will share a separete patch having a test-case for hash index. Infact I will try to modify an already existing patch by Peter

Re: [HACKERS] pageinspect: Hash index support

2017-01-04 Thread Jesper Pedersen
e in 1 file, as we need something to install from. Removing --1.5.sql otherwise would give test=# CREATE EXTENSION "pageinspect"; ERROR: extension "pageinspect" has no installation script nor update path for version "1.6" * Minor documentation changes Look it over, a

Re: [HACKERS] Microvacuum support for Hash Index

2016-12-30 Thread Jesper Pedersen
On 11/11/2016 12:11 AM, Ashutosh Sharma wrote: Hi Jesper, Some initial comments. _hash_vacuum_one_page: + END_CRIT_SECTION(); + _hash_chgbufaccess(rel, metabuf, HASH_READ, HASH_NOLOCK); The _hash_chgbufaccess() needs a comment. You also need a place where you

Re: [HACKERS] pageinspect: Hash index support

2016-12-20 Thread Jesper Pedersen
eek for CF/January, so I'll review your changes with the previous feedback in mind. Thanks for working on this ! Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_catversion builtin function

2016-12-14 Thread Jesper Pedersen
rol_system(); catalog_version_no 201612081 (1 row) Ah, perfect ! Thanks, Robert Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_catversion builtin function

2016-12-14 Thread Jesper Pedersen
On 12/13/2016 10:33 AM, Tom Lane wrote: Jesper Pedersen writes: Attached is a new builtin function that exposes the CATALOG_VERSION_NO constant under the pg_catversion() function, e.g. I'm pretty sure that we intentionally didn't expose that, reasoning that users should only care

Re: [HACKERS] Hash Indexes

2016-12-13 Thread Jesper Pedersen
don't think it will impact the test results if you have already started the test with the previous patch, but if not, then it is better to test with attached. A mix work load (INSERT, DELETE and VACUUM primarily) is successful here too using _v2. Thanks ! Best regards, Jesper -- Sen

[HACKERS] pg_catversion builtin function

2016-12-13 Thread Jesper Pedersen
/restore is needed; it could have other use-cases. I'm unsure of the OID assignment rules - feel free to point me towards information regarding this. I'll register this patch with the next CF. Best regards, Jesper >From 39d52f5389bf3ef1814c1f201df6531feb2a5c7f Mon Sep 17 00:00

Re: [HACKERS] Microvacuum support for Hash Index

2016-12-01 Thread Jesper Pedersen
you a next version of patch for supporting microvacuum in hash index. As the concurrent hash index patch was committed in 6d46f4 this patch needs a rebase. I have moved this submission to the next CF. Thanks for working on this ! Best regards, Jesper -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] Cache Hash Index meta page.

2016-12-01 Thread Jesper Pedersen
eds a rebase. I have moved this submission to the next CF. Thanks for working on this ! Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Microvacuum support for Hash Index

2016-11-03 Thread Jesper Pedersen
_hash_chgbufaccess() needs a comment. You also need a place where you call pfree for so->killedItems - maybe in hashkillitems(). Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailp

Re: [HACKERS] pageinspect: Hash index support

2016-11-02 Thread Jesper Pedersen
have also committed the tests that I proposed and will work through the failures. As no new patch has been posted for the 2016-11 CF, I will close the patch entry now. Please submit an updated patch when you have the time, keeping an eye on ongoing work to update hash indexes. Agreed. Best regard

Re: [HACKERS] pageinspect: Hash index support

2016-10-03 Thread Jesper Pedersen
Ok, thanks for your feedback. Maybe "Returned with Feedback" is more appropriate, as there is still development left. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pageinspect: Hash index support

2016-09-29 Thread Jesper Pedersen
On 09/29/2016 11:58 AM, Peter Eisentraut wrote: On 9/27/16 10:10 AM, Jesper Pedersen wrote: contrib/pageinspect/pageinspect--1.5--1.6.sql | 59 contrib/pageinspect/pageinspect--1.5.sql | 279 -- contrib/pageinspect/pageinspect--1.6.sql | 346

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-27 Thread Jesper Pedersen
.gmail.com Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Indexes

2016-09-27 Thread Jesper Pedersen
id/CAA4eK1KE%3D%2BkkowyYD0vmch%3Dph4ND3H1tViAB%2B0cWTHqjZDDfqg%40mail.gmail.com Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pageinspect: Hash index support

2016-09-27 Thread Jesper Pedersen
On 09/26/2016 10:45 PM, Peter Eisentraut wrote: On 9/26/16 1:39 PM, Jesper Pedersen wrote: Left as is, since BuildTupleFromCStrings() vs. xyzGetDatum() are equally readable in this case. But, I can change the patch if needed. The point is that to use BuildTupleFromCStrings() you need to

Re: [HACKERS] pageinspect: Hash index support

2016-09-26 Thread Jesper Pedersen
v6, since we don't display average item size anymore. Thanks for the feedback ! Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pageinspect: Hash index support

2016-09-26 Thread Jesper Pedersen
Hi, On 09/23/2016 12:10 AM, Peter Eisentraut wrote: On 9/21/16 9:30 AM, Jesper Pedersen wrote: Attached is v5, which add basic page verification. There are still some things that I found that appear to follow the old style (btree) conventions instead the new style (brin, gin) conventions

Re: [HACKERS] pageinspect: Hash index support

2016-09-21 Thread Jesper Pedersen
page, and the meta function on a non-meta page, but at my surprise I did not see a crash, so perhaps I was lucky and perhaps that was because of OSX. Attached is v5, which add basic page verification. Thanks for the feedback ! Best regards, Jesper >F

Re: [HACKERS] pageinspect: Hash index support

2016-09-21 Thread Jesper Pedersen
ch looks in pretty good shape to me, so I am switching it as ready for committer. Thanks for your feedback ! Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pageinspect: Hash index support

2016-09-21 Thread Jesper Pedersen
passing valid data. 5). I think we have added enough functions to show the page level statistics but not the index level statistics like the total number of overflow pages , bucket pages, number of free overflow pages, number of bitmap pages etc. in the hash index. How about adding a functi

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-20 Thread Jesper Pedersen
updated README. Thanks. This needs a rebase against the CHI v8 [1] patch. [1] https://www.postgresql.org/message-id/CAA4eK1+X=8sud1uczdzne3d9cgi9kw+kjxp2tnw7sx5w8pl...@mail.gmail.com Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] pageinspect: Hash index support

2016-09-20 Thread Jesper Pedersen
have just started reviewing the patch, will keep on posting my comments upon further review. Fixed in v4. Thanks for the review. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pageinspect: Hash index support

2016-09-20 Thread Jesper Pedersen
nt what the single letter values mean in the hash_page_stats.type column? It is not obvious that 'i' means bitmap, for example. Adjusted in v4. Code/doc will need an update once the CHI patch goes in. Best regards, Jesper >From 1f27a2bb28cc6dfea9cba015d7cceab768f67d0a Mon Sep 17

Re: [HACKERS] pageinspect: Hash index support

2016-09-20 Thread Jesper Pedersen
both. Attached is v3 with only the bytea based methods. Alvaro, Michael and Jeff - Thanks for the review ! Best regards, Jesper >From 0aff82ccb40f00efe9e48cacef9c8a45c1327da2 Mon Sep 17 00:00:00 2001 From: jesperpedersen Date: Fri, 5 Aug 2016 10:16:32 -0400 Subject: [PATCH] pageinspect: H

Re: [HACKERS] pageinspect: Hash index support

2016-09-19 Thread Jesper Pedersen
end a v3 with the set voted for. I kept the 'data' field as is, for now. Best regards, Jesper >From 0dc44e4b3cc1d31c53684a41fbd7959978e69089 Mon Sep 17 00:00:00 2001 From: jesperpedersen Date: Fri, 5 Aug 2016 10:16:32 -0400 Subject: [PATCH] pageinspect: Hash index support --

Re: [HACKERS] Hash Indexes

2016-09-16 Thread Jesper Pedersen
emove the usage of hashscan.c from code if no one objects to it, do let me know if I am missing something here. Like Robert said, hashscan.c can always come back, and it would take a call-stack out of the 'am' methods. Best regards, Jesper -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Hash Indexes

2016-09-15 Thread Jesper Pedersen
. [1] https://www.postgresql.org/message-id/5f8b4681-1229-92f4-4315-57d780d9c128%40redhat.com Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Indexes

2016-09-14 Thread Jesper Pedersen
Hi, On 09/14/2016 07:24 AM, Amit Kapila wrote: On Wed, Sep 14, 2016 at 12:29 AM, Jesper Pedersen wrote: On 09/13/2016 07:26 AM, Amit Kapila wrote: Attached, new version of patch which contains the fix for problem reported on write-ahead-log of hash index thread [1]. I have been testing

Re: [HACKERS] Hash Indexes

2016-09-13 Thread Jesper Pedersen
page instead of having it on the main meta page. Likely a question for the assigned committer. Thanks for working on this ! Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-13 Thread Jesper Pedersen
in hash_xlog_move_page_contents, hash_xlog_squeeze_page and hash_xlog_delete. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Indexes

2016-09-13 Thread Jesper Pedersen
It completed for me as well. The second index creation is taking more time and cpu, because it is just inserting duplicate values which need lot of overflow pages. Yeah, sorry for the false alarm. It just took 3m45s to complete on my machine. Best regards, Jesper -- Sent via pgsql-hackers ma

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-12 Thread Jesper Pedersen
_xlog_delete has the "if" issue too. hash.h: Move the XLog related content to hash_xlog.h Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Indexes

2016-09-12 Thread Jesper Pedersen
On 09/01/2016 11:55 PM, Amit Kapila wrote: I have fixed all other issues you have raised. Updated patch is attached with this mail. The following script hangs on idx_val creation - just with v5, WAL patch not applied. Best regards, Jesper zero.sql Description: application/sql

Re: [HACKERS] Cache Hash Index meta page.

2016-09-08 Thread Jesper Pedersen
On 09/05/2016 02:50 PM, Mithun Cy wrote: On Sep 2, 2016 7:38 PM, "Jesper Pedersen" wrote: Could you provide a rebased patch based on Amit's v5 ? Please find the the patch, based on Amit's V5. I have fixed following things 1. now in "_hash_first" we che

Re: [HACKERS] Cache Hash Index meta page.

2016-09-02 Thread Jesper Pedersen
apply the same patch on Amit's concurrent hash index patch [1] we can see improvements at lower clients also. Amit's patch has removed a heavy weight page lock which was the bottle neck at lower clients. Could you provide a rebased patch based on Amit's v5 ? Best regards, Jesper

Re: [HACKERS] Hash Indexes

2016-09-01 Thread Jesper Pedersen
lumn which showed good results. Will have to compare with a run with the WAL patch applied. make check-world passes. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] pageinspect: Hash index support

2016-08-30 Thread Jesper Pedersen
ccount for the new meta-page constants. I'll create a CommitFest entry for this submission. Feedback is most welcome. Best regards, Jesper >From 55262d5fa3822afbae94f4989627dd65e91fe098 Mon Sep 17 00:00:00 2001 From: jesperpedersen Date: Fri, 5 Aug 2016 10:16:32 -0400 Subject: [PATCH]

Re: [HACKERS] Speedup twophase transactions

2016-05-20 Thread Jesper Pedersen
ers on the matter would be appreciated. [1] http://www.postgresql.org/message-id/e7497864-de11-4099-83f5-89fb97af5...@postgrespro.ru [2] http://www.postgresql.org/message-id/5693f703.3000...@redhat.com [3] https://commitfest.postgresql.org/10/523/ Best regards, Jesper -- Sent via pgsql-hacke

Re: [HACKERS] Speedup twophase transactions

2016-04-08 Thread Jesper Pedersen
On 04/08/2016 02:37 PM, Robert Haas wrote: On Fri, Apr 8, 2016 at 8:49 AM, Jesper Pedersen wrote: On 04/07/2016 02:29 AM, Michael Paquier wrote: So recovery is conflicting here. My guess is that this patch is missing some lock cleanup. With the test case attached in my case the COMMIT

Re: [HACKERS] Speedup twophase transactions

2016-04-08 Thread Jesper Pedersen
mmitter" yet. [0] http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=978b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e [1] https://commitfest.postgresql.org/9/523/ Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript

Re: [HACKERS] Speedup twophase transactions

2016-04-08 Thread Jesper Pedersen
3/ [3] http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=978b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-04-04 Thread Jesper Pedersen
On 04/01/2016 04:39 PM, Andres Freund wrote: On April 1, 2016 10:25:51 PM GMT+02:00, Jesper Pedersen wrote: Hi, On 03/31/2016 06:21 PM, Andres Freund wrote: On March 31, 2016 11:13:46 PM GMT+02:00, Jesper Pedersen wrote: I can do a USE_CONTENT_LOCK run on 0003 if it is something for

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-04-01 Thread Jesper Pedersen
Hi, On 03/31/2016 06:21 PM, Andres Freund wrote: On March 31, 2016 11:13:46 PM GMT+02:00, Jesper Pedersen wrote: I can do a USE_CONTENT_LOCK run on 0003 if it is something for 9.6. Yes please. I think the lock variant is realistic, the lockless did isn't. I have done a run wi

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-31 Thread Jesper Pedersen
unlogged. I can do a USE_CONTENT_LOCK run on 0003 if it is something for 9.6. Thanks for your work on this ! Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Speedup twophase transactions

2016-03-30 Thread Jesper Pedersen
omFiles have a lot of > code in common, having this duplication is not good, and you could > simplify your patch. I reworked patch to avoid duplicated code between RecoverPreparedFromXLOG/RecoverPreparedFromFiles and also between FinishPreparedTransaction/XlogRedoFinishPrepared. Patch app

Re: [HACKERS] Speedup twophase transactions

2016-03-21 Thread Jesper Pedersen
On 03/18/2016 12:50 PM, Stas Kelvich wrote: On 11 Mar 2016, at 19:41, Jesper Pedersen wrote: Thanks for review, Jesper. Some comments: * The patch needs a rebase against the latest TwoPhaseFileHeader change Done. * Rework the check.sh script into a TAP test case (src/test/recovery

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-19 Thread Jesper Pedersen
56T @ 2GHz w/ 256GB + 2 x RAID10 (data + xlog) SSD. I can provide perf / flamegraph profiles if needed. Thanks for working on this ! Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

  1   2   3   >