Re: [HACKERS] Is the unfair lwlock behavior intended?

2016-05-24 Thread Alexander Korotkov
CSN should fix various scalability issues including high ProcArrayLock contention. References. 1. http://www.postgresql.org/message-id/CAPpHfdsytkTFMy3N-zfSo+kAuUx=u-7jg6q2byb6fpuw2cd...@mail.gmail.com 2. http://www.postgresql.org/message-id/20151211130413.go14...@awork2.anarazel.d

[HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-26 Thread Alexander Korotkov
transaction. If not, could pg_dump return some more friendly error before actually trying to dump? -- With best regards, Alexander Korotkov.

Re: [HACKERS] Cube extension kNN support

2015-02-07 Thread Alexander Korotkov
s.coord->-4 LIMIT 10; > I've checked the patch. The first notes are so: 1) Check coding style, in particular braces. Postgres coding style require using it for multiline statements. 2) Update documentation according to new features. -- With best regards, Alexander Korotkov.

Re: [HACKERS] GSoC 2015 - mentors, students and admins.

2015-02-12 Thread Alexander Korotkov
or a mentor. > I'm ready to participate as mentor again! -- With best regards, Alexander Korotkov.

Re: [HACKERS] KNN-GiST with recheck

2015-02-15 Thread Alexander Korotkov
On Thu, Jan 8, 2015 at 1:12 AM, Alexander Korotkov wrote: > On Tue, Dec 16, 2014 at 4:37 PM, Heikki Linnakangas < > hlinnakan...@vmware.com> wrote: > >> Patch attached. It should be applied on top of my pairing heap patch at >> http://www.postgresql.org/message-id/

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2015-02-17 Thread Alexander Korotkov
if item a should be checked first function should return 1. Current behavior doesn't lead to incorrect query answers, but it could be slower than correct version. -- With best regards, Alexander Korotkov. pairing_heap_cmp_fix.patch Description: Binary data -- Sent via pgsql-hackers maili

Re: [HACKERS] KNN-GiST with recheck

2015-02-17 Thread Alexander Korotkov
On Sun, Feb 15, 2015 at 2:08 PM, Alexander Korotkov wrote: > Following changes has been made in attached patch: > > * Get sort operators from pathkeys. > * Recheck argument of distance function has been reverted. > Few comments were added and pairing heap comparison functi

Re: [HACKERS] GSoC 2015 - mentors, students and admins.

2015-02-17 Thread Alexander Korotkov
posals accepted. -- With best regards, Alexander Korotkov.

Re: [HACKERS] KNN-GiST with recheck

2015-02-25 Thread Alexander Korotkov
Hi! On Tue, Feb 24, 2015 at 5:39 PM, Tomas Vondra wrote: > On 17.2.2015 14:21, Alexander Korotkov wrote: > > On Sun, Feb 15, 2015 at 2:08 PM, Alexander Korotkov > > mailto:aekorot...@gmail.com>> wrote: > > > > Revised patch with reordering in GiST is attached &g

Re: [HACKERS] Fillfactor for GIN indexes

2015-02-25 Thread Alexander Korotkov
the heap (perhaps do some deletes and vacuum). Actually, this is narrowing benefit from GIN fillfactor. Probably, that means that we should still have default value of 100. But I think GIN fillfactor still might be useful. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Report search_path value back to the client.

2015-03-02 Thread Alexander Kukushkin
VSearchPath }, {NULL}, }; We could possibly alleviate problem #1 by changing the behavior of guc.c > so it doesn't report every single transition of flagged variables, but > only (say) once just before ReadyForQuery if the variable changed in > the just-finished command. That's not exactly a one-line fix though. > Probably for some variables this really make sense. Inside stored procedures any of GUC_REPORT variables can be changed the same way as search_path (thousands of times), but not all of these variables directly relate to the application visible behaviour. Regards, Alexander Kukushkin

Re: [HACKERS] Fillfactor for GIN indexes

2014-11-28 Thread Alexander Korotkov
ages are always compressed excepts pg_upgraded indexes from pre-9.4. Take care about it in following code. if (GinPageIsCompressed(page)) freespace = GinDataLeafPageGetFreeSpace(page); + else if (btree->isBuild) + freespace = BLCKSZ * (100 - fillfactor) / 100; -- With best regards, Alexander Korotkov.

Re: [HACKERS] KNN-GiST with recheck

2014-11-28 Thread Alexander Korotkov
On Sat, Nov 22, 2014 at 2:20 AM, Peter Geoghegan wrote: > On Mon, Jan 13, 2014 at 9:17 AM, Alexander Korotkov > wrote: > > This patch was split from thread: > > > http://www.postgresql.org/message-id/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=gon-...@mail.gmail.com >

[HACKERS] Report search_path value back to the client.

2014-12-02 Thread Alexander Kukushkin
h order for names that are not schema-qualified."), NULL, - GUC_LIST_INPUT | GUC_LIST_QUOTE + GUC_LIST_INPUT | GUC_LIST_QUOTE | GUC_REPORT }, &namespace_search_path, "\"$user\",public", What do you think? Regards, -- Alexander Kukushkin

Re: [HACKERS] Commit fest 2014-12, let's begin!

2014-12-15 Thread Alexander Korotkov
;-> 'SRID=26918;POINT(583571.905921312 4506714.34119218)'::geometry LIMIT 100 ) SELECT gid, name FROM closest_candidates ORDER BY ST_Distance( geom, 'SRID=26918;POINT(583571.905921312 4506714.34119218)'::geometry ) LIMIT 1; See blog posts: http://blog.light42.com/wordpress/?p=102 http://workshops.boundlessgeo.com/postgis-intro/knn.html -- With best regards, Alexander Korotkov.

Re: [HACKERS] Commit fest 2014-12, let's begin!

2014-12-15 Thread Alexander Korotkov
On Mon, Dec 15, 2014 at 6:20 PM, Tom Lane wrote: > > Alexander Korotkov writes: > > On Mon, Dec 15, 2014 at 4:12 PM, Heikki Linnakangas < > hlinnakan...@vmware.com > >> wrote: > >> Right. I also looked at it briefly, but I wasn't sure if we really want &

Re: [HACKERS] Commit fest 2014-12, let's begin!

2014-12-15 Thread Alexander Korotkov
On Mon, Dec 15, 2014 at 7:47 PM, Heikki Linnakangas wrote: > > On 12/15/2014 05:22 PM, Alexander Korotkov wrote: > >> On Mon, Dec 15, 2014 at 6:20 PM, Tom Lane wrote: >> >>> >>> Alexander Korotkov writes: >>> >>>> On Mon, Dec

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Alexander Korotkov
On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier wrote: > On Wed, Dec 3, 2014 at 2:37 AM, Robert Haas wrote: > > On Fri, Nov 28, 2014 at 4:27 AM, Alexander Korotkov > > wrote: > >> On Fri, Nov 21, 2014 at 8:12 AM, Michael Paquier < > michael.paqu...@gmail.com&

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Alexander Korotkov
On Thu, Jan 8, 2015 at 12:31 AM, Alexander Korotkov wrote: > > Rewritten version of patch is attached. I made following changes: > > 1) I removed fillfactor handling from entry tree. Because in this case > fillfactor parameter would be only upper bound for actual page fullness. &

Re: [HACKERS] KNN-GiST with recheck

2015-01-07 Thread Alexander Korotkov
t doesn't get a > recheck argument. It is just assumed that if the consistent function sets > the recheck flag, then the distance needs to be rechecked as well. We might > want to add the recheck argument, like you Alexander did in your patch, but > it's not important right now.

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-15 Thread Alexander Korotkov
On Thu, Jan 15, 2015 at 10:19 AM, Michael Paquier wrote: > On Thu, Jan 8, 2015 at 2:03 PM, Michael Paquier > wrote: > > On Thu, Jan 8, 2015 at 6:31 AM, Alexander Korotkov > wrote: > >> On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier < > michael.paqu...@gmai

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-17 Thread Alexander Korotkov
On Fri, Jan 16, 2015 at 8:40 PM, Robert Haas wrote: > On Thu, Jan 15, 2015 at 7:06 AM, Michael Paquier > wrote: > > Alexander Korotkov wrote: > >> I'm not sure. On the one hand it's unclear why fillfactor should be > >> different from 9.4. > >>

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-17 Thread Alexander Korotkov
On Sat, Jan 17, 2015 at 12:49 PM, Alexander Korotkov wrote: > BTW, previous version of patch contained some bugs. Revised version is > attached. > Ooops, it's here. -- With best regards, Alexander Korotkov. gin_fillfactor_5.patch Description: Binary data -- Sent via

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-19 Thread Alexander Korotkov
On Mon, Jan 19, 2015 at 5:46 PM, Cédric Villemain wrote: > Le lundi 19 janvier 2015 08:24:08 Robert Haas a écrit : > > > On Sat, Jan 17, 2015 at 4:49 AM, Alexander Korotkov > > > > > > wrote: > > > > I already wrote quite detailed explanation of subje

Re: [HACKERS] multibyte charater set in levenshtein function

2010-06-07 Thread Alexander Korotkov
0502 width=8) (actual time=5.391..42.143 rows=5 loops=1) Filter: (levenshtein_less_equal(word, 'consistent'::text, 2) <= 2) Total runtime: 42.292 ms (6 rows) In the example above levenshtein_less_equal works about 5 times faster. With best regards, Alexander Korotkov. fuzzystrm

[HACKERS] Parameters of GiST indexes

2010-06-07 Thread Alexander Korotkov
REATE INDEX name ON table USING gist(column) WITH (SIGLENINT=63); With best regards, Korotkov Alexander.

[HACKERS] Using multidimensional indexes in ordinal queries

2010-06-07 Thread Alexander Korotkov
v3]) <@ cube(array[480,480,'-Infinity'::float],array[500,500,'+Infinity'::float])" and "order by v3" by "order by cube(array[v1,v2,v3])", but it will be a great option if planner could consider plan with gist cube index for original query. With best re

Re: [HACKERS] Using multidimensional indexes in ordinal queries

2010-06-21 Thread Alexander Korotkov
I didn't get the same plans as Alexander > did, though, so it may not really be apples to apples. See attached > session trace. > Benefit of multicolumn btree index was more or less the same than cube benefit because of very bad picksplit behavior in this case. I attached the patch w

Re: [HACKERS] Using multidimensional indexes in ordinal queries

2010-06-22 Thread Alexander Korotkov
On Tue, Jun 22, 2010 at 1:58 AM, Robert Haas wrote: > It doesn't? I didn't think it was making any assumptions about the > ordering data type beyond the fact that it had a default btree > opclass. > Actually, the return type of consistent method was replaced by float8. Negative values are used

Re: [HACKERS] multibyte charater set in levenshtein function

2010-07-13 Thread Alexander Korotkov
tion which function supports mbchars not to confuse users. > I've updated this notification. Also I've added documentation for levenshtein_less_equal function. * (Not an issue for the patch, but...) > Could you rewrite PG_GETARG_TEXT_P, VARDATA, and VARSIZE to > PG_GETARG_TEXT_PP,

Re: [HACKERS] multibyte charater set in levenshtein function

2010-07-21 Thread Alexander Korotkov
code. If everyone who touches this code adds their name, we're > quickly going to have a mess. If we're not going to remove the > acknowledgments section, then please add my name, too, because I've > already patched this code once... > In that case I think we can leave original acknowledgments section. With best regards, Alexander Korotkov.

Re: [HACKERS] multibyte charater set in levenshtein function

2010-07-21 Thread Alexander Korotkov
return d; } I tested it with american-english dictionary with 98569 words. test=# select sum(levenshtein(word, 'qwerqwerqwer')) from words; sum - 1074376 (1 row) Time: 131,435 ms test=# select sum(levenshtein_less_equal(word, 'qwerqwerqwer',100)) from words; sum - 1074376 (1 row) Time: 221,078 ms test=# select sum(levenshtein_less_equal(word, 'qwerqwerqwer',-1)) from words; sum - 1074376 (1 row) Time: 254,819 ms The function with negative value of max_d didn't become faster than with just big value of max_d. With best regards, Alexander Korotkov.

Re: [HACKERS] multibyte charater set in levenshtein function

2010-07-22 Thread Alexander Korotkov
} temp = curr; curr = prev; prev = temp; } d = prev[m - 1]; /* * Because the final value was swapped from the previous row to the * current row, that's where we'll find it. */ return d; } What do you thing about it? With best regards, Alexander Korotkov.

Re: [HACKERS] multibyte charater set in levenshtein function

2010-07-22 Thread Alexander Korotkov
ludes (like in 'backend/utils/adt/like.c'). Do you think it is acceptable in this case? With best regards, Alexander Korotkov.

Re: [HACKERS] multibyte charater set in levenshtein function

2010-07-27 Thread Alexander Korotkov
nshtein(word, 'Dhaka craziness savvies teeae luhs Barentss unwe zher')<=10; id | word | next_id --+-+- 4589 | Dhaka craziness savvies teenager ploughs Barents's unwed zither | 70983

Re: [HACKERS] multibyte charater set in levenshtein function

2010-07-29 Thread Alexander Korotkov
I forgot attribution in levenshtein.c file. With best regards, Alexander Korotkov. fuzzystrmatch-0.5.1.tar.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] knngist - 0.8

2010-07-30 Thread Alexander Korotkov
g_trgm it makes it possible to combine different similarity levels in one query. For example: select * from test_trgm order by t <-> 'asdf' < 0.5 or t <-> 'qwer' < 0.4; Is there any chance to handle this syntax also? With best regards, Alexander Korotkov.

Re: [HACKERS] multibyte charater set in levenshtein function

2010-08-04 Thread Alexander Korotkov
e freedom for optimization. After replacing memcmp with inline function like following in your implementation: static inline bool char_cmp(const char *s, const char *d, int len) { while (len--) { if (*s++ != *d++) return false; } return true; } Code becomes much faster: test=# select sum(levenshtein(word, 'фывафыва')) from test; sum - 1675281 (1 row) Time: 241,272 ms With best regards, Alexander Korotkov.

Re: [HACKERS] multibyte charater set in levenshtein function

2010-08-04 Thread Alexander Korotkov
Now I think patch is as good as can be. :) I'm going to prepare less-or-equal function in same manner as this patch. With best regards, Alexander Korotkov.

Re: [HACKERS] knngist - 0.8

2010-08-09 Thread Alexander Korotkov
reshold is global for session. That's why we can't create complex queries which contain similarity filtering with different threshold. With best regards, Alexander Korotkov. On Mon, Aug 2, 2010 at 8:14 PM, Robert Haas wrote: > 2010/7/29 Alexander Korotkov : > > But, in pg

Re: [HACKERS] knngist - 0.8

2010-08-10 Thread Alexander Korotkov
with GiST and I'm really feeling the need of it. With best regards, Alexander Korotkov.

Re: [HACKERS] multibyte charater set in levenshtein function

2010-08-28 Thread Alexander Korotkov
Here is the patch which adds levenshtein_less_equal function. I'm going to add it to current commitfest. With best regards, Alexander Korotkov. On Tue, Aug 3, 2010 at 3:23 AM, Robert Haas wrote: > On Mon, Aug 2, 2010 at 5:07 PM, Alexander Korotkov > wrote: > > Now I

Re: [HACKERS] multibyte charater set in levenshtein function

2010-08-28 Thread Alexander Korotkov
t=# select sum(levenshtein_less_equal('таяй раскупорившийся передислоцируется юлианович праздничный лачужка присыхать опппливший ффехтовальный добряющий', a, 200)) from phrases; sum - 1091878 (1 row) Time: 674,734 ms test=# select sum(levenshtein('таяй раскупорившийся передислоцируется юлианович праздничный лачужка присыхать опппливший ффехтовальный добряющий', a)) from phrases; sum - 1091878 (1 row) Time: 673,515 ms With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-08 Thread Alexander Korotkov
tab1. But actually, it isn't always so. In aggregate with only few used test cases it can cause significant error. I'm going to make use some more thought-out testing method. Probably, some more precise index quality measure exists (even for R-tree). -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-12 Thread Alexander Korotkov
On Fri, Jul 8, 2011 at 6:18 PM, Tom Lane wrote: > For test purposes, you could turn off synchronize_seqscans to prevent > that. Thanks, it helps. I'm rerunning tests now. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-12 Thread Alexander Korotkov
New version of patch with a little more refactoring and comments. -- With best regards, Alexander Korotkov. gist_fast_build-0.6.0.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] WIP: Fast GiST index build

2011-07-13 Thread Alexander Korotkov
tive_cache_size?), and switch to the buffering > method after that. Yes, it seems to be possible. It also would be great to somehow detect case of ordered data when regular index build performs well. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-13 Thread Alexander Korotkov
On Wed, Jul 13, 2011 at 12:40 PM, Alexander Korotkov wrote: > On Wed, Jul 13, 2011 at 12:33 PM, Heikki Linnakangas < > heikki.linnakan...@enterprisedb.com> wrote: > >> Is it possible to switch to the new buffering method in the middle of an >> index build? We could use

Re: [HACKERS] Small patch for GiST: move childoffnum to child

2011-07-13 Thread Alexander Korotkov
Thank you very much for detail explanation. But this line of modified patch seems strange for me: *newchildoffnum = blkno; I believe it should be: *newchildoffnum = i; -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-14 Thread Alexander Korotkov
od. But concurent backends makes this method unsafe. There are other difficulties with concurrent backends: it would be nice estimate usage of effective cache by other backeds before switching to buffering method. If don't take care about it then we can don't switch to buffering method wh

Re: [HACKERS] Small patch for GiST: move childoffnum to child

2011-07-14 Thread Alexander Korotkov
lidOffsetNumber; >ptr->parent = top->parent; >ptr->next = top->next; >top->next = ptr; >if (tail == top); >tail = ptr; } > -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-14 Thread Alexander Korotkov
Do you think using "rightlink" as pointer to parent page is possible during index build? It would allow to simplify code significantly, because of no more need to maintain in-memory structures for parents memorizing. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-14 Thread Alexander Korotkov
fd.c directly, ie. > OpenTemporaryFile/FileRead/**FileWrite. BufFile is distributing temporary data through several files. AFAICS postgres avoids working with files larger than 1GB. Size of tree buffers can easily be greater. Without BufFile I need to maintain set of files manually. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Patch Review: Collect frequency statistics and selectivity estimation for arrays

2011-07-15 Thread Alexander Korotkov
tree and frequency/length statistics for arrays? > 1) In calc_distr you go to some lengths to avoid round off errors. Since it > is > certainly just the order of the estimate that matters, why not just > perform the calculation in log space? > It seems to me that I didn't anything to avoid round off errors there... -- With best regards, Alexander Korotkov.

Re: [HACKERS] Patch Review: Collect frequency statistics and selectivity estimation for arrays

2011-07-15 Thread Alexander Korotkov
standard use of histogram slot should be avoided. -- With best regards, Alexander Korotkov.

Re: [HACKERS] Small patch for GiST: move childoffnum to child

2011-07-15 Thread Alexander Korotkov
g subtly different in > different versions, if we need to backpatch bug fixes that use that field. Yes, it seems very reasonable. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-15 Thread Alexander Korotkov
Fri, Jul 15, 2011 at 12:53 AM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > On 14.07.2011 23:41, Alexander Korotkov wrote: > >> Do you think using "rightlink" as pointer to parent page is possible >> during >> index build? It wo

Re: [HACKERS] WIP: Fast GiST index build

2011-07-18 Thread Alexander Korotkov
tomatic switching. 2) Take care about concurrent backends in automatic switching. -- With best regards, Alexander Korotkov. gist_fast_build-0.7.0.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: Fast GiST index build

2011-07-22 Thread Alexander Korotkov
ome researches about distribution law for such cases (while I didn't find anything relevant in google scholar)? As an alternative I can propose take into account actual average IO operations per tuple rather then an estimate. -- With best regards, Alexander Korotkov. On Mon, Jul 18, 20

Re: [HACKERS] WIP: Fast GiST index build

2011-07-26 Thread Alexander Korotkov
ry to evade keeping loaded-tree hash. This might help me a lot. Thanks. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-27 Thread Alexander Korotkov
think modification of WAL record structure is possible or I have to insert downlink one by one in buffering build too? -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-27 Thread Alexander Korotkov
2) blk: 564 numTuple: 109 free: 3352b(58.92%) rightlink:4294967295 (InvalidBlockNumber) > 7(l:2) blk: 165 numTuple: 108 free: 3396b(58.38%) rightlink:4294967295 (InvalidBlockNumber) . Isn't it a bug? -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-28 Thread Alexander Korotkov
buffers. But those paths can become invalid on page splits. It seems to me that approximately same volume of code for maintaining parent links should be added to this version of patch in order to get it working correctly. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-07-28 Thread Alexander Korotkov
Also I need to verify final emptying, because IO guarantees of original paper is based on strict descending final emptying. -- With best regards, Alexander Korotkov.

Re: Hot standby and GiST page splits (was Re: [HACKERS] WIP: Fast GiST index build)

2011-08-01 Thread Alexander Korotkov
quot;Concurrency and Recovery for GiSTs". http://www.sai.msu.su/~megera/postgres/gist/papers/concurrency/access-methods-for-next-generation.pdf.gz -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-02 Thread Alexander Korotkov
ffset - (0001004000D2,10982288) (1 row) May be you have any ideas about it? -- With best regards, Alexander Korotkov. gist_fast_build-0.9.0.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] WIP: Fast GiST index build

2011-08-03 Thread Alexander Korotkov
only to 44/9C518750" appears. Seems that there is some totally wrong use of WAL if even optimization level does matter... -- With best regards, Alexander Korotkov. gist_fast_build-0.9.1.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] WIP: Fast GiST index build

2011-08-04 Thread Alexander Korotkov
Uhh, my bad, really stupid bug. Many thanks. -- With best regards, Alexander Korotkov. On Wed, Aug 3, 2011 at 8:31 PM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > On 03.08.2011 11:18, Alexander Korotkov wrote: > >> I found that in previous version

Re: [HACKERS] WIP: Fast GiST index build

2011-08-07 Thread Alexander Korotkov
items. If we imagine extremely large tree with levelstep = 1 size of this datastructures will be singnificant. And it's hard to predict that size without knowing of tree size. -- With best regards, Alexander Korotkov. gist_fast_build-0.10.0.patch.gz Description: GNU Zip compressed data

Re: [HACKERS] WIP: Fast GiST index build

2011-08-08 Thread Alexander Korotkov
t; * Any user-visible options need to be documented in the user manual. > * And of course, make sure comments and the readme are up-to-date. > * Compiler warning: > > reloptions.c:259: warning: initializer-string for array of chars is too > long > reloptions.c:259: warning: (near initialization for > ‘stringRelOpts[0].default_val’**) > > I don't think there's a way to add an entry to stringRelOpts in a way that > works correctly. That's a design flaw in the reloptions.c code that has > never come up before, as there hasn't been any string-formatted relopts > before (actually buffering option might be better served by an enum > reloption too, if we had that). Please start a new thread on that on > pgsql-hackers. Ok. -- With best regards, Alexander Korotkov.

[HACKERS] Compiler warnings with stringRelOpts (was WIP: Fast GiST index build)

2011-08-08 Thread Alexander Korotkov
pts[0].default_val’**) It is caused by definition of default field of relopt_string structure as 1-length character array. This seems to be a design flaw in the reloptions.c code. Any thoughts? -- With best regards, Alexander Korotkov.

Re: [HACKERS] Compiler warnings with stringRelOpts (was WIP: Fast GiST index build)

2011-08-08 Thread Alexander Korotkov
is possible? -- With best regards, Alexander Korotkov.

Re: [HACKERS] Compiler warnings with stringRelOpts (was WIP: Fast GiST index build)

2011-08-08 Thread Alexander Korotkov
it also seems that both were added by your commit of table-based parser: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=ba748f7a11ef884277b61d1708a17a44acfd1736 -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-10 Thread Alexander Korotkov
effective_cache_size. 4) Some renames. In particular GISTLoadedPartItem to GISTBufferingInsertStack. 5) Some comments were corrected and some were added. 6) pgindent 7) rebased with head Readme update and user documentation coming soon. -- With best regards, Alexander Korotkov. gist_fast_build-0.11.0

Re: [HACKERS] WIP: Fast GiST index build

2011-08-10 Thread Alexander Korotkov
Manual and readme updates. -- With best regards, Alexander Korotkov. gist_fast_build-0.12.0.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] WIP: Fast GiST index build

2011-08-11 Thread Alexander Korotkov
ptional. We can try it as add-on patch. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-11 Thread Alexander Korotkov
build(). That's why I avoided deletion from lists. Now I've added additional check for root node in loop over list. > The calculations in initBuffering() desperately need comments. As does the > rest of the code too, but the heuristics in that function are particularly > hard t

Re: [HACKERS] WIP: Fast GiST index build

2011-08-11 Thread Alexander Korotkov
On Thu, Aug 11, 2011 at 2:28 PM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > On 10.08.2011 22:44, Alexander Korotkov wrote: > >> Manual and readme updates. >> > > Thanks, I'm reviewing these now. > > Do we want to expose the leve

Re: [HACKERS] WIP: Fast GiST index build

2011-08-12 Thread Alexander Korotkov
-level > buffer. Wouldn't it be fairly simple to detect that case explicitly, and > stop the emptying process only if one of the lower-level buffers really > fills up? That should be more efficient, as you would have "swap" between > different subtrees less often. Yes, it seems reasonable to me. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-14 Thread Alexander Korotkov
Hi! Thank you for your notes. On Fri, Aug 12, 2011 at 7:04 PM, Robert Haas wrote: > On Thu, Aug 11, 2011 at 6:21 AM, Alexander Korotkov > wrote: > > [ new patch ] > > Some random comments: > > - It appears that the "noFollowFight" flag is really supp

Re: [HACKERS] WIP: Fast GiST index build

2011-08-16 Thread Alexander Korotkov
I found that I forgot to remove levelstep and buffersize from reloptions.c. Updated patch is attached. -- With best regards, Alexander Korotkov. gist_fast_build-0.14.1.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] WIP: Fast GiST index build

2011-08-16 Thread Alexander Korotkov
Thus, we can calculate: levelstep = min(log_r(1 + effective_cache_size_in_pages*(r - 1)) - 1, log_r(maintenance_work_mem_in_pages - 1)) and get more precise result. But also we need at least very rough estimate of memory occupied by node buffers hash tab and path items. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-16 Thread Alexander Korotkov
u might as well pull tuples directly from the > heap. > Yes, seems reasonable. Buffer on the root node was in the paper. But now I don't see the need of it too. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-17 Thread Alexander Korotkov
an attachment... Thanks a lot. I'm going to start rerunning the tests now. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-22 Thread Alexander Korotkov
On Wed, Aug 17, 2011 at 11:11 AM, Alexander Korotkov wrote: > On Tue, Aug 16, 2011 at 11:15 PM, Heikki Linnakangas < > heikki.linnakan...@enterprisedb.com> wrote: > >> On 16.08.2011 22:10, Heikki Linnakangas wrote: >> >>> Here's an version of the patch wit

Re: [HACKERS] UUID datatype GiST index support

2011-08-22 Thread Alexander Korotkov
eciated if anyone could help me and suggest the best > way to make Gist support for UUID datatype... > I think you're on the right way. btree_gist is an extension which provides GiST indexing of scalar datatype. UUID is one of them. So, the module you are writing should be quite similar. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-24 Thread Alexander Korotkov
It runs more than 60 hours and while only 50% of index is complete (estimated by file sizes). Also, automatic switching to buffering build shows better index quality results in all the tests. While it's hard for me to explain that. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-26 Thread Alexander Korotkov
is appropriate only for testing purposes. 5) For indexes creation I use simple script which is attached as 'indexes.sql'. Also, similar script with different index names I'm running with my split patch. Feel free to ask questions about all this stuff. -- With best regards, Alexande

Re: [HACKERS] WIP: Fast GiST index build

2011-08-26 Thread Alexander Korotkov
fy this a little. I don't think my split algorithm is 10 times better than state of the art algorithms. I think that currently used new linear split shows unreasonably bad results in may cases. For example, uniformly distributed data is pretty easy case. And with almost any splitting algorithm we can get index with almost zero overlaps. But new linear split produces huge overlaps in this case. That's why I decided to make some experiments with another split algorithm. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-30 Thread Alexander Korotkov
g. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-30 Thread Alexander Korotkov
generation on the backend like this? CREATE TABLE points AS (SELECT point(random(), random() FROM generate_series(1,1000)); -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-08-30 Thread Alexander Korotkov
On Tue, Aug 30, 2011 at 9:29 PM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > On 30.08.2011 13:29, Alexander Korotkov wrote: > >> On Tue, Aug 30, 2011 at 1:13 PM, Heikki Linnakangas< >> heikki.linnakangas@**enterprisedb.com> >> wrote: &g

Re: [HACKERS] WIP: Fast GiST index build

2011-09-01 Thread Alexander Korotkov
your test setup (it this case I advice to start from smaller number of rows, 100 M for example). I'm requesting real-life datasets which makes troubles in real life from Oleg. Probably those datasets is even larger or new linear split produce less overlaps on them. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: SP-GiST, Space-Partitioned GiST

2011-09-01 Thread Alexander Korotkov
tographer's 28433 | cartography 28435 | carton 28437 | cartons 28439 | cartoon's 28441 | cartooning 28443 | cartoonist's 28445 | cartoons 28447 | cartridge's 28449 | carts 28451 | cartwheel's 28453 | cartwheeling 28455 | carve 28457 | carver 28459 | carvers 28461 | carving 28463 | carvings (23 rows) -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-09-05 Thread Alexander Korotkov
Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum doesn't exceed maximal offset number. -- With best regards, Alexander Korotkov. gist_fast_build-0.14.3.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-ha

Re: [HACKERS] WIP: Fast GiST index build

2011-09-06 Thread Alexander Korotkov
> unlogged tables? > With tecnhique discussing in this thread GiST build can win form unlogged as much as with regular build. -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: SP-GiST, Space-Partitioned GiST

2011-09-06 Thread Alexander Korotkov
like btree with *_pattern_ops. (Oleg will corect me if I'm missing something) For the queries you mentioned you may see LIKE acceleration in wildspeed module (http://www.sai.msu.su/~megera/wiki/wildspeed) and pg_trgm of 9.1 ( http://www.postgresql.org/docs/9.1/static/pgtrgm.html). -- With best regards, Alexander Korotkov.

Re: [HACKERS] WIP: Fast GiST index build

2011-09-08 Thread Alexander Korotkov
Thanks for congratulations! Tnanks to Heikki for mentoring and his work on patch! -- With best regards, Alexander Korotkov.

[HACKERS] Double sorting split patch

2011-09-11 Thread Alexander Korotkov
3 -> Bitmap Index Scan on geonames_double_sorting_idx (cost=0.00..344.11 rows=7604 width=0) (actual time=0.293..0.293 rows=10 loops=1) Index Cond: (point <@ '(46.2088,-104.65),(46.1384,-104.72)'::box) Buffers: shared hit=7 Total runtime: 0.429 ms (7 rows) --

Re: [HACKERS] What Would You Like To Do?

2011-09-13 Thread Alexander Korotkov
t. * Signature indexing with gist/spgist in various fields. For example, indexing of image signatures with similar images retreival. * Statistics collection and selectivity estimation for geometric datatypes. -- With best regards, Alexander Korotkov.

<    5   6   7   8   9   10   11   12   >