Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-03 Thread Tomas Vondra
Hi, On 11/02/2017 06:45 PM, Alvaro Herrera wrote: > Tomas Vondra wrote: > >> Unfortunately, I think we still have a problem ... I've been wondering >> if we end up producing correct indexes, so I've done a simple test. > > Here's a proposed patch that s

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Tomas Vondra
On 10/31/2017 11:44 PM, Tomas Vondra wrote: > ... > Unfortunately, I think we still have a problem ... I've been wondering > if we end up producing correct indexes, so I've done a simple test. > > 1) create the table as before > > 2) let the insert + vacuum run

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Tomas Vondra
select count(*) from brin_test where a = 0; count --- 9062 (1 row) test=# set enable_bitmapscan = off; SET test=# select count(*) from brin_test where a = 0; count --- 9175 (1 row) Attached is a SQL script with commands I used. You'll need

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-31 Thread Tomas Vondra
or handling / killing suspended workers (which didn't occur to me before as a possible issue at all, so that's for pointing that out). But that's a significantly more limited issue to fix than all the parallel-unsafe bits. Now, I agree this is somewhat more limited than I hoped fo

Re: [HACKERS] Re: Anyone have experience benchmarking very high effective_io_concurrency on NVME's?

2017-10-31 Thread Tomas Vondra
then the gains become much more modest - not because the device could not handle more, but because of the prefetch/processing ratio reached the optimal value. But all this is actually per-process, if you can run multiple backends (particularly when doing bitmap index scans), I'm sure you'l

Re: [HACKERS] WIP: BRIN multi-range indexes

2017-10-30 Thread Tomas Vondra
tps://www.postgresql.org/message-id/5d78b774-7e9c-c94e-12cf-fef51cc89b1a%402ndquadrant.com -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 0001-Pass-all-keys-to-BRIN-consistent-function-at-once.patch.gz De

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-30 Thread Tomas Vondra
rm Michael's findings - I've been unable to reproduce the issue on 1a4be103a5 even after 20 minutes, and on 24992c6db9 it failed after only 2. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services --

[HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-29 Thread Tomas Vondra
FWIW I can reproduce this on REL_10_STABLE, but not on REL9_6_STABLE. So it seems to be due to something that changed in the last release. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent

[HACKERS] PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-29 Thread Tomas Vondra
th the message, only the value (offset) changes. The stack trace always looks exactly the same - see the attachment. At first it seemed the idxrel is always the index on 'e' (i.e. the UUID column), but it seems I also got failures on the other indexes. regards -- Tomas Vondra

Re: [HACKERS] WIP: BRIN bloom indexes

2017-10-27 Thread Tomas Vondra
hi, On 10/28/2017 02:41 AM, Nico Williams wrote: > On Fri, Oct 27, 2017 at 10:06:58PM +0200, Tomas Vondra wrote: >>> + * We use an optimisation that initially we store the uint32 values >>> directly, >>> + * without the extra hashing step. And only later fillin

Re: [HACKERS] WIP: BRIN bloom indexes

2017-10-27 Thread Tomas Vondra
Hi, On 10/27/2017 05:22 PM, Sokolov Yura wrote: > > Hi, Tomas > > BRIN bloom index is a really cool feature, that definitely should be in > core distribution (either in contrib or builtin)!!! > > Small suggestion for algorithm: > > It is well known practice

Re: [HACKERS] WIP: BRIN bloom indexes

2017-10-27 Thread Tomas Vondra
Hi, On 10/27/2017 07:17 PM, Nico Williams wrote: > On Thu, Oct 19, 2017 at 10:15:32PM +0200, Tomas Vondra wrote: > > A bloom filter index would, indeed, be wonderful. > > Comments: > > + * We use an optimisation that initially we store the uint32 values directly, &g

Re: [HACKERS] WIP: BRIN bloom indexes

2017-10-27 Thread Tomas Vondra
hi, On 10/27/2017 09:34 AM, Simon Riggs wrote: > On 27 October 2017 at 07:20, Robert Haas wrote: >> On Thu, Oct 19, 2017 at 10:15 PM, Tomas Vondra >> wrote: >>> Let's see a query like this: >>> >>> select * from bloom_test >>&g

Re: [HACKERS] Burst in WAL size when UUID is used as PK while full_page_writes are enabled

2017-10-27 Thread Tomas Vondra
af pages will be created but merely that the inserts will touch a different (possibly existing) leaf page. That's a direct consequence of the inherent UUID randomness. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Trai

Re: [HACKERS] WIP: BRIN multi-range indexes

2017-10-25 Thread Tomas Vondra
Apparently I've managed to botch the git format-patch thing :-( Attached are both patches (the first one adding BRIN bloom indexes, the other one adding the BRIN multi-range). Hopefully I got it right this time ;-) regards -- Tomas Vondra http://www.2ndQuadrant.com Postg

[HACKERS] WIP: BRIN multi-range indexes

2017-10-25 Thread Tomas Vondra
And I haven't modified the existing brin_minmax_consistent() function (yeah, I'm lazy, but this should be enough for interested people to try it out I believe). 2) It only works with float8 (and also timestamp data types) for now, but it should be straightforward to add support for addition

[HACKERS] CUBE seems a bit confused about ORDER BY

2017-10-19 Thread Tomas Vondra
6, 333) 377 | (11399, 377),(11360, 294) (15 rows) Seems like a bug somewhere in gist_cube_ops, I guess? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pg

[HACKERS] WIP: BRIN bloom indexes

2017-10-19 Thread Tomas Vondra
uuid, md5(i::text) FROM generate_series(1,2) s(i); VACUUM ANALYZE bloom_test; List of relations Schema |Name| Type | Owner | Size | Description ++---+---+---+- public | bloom_test | table | tomas

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-17 Thread Tomas Vondra
On 10/17/2017 03:16 PM, Robert Haas wrote: > On Sat, Oct 14, 2017 at 2:14 PM, Tomas Vondra > wrote: >> I propose is to add a new cursor option (PARALLEL), which would allow >> parallel plans for that particular user-defined cursor. Attached is an >> experimental patch do

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Tomas Vondra
he same compilation options you can run pg_config, look for CONFIGURE line and then just modify add --prefix option. And after `make install` you can add it to $PATH and start the server using those binaries. $ export PATH=/path/to/alternative/binaries/bin:$PATH $ which pg_ctl $ pg_ctl -D $DATADIR

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-16 Thread Tomas Vondra
On 10/16/2017 01:13 PM, Amit Kapila wrote: > On Sat, Oct 14, 2017 at 11:44 PM, Tomas Vondra > wrote: >> Hi, >> >> >> I propose is to add a new cursor option (PARALLEL), which would allow >> parallel plans for that particular user-defined cursor. Attached is

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-15 Thread Tomas Vondra
> http://telsasoft.com/tmp/coredump-postgres-autovacuum-brin-summarize.gz > Thanks, but I'm not sure that'll help, at this point. We already know what happened (corrupted memory), we don't know "how". And core files are mostly just "snapshots" so are not very u

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-14 Thread Tomas Vondra
no 1 :varattno 1 :vartype 1184 > :vartypmod -1 :varcollid 0 :varlevelsup 0 :varnoold 1 :varoattno 1 :location > 146} {CONST :consttype 1184 :consttypmod -1 :constcollid 0 :constlen 8 > :constbyv"... > cur_relname = 0x298cd68 > "cdrs_eric_msc_sms_2017_10_14_startof

Re: [HACKERS] Extended statistics is not working on Vars hidden under a RelabelType

2017-10-14 Thread Tomas Vondra
x27;s rather extreme (statistics on empty table). So again, likely well within noise, and on larger tables it'll get even less significant. But of course - it's not free. It's a bit more work we need to do. But if you don't need multi-column statistics, don't create them. I

[HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-14 Thread Tomas Vondra
fJ%2B4SQwgG%3D6CVHWoisiU0%2B7jtXSuiyXBM3y%3DA%3DeJzmg%40mail.gmail.com -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c index

Re: [HACKERS] Extended statistics is not working on Vars hidden under a RelabelType

2017-10-13 Thread Tomas Vondra
enerated on a assert-enabled build and on a laptop (both of which adds quite a bit of noise to occasional timings). The patch has no impact on planning time (at least I've been unable to measure any). regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Developme

Re: [HACKERS] Parallel Bitmap Heap Scans segfaults due to (tbm->dsa==NULL) on PostgreSQL 10

2017-10-12 Thread Tomas Vondra
On 10/12/2017 02:40 PM, Dilip Kumar wrote: > On Thu, Oct 12, 2017 at 4:31 PM, Tomas Vondra > wrote: >> Hi, >> >> It seems that Q19 from TPC-H is consistently failing with segfaults due >> to calling tbm_prepare_shared_iterate() with (tbm->dsa==NULL). >> &g

[HACKERS] Parallel Bitmap Heap Scans segfaults due to (tbm->dsa==NULL) on PostgreSQL 10

2017-10-12 Thread Tomas Vondra
uery: * query.sql * plan.txt * backtrace.txt 2) details for the "minimal" query triggering the issue: * query-minimal.sql * plan-minimal.txt * backtrace-minimal.txt regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA,

[HACKERS] Re: Extended statistics is not working on Vars hidden under a RelabelType

2017-10-10 Thread Tomas Vondra
ther node. > Thanks for noticing this. The patch seems fine to me. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] PATCH : Generational memory allocator (was PATCH: two slab-like memory allocators)

2017-09-24 Thread Tomas Vondra
away. 3) I've also added a brief note into reorderbuffer.c mentioning that it uses SlabContext and GenerationContext. As I explained, I don't think we should include details about how we tested the patch or whatever here. regard -- Tomas Vondra http://www.2ndQuadrant.co

Re: [HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-09-24 Thread Tomas Vondra
s to be actually bitten by it. For people relying on autovacuum that won't happen, as it only runs on tables with certain number of dead tuples. So you'd have to be running VACUUM in a loop or something (but not VACUUM ANALYZE, because that works fine) from a script, or something

Re: [HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-09-24 Thread Tomas Vondra
On 09/06/2017 09:45 AM, Haribabu Kommi wrote: > > > On Tue, Jul 25, 2017 at 9:33 PM, Tomas Vondra > mailto:tomas.von...@2ndquadrant.com>> wrote: > > On 7/25/17 12:55 AM, Tom Lane wrote: > > Tomas Vondra <mailto:tomas.von...@2ndquadrant.

Re: [HACKERS] user-defined numeric data types triggering ERROR: unsupported type

2017-09-23 Thread Tomas Vondra
On 09/21/2017 04:24 PM, Tom Lane wrote: > Tomas Vondra writes: >> [ scalarineqsel may fall over when used by extension operators ] > > I concur with your thought that we could have > ineq_histogram_selectivity fall back to a "mid bucket" default if > it's wor

Re: [HACKERS] Boom filters for hash joins (was: A design for amcheck heapam verification)

2017-09-19 Thread Tomas Vondra
On 09/19/2017 06:03 PM, Peter Geoghegan wrote: > On Tue, Sep 19, 2017 at 6:28 AM, Tomas Vondra > wrote: >> The patch is fairly simple, and did not try to push the bloom filters to >> scan nodes or anything like that. It might be a meaningful first step, >> though, particul

Re: [HACKERS] Boom filters for hash joins (was: A design for amcheck heapam verification)

2017-09-19 Thread Tomas Vondra
in be a lot more likely in this case anyway? Low >> selectivity hash joins with multiple batches are inherently slow; the >> wasted overhead of using a bloom filter may not matter. >> >> Obviously this is all pretty speculative. I suspect that this could be >> true, and it s

[HACKERS] user-defined numeric data types triggering ERROR: unsupported type

2017-09-19 Thread Tomas Vondra
) (2 rows) That is, we get 50% estimate, because that's what scalarineqsel uses when it ineq_histogram_selectivity can't compute selectivity from a histogram for some reason. Wouldn't it make it more sense to use the default 33% estimate here? regards -- Tomas Vondra

[HACKERS] valgrind vs. shared typmod registry

2017-09-16 Thread Tomas Vondra
27;ve been running tests under valgrind not too long ago and I don't recall such failures, so perhaps something broke it in the past few days. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services ==18897

Re: [HACKERS] PATCH : Generational memory allocator (was PATCH: two slab-like memory allocators)

2017-09-14 Thread Tomas Vondra
On 09/14/2017 04:21 PM, Simon Riggs wrote: > On 14 August 2017 at 01:35, Tomas Vondra wrote: >> Hi, >> >> Attached is a rebased version of the Generational context, originally >> submitted with SlabContext (which was already committed into Pg 10). >> >> The

Re: [HACKERS] Hooks to track changed pages for backup purposes

2017-09-13 Thread Tomas Vondra
tem >> can make it as durable as the WAL segments, etc. > > Well, in some not so rare cases users encrypt backups and send to S3. > And there is no system with CPUs that can handle that WAL parsing. > Currently, I'm considering mocking prototype for wal-g, which works > e

Re: [HACKERS] Patches that don't apply or don't compile: 2017-09-12

2017-09-13 Thread Tomas Vondra
Hi Aleksander, On 09/13/2017 11:49 AM, Aleksander Alekseev wrote: > Hi Tomas, > > I appreciate your feedback, although it doesn't seem to be completely > fair. Particularly: > >> You gave everyone about 4 hours to object > > This is not quite accurate since my p

Re: [HACKERS] Patches that don't apply or don't compile: 2017-09-12

2017-09-12 Thread Tomas Vondra
l be just annoying and disrupting. I suggest we inspect the reported patches manually, investigate whether the failures are legitimate or not, and eliminate as many false positives as possible. Once we are happy with the accuracy, we can enable it again. kind regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2017-09-12 Thread Tomas Vondra
Attached is an updated version of the patch, dealing with fallout of 821fb8cdbf700a8aadbe12d5b46ca4e61be5a8a8 which touched the SGML documentation for CREATE STATISTICS. regards On 09/07/2017 10:07 PM, Tomas Vondra wrote: > Hi, > > Attached is an updated version of the patch, fixing t

Re: [HACKERS] The case for removing replacement selection sort

2017-09-11 Thread Tomas Vondra
say. So I think you're right it to 1B rows may break this assumption, and make it perform worse. But perhaps the fact that we're testing with multiple work_mem values, and with smaller data sets (100k or 1M rows) makes this a non-issue? regards -- Tomas Vondra http://

Re: [HACKERS] Automatic testing of patches in commit fest

2017-09-11 Thread Tomas Vondra
On 09/11/2017 03:01 PM, Aleksander Alekseev wrote: > Hi Tomas, > >>> Unless there are any objections to give this idea a try I'm willing to >>> write and host a corresponding script. >>> >> That won't work until (2) is reliable enough. There

Re: [HACKERS] Remove 1MB size limit in tsvector

2017-09-11 Thread Tomas Vondra
ed tsvectors. So the question is how realistic that benchmark actually is. How likely are we to do queries on fts directly, not through a GIN/GiST index? Particularly in performance-sensitive cases? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7

Re: [HACKERS] Automatic testing of patches in commit fest

2017-09-11 Thread Tomas Vondra
would be damaging, as (a) there's nothing wrong with the patch, and (b) it's not clear what to do to fix the problem. So -1 to this for now, until we make this part smart enough. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remo

Re: [HACKERS] Polyphase merge is obsolete

2017-09-11 Thread Tomas Vondra
Hi, I planned to do some benchmarking on this patch, but apparently the patch no longer applies. Rebase please? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing

Re: [HACKERS] The case for removing replacement selection sort

2017-09-10 Thread Tomas Vondra
On 09/11/2017 02:22 AM, Peter Geoghegan wrote: > On Sun, Sep 10, 2017 at 5:07 PM, Tomas Vondra > wrote: >> I'm currently re-running the benchmarks we did in 2016 for 9.6, but >> those are all sorts with a single column (see the attached script). But >> it'd be

Re: [HACKERS] The case for removing replacement selection sort

2017-09-10 Thread Tomas Vondra
somehow, only testing the largest/smallest work_mem values? So that we could get some numbers now, possibly running the complete test suite later. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services sort-benc

Re: [HACKERS] WIP: Separate log file for extension

2017-09-09 Thread Tomas Vondra
n extension attempts to use LOG_DESTINATION_CSVLOG? Or will it start a separate collector for each such extension? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mail

Re: [HACKERS] segment size depending *_wal_size defaults (was increasing the default WAL segment size)

2017-09-09 Thread Tomas Vondra
rd to GUC support. > It's not hard - it's just a lot of copy-pasting of infrastructure code. Incidentally, I already have a patch doing that, as we had to add that support to XL, and I can submit it to PostgreSQL. Might save some boring coding. regards -- Tomas Vondra

Re: [HACKERS] psql: new help related to variables are not too readable

2017-09-09 Thread Tomas Vondra
On 09/09/2017 01:24 AM, Tom Lane wrote: > Tomas Vondra writes: >> The translator has exactly the same context in both cases, and the >> struggle to wrap it at 80 characters will be pretty much the same too. > > Really? With the old way, you had something under 60 charact

Re: [HACKERS] psql: new help related to variables are not too readable

2017-09-08 Thread Tomas Vondra
t a matter of > taste and I expect there should be half a dozen different opinions on > the matter of formatting. > FWIW, +1 to extra lines from me - I find it way more readable, as it clearly separates the items. You're right this is also a matter of taste to some degree, so I unde

Re: [HACKERS] [PATCH] Pageinspect - add functions on GIN and GiST indexes from gevel

2017-09-07 Thread Tomas Vondra
l pages. Why should it be part of pageinspect? For example we have pgstattuple extension, which seems like a better match. Or just create a new extension - if the code is valuable, surely we can live one more extension instead of smuggling it in inside pageinspect. regards -- Tomas Vondra

Re: [HACKERS] The case for removing replacement selection sort

2017-09-07 Thread Tomas Vondra
ment selection actually did do > better with on 9.6. I clearly remember Tomas Vondra doing lots of > benchmarking, showing some benefit with RS with a work_mem of 8MB or > less. As I said in my introduction on this thread, we weren't wrong to > add replacement_sort_tuples to 9.6, given

Re: [HACKERS] GnuTLS support

2017-09-07 Thread Tomas Vondra
;t be able to set the inactive options, just like you can't set ssl=on when you build without OpenSSL support. But perhaps we could simply not include the inactive options into the config file, no? I don't see how breaking the TLS config into separate files improves the situation - inste

Re: [HACKERS] Remove 1MB size limit in tsvector

2017-09-07 Thread Tomas Vondra
shared buffers, but also page cache). Can you share the benchmarks, so that others can retry running them? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] Hooks to track changed pages for backup purposes

2017-09-07 Thread Tomas Vondra
s, has to actually re-read the WAL segments from disk (which may be a lot of I/O, particularly when done from multiple processes). >From this POV, the idea to collect this information on the backup system (WAL archive) by pre-processing the arriving WAL segments seems like the most promising. I

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2017-09-07 Thread Tomas Vondra
Hi, Attached is an updated version of the patch, fixing the issues reported by Adrien Nayrat, and also a bunch of issues pointed out by valgrind. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 0001

Re: [HACKERS] error-handling in ReorderBufferCommit() seems somewhat broken

2017-08-29 Thread Tomas Vondra
On 08/30/2017 02:19 AM, Andres Freund wrote: > On 2017-08-30 02:11:10 +0200, Tomas Vondra wrote: >> >> I'm not really following your reasoning. You may very well be right that >> the BeginInternalSubTransaction() example is not quite valid on postgres >> core,

Re: [HACKERS] error-handling in ReorderBufferCommit() seems somewhat broken

2017-08-29 Thread Tomas Vondra
On 08/30/2017 01:34 AM, Andres Freund wrote: > Hi, > > On 2017-08-30 01:27:34 +0200, Tomas Vondra wrote: >> I've been investigating some failures in test_decoding regression tests, >> and it seems to me the error-handling in ReorderBufferCommit() is >> some

[HACKERS] error-handling in ReorderBufferCommit() seems somewhat broken

2017-08-29 Thread Tomas Vondra
s-XL 10, which does not support subtransactions and so the BeginInternalSubTransaction() call in the try branch always fails, triggering the issue. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sen

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2017-08-26 Thread Tomas Vondra
; > > Both crashes. > Thanks for the report, this is clearly a bug. I don't think we need to test the stxkind, but rather a missing check that the requested type is actually built. > Unfotunately, I don't have the knowledge to produce a patch :/ > > Small fix in d

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-08-15 Thread Tomas Vondra
On 08/15/2017 09:55 PM, Robert Haas wrote: On Tue, Aug 15, 2017 at 3:42 PM, Tomas Vondra wrote: What I think we should not do is interpret the bitmasks (omitting some of the information) assuming all the bits were set correctly. I'm still confused. HEAP_XMIN_COMMITTED|HEAP_XMIN_AB

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-08-15 Thread Tomas Vondra
On 08/15/2017 07:54 PM, Robert Haas wrote: On Tue, Aug 15, 2017 at 9:59 AM, Tomas Vondra wrote: I don't think so -- the "committed" and "invalid" meanings are effectively canceled when the "frozen" mask is present. I mean, "committed" and &quo

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-08-15 Thread Tomas Vondra
nvalid" contradict each other... FWIW I agree with Craig - the functions should output the masks raw, without any filtering. The reason is that when you're investigating data corruption or unexpected behavior, all this is very useful when reasoning about what might (not) have hap

[HACKERS] PATCH : Generational memory allocator (was PATCH: two slab-like memory allocators)

2017-08-13 Thread Tomas Vondra
and timing. Admittedly, this is a single test, so ideas of other useful test cases are welcome. regards [1] https://www.postgresql.org/message-id/20170227111732.vrx5v72ighehwpkf%40alap3.anarazel.de [2] https://www.postgresql.org/message-id/20160706185502.1426.28143%40wrigleys.postgresql

Re: [HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-07-25 Thread Tomas Vondra
On 7/25/17 5:04 PM, Tom Lane wrote: Tomas Vondra writes: On 7/25/17 12:55 AM, Tom Lane wrote: I think the planner basically assumes that reltuples is the live tuple count, so maybe we'd better change VACUUM to get in step. Attached is a patch that (I think) does just that

Re: [HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-07-25 Thread Tomas Vondra
On 7/25/17 12:55 AM, Tom Lane wrote: Tomas Vondra writes: It seems to me that VACUUM and ANALYZE somewhat disagree on what exactly reltuples means. VACUUM seems to be thinking that reltuples = live + dead while ANALYZE apparently believes that reltuples = live The question is - which of the

[HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-07-24 Thread Tomas Vondra
ven't looked at the code yet, but I've confirmed this happens both on 9.6 and 10. I haven't checked older versions, but I guess those are affected too. The question is - which of the reltuples definitions is the right one? I've always assumed that "reltuples = live + d

Re: [HACKERS] Pluggable storage

2017-06-26 Thread Tomas Vondra
le how not to do things - there's also a plenty of references in the MySQL "Restrictions and Limitations" section of the manual: https://downloads.mysql.com/docs/mysql-reslimits-excerpt-5.7-en.pdf regards -- Tomas Vondra http://www.2ndQuadrant.com Postgr

Re: [HACKERS] Pluggable storage

2017-06-23 Thread Tomas Vondra
bitmap would not result in mostly sequential access pattern and things like prefetch would not be very efficient, I think. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers

Re: [HACKERS] Pluggable storage

2017-06-22 Thread Tomas Vondra
d be impossible to implement with option #1? I'm not trying to be annoying or anything like that - I don't know the answer and discussing those things is exactly why this thread exists. I do agree #1 has limitations, and that it'd be great to get API that supports all k

Re: [HACKERS] Pluggable storage

2017-06-22 Thread Tomas Vondra
ptr->ip_posid < (1 << MaxHeapTuplesPerPageBits)) But perhaps there are some other issues? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postg

Re: [HACKERS] Parallel Aggregation support for aggregate functions that use transitions not implemented for array_agg

2017-06-18 Thread Tomas Vondra
you can (for example) sort the per-worker results as part of the "serialize" function, and benefit from that while combining that in the gather, then sure, that should be a huge win. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Supp

Re: [HACKERS] WIP: Data at rest encryption

2017-06-18 Thread Tomas Vondra
course, if you happen to use another filesystem (e.g. XFS), this won't work for you. But then there's eCryptfs, for example: https://en.wikipedia.org/wiki/ECryptfs regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] TPC-H Q20 from 1 hour to 19 hours!

2017-06-11 Thread Tomas Vondra
it in the archives. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] TPC-H Q20 from 1 hour to 19 hours!

2017-05-25 Thread Tomas Vondra
Hi, On 5/25/17 6:03 AM, Robert Haas wrote: On Thu, Apr 6, 2017 at 4:37 PM, Tomas Vondra wrote: Which brings me to the slightly suspicious bit. On 9.5, there's no difference between GROUP and GROUP+LIKE cases - the estimates are exactly the same in both cases. This is true too, but

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-12 Thread Tomas Vondra
haps a price for the simplicity of this statistics type. If your application executes queries that are likely not consistent with this, don't use functional dependencies. The simplicity is why dependencies were implemented first, mostly to introduce all the infrastructur

Re: [HACKERS] CTE inlining

2017-05-04 Thread Tomas Vondra
ly once" and "producing the same result as if executed exactly once". I may be misunderstanding what other people proposed in this thread, but I think the plan was to only inline CTEs where we know it won't change the results, etc. So e.g. CTEs with volatile functions would

Re: [HACKERS] CTE inlining

2017-05-04 Thread Tomas Vondra
ever coders ... nextval is a volatile function, and in my understanding the plan was not to inline CTEs with volatile functions (or CTEs doing writes etc.). That is, we'd guarantee the same results as we get now. regards -- Tomas Vondra http://www.2ndQuadrant.com Pos

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-03 Thread Tomas Vondra
parentheses for the USING clause? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.

Re: [HACKERS] CTE inlining

2017-05-03 Thread Tomas Vondra
No anger, but nobody is happy either. +1 for option 1. And while I would not like if we had to combine it with a backwards compatibility GUC which enables the old behavior to get it merged I still personally would prefer that over option 2 and 3. > Andreas > +1 to what Andreas says -- Toma

Re: [HACKERS] CTE inlining

2017-05-02 Thread Tomas Vondra
On 5/2/17 11:23 PM, Merlin Moncure wrote: \On Tue, May 2, 2017 at 12:05 PM, Tomas Vondra wrote: On 5/2/17 6:34 PM, David Fetter wrote: On Tue, May 02, 2017 at 02:40:55PM +0200, Andreas Karlsson wrote: On 05/02/2017 04:38 AM, Craig Ringer wrote: On 1 May 2017 at 22:26, Andreas Karlsson

Re: [HACKERS] CTE inlining

2017-05-02 Thread Tomas Vondra
acility. This is a lot of work and something which would be very hard to get consensus for. Just the design of the thing would be the work of months at a minimum, assuming we got to some consensus at all. Maybe it's worth doing. While I came to conclusion that query hints may be qu

Re: [HACKERS] CTE inlining

2017-05-02 Thread Tomas Vondra
realize OFFSET 0 is no-op? In that case replacing CTE optimization fence with "OFFSET 0" would be akin to painting yourself into a corner, waiting for the pain to dry, walking over to another corner and painting yourself into that one. cheers -- Tomas Vondra ht

Re: [HACKERS] CTE inlining

2017-05-01 Thread Tomas Vondra
be impossible to use multiple CTEs in the query with different fencing behavior, and it would be just as difficult to investigate. So no more planner-affecting GUCs, please, particularly if we expect regular users to use them. regards -- Tomas Vondra http://www.2ndQuadran

Re: [HACKERS] CTE inlining

2017-04-30 Thread Tomas Vondra
ould be great. Say, being able to define indexes on them, but that's really a separate topic. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] CTE inlining

2017-04-30 Thread Tomas Vondra
multiple attempts at this in the past, none of them succeeded. But perhaps we could at least propagate some of the CTE features, so that the outside query can benefit from that (e.g. when the CTE is sorted, we could set the sortkeys). That wouldn't break the fence thing, but it would

Re: [HACKERS] to-do item for explain analyze of hash aggregates?

2017-04-24 Thread Tomas Vondra
On 04/24/2017 10:55 PM, Jeff Janes wrote: On Mon, Apr 24, 2017 at 12:13 PM, Tomas Vondra mailto:tomas.von...@2ndquadrant.com>> wrote: On 04/24/2017 08:52 PM, Andres Freund wrote: ... I've wanted that too. It's not impossible at all. Why wouldn't

Re: [HACKERS] to-do item for explain analyze of hash aggregates?

2017-04-24 Thread Tomas Vondra
ldn't that be possible? We probably can't use exactly the same approach as Hash, because hashjoins use custom hash table while hashagg uses dynahash IIRC. But why couldn't measure the amount of memory by looking at the memory context, for example? regards -- Tomas Vondra

Re: Do we need multi-column frequency/histogram stats? WAS Re: [HACKERS] Statistics "dependency"

2017-04-23 Thread Tomas Vondra
s. The other types of statistics actually track correlation between values in the columns, not just "column A implies column B". regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via

Re: [HACKERS] Statistics "dependency"

2017-04-23 Thread Tomas Vondra
dependency stored, hence "dependencies". I don't like it, but its the best term I can see at present. That is a good point, actually. It should probably be plural. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-04-20 Thread Tomas Vondra
re a < 100 with (fillfactor=10); ERROR: syntax error at or near "with" LINE 1: create index on t (a) where a < 100 with (fillfactor=10); ^ test=# create index on t (a) with (fillfactor=10) where a < 100; regards -- Tomas

Re: [HACKERS] pg_statistic_ext.staenabled might not be the best column name

2017-04-12 Thread Tomas Vondra
On 04/12/2017 03:36 PM, David Rowley wrote: "stakind" seems like a better name. I'd have personally gone with "statype" but pg_statistic already thinks stakind is better. +1 to stakind -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Developm

Re: [HACKERS] pg_stats_ext view does not seem all that useful

2017-04-10 Thread Tomas Vondra
t casts to text), because pg_statistic_ext would simply return the whole bytea value. But since then the view kinda lost the purpose and no one really noticed. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Ser

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-10 Thread Tomas Vondra
g a new GUC parameter, you're adding a constant which is then used as a max value for max for the two existing parallel GUCs. I think this is fine. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & S

Re: [HACKERS] Performance issue with postgres9.6

2017-04-07 Thread Tomas Vondra
that one backend using 'perf'. I assume you're using the same hardware / machine for the tests? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread Tomas Vondra
On 04/06/2017 11:45 PM, David Steele wrote: On 4/6/17 5:05 PM, Tomas Vondra wrote: On 04/06/2017 08:33 PM, David Steele wrote: On 4/5/17 7:29 AM, Simon Riggs wrote: 2. It's not clear to me the advantage of being able to pick varying filesizes. I see great disadvantage in having too

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread Tomas Vondra
, even if there's no time to add new tests. Do we actually have any infrastructure for that? Or do you plan to add some new animals with different WAL segment sizes? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Train

  1   2   3   4   5   6   7   8   9   10   >