Re: [GENERAL] Bad planning data resulting in OOM killing of postgres

2017-02-16 Thread David Hinkle
th=10) (6 rows) I guess I will also try throwing in 'set enable_hashjoin = false;' and see if that gets these purges to go. On Thu, Feb 16, 2017 at 2:22 PM, David Hinkle wrote: > Yep, 420ish million records out of 540 million records have a titleid > of 1. There are about 880,000 other u

Re: [GENERAL] Bad planning data resulting in OOM killing of postgres

2017-02-16 Thread David Hinkle
;, 'urls'); select patch_ndistinct('log_raw', 'hostid', 'hosts'); ANALYZE log_raw; On Thu, Feb 16, 2017 at 10:54 AM, Tom Lane wrote: > David Hinkle writes: >> Tom, there are three columns in this table that exhibit the problem, >> h

Re: [GENERAL] Bad planning data resulting in OOM killing of postgres

2017-02-15 Thread David Hinkle
Maybe also useful to know: cipafilter=# select reltuples from pg_class where relname = 'log_raw'; reltuples - 5.40531e+08 (1 row) On Wed, Feb 15, 2017 at 7:55 PM, David Hinkle wrote: > Thanks for your help! > > Karsten: The system does fill up swap before

Re: [GENERAL] Bad planning data resulting in OOM killing of postgres

2017-02-15 Thread David Hinkle
ller going wild with >> 4.7/4.8/some 4.9 kernels. > > I guess what I'm trying to say is that it may actually not be > PostgreSQL's fault but rather the kernel invoking the OOM > killer way prematurely. > > Karsten > -- > GPG key ID E4071346 @ eu.pool.sks-ke

Re: [GENERAL] Bad planning data resulting in OOM killing of postgres

2017-02-13 Thread David Hinkle
0) (actual time=0.003..274920.970 rows=544670242 loops=1) Buffers: shared hit=918 read=7323762 Planning time: 0.158 ms Execution time: 727533.213 ms (17 rows) On Mon, Feb 13, 2017 at 3:57 PM, David Hinkle wrote: > I managed to get this version to finish: > >

Re: [GENERAL] Bad planning data resulting in OOM killing of postgres

2017-02-13 Thread David Hinkle
loops=1) Buffers: shared hit=598 read=7324082 dirtied=34 Planning time: 0.072 ms Execution time: 248807.285 ms (9 rows) On Mon, Feb 13, 2017 at 3:47 PM, David Hinkle wrote: > psql:postgres@cipafilter = EXPLAIN (ANALYZE, BUFFERS) select titleid > from titles WHERE NOT EXIST

Re: [GENERAL] Bad planning data resulting in OOM killing of postgres

2017-02-13 Thread David Hinkle
swap. On Mon, Feb 13, 2017 at 3:21 PM, Jeff Janes wrote: > On Mon, Feb 13, 2017 at 12:43 PM, David Hinkle > wrote: >> >> Thanks Jeff, >> >> No triggers or foreign key constrains: >> >> psql:postgres@cipafilter = \d+ titles >>

Re: [GENERAL] Bad planning data resulting in OOM killing of postgres

2017-02-13 Thread David Hinkle
;::regclass) │ plain│ │ Indexes: "titles_pkey" PRIMARY KEY, btree (titleid) "titles_md5_title_idx" btree (md5(title::text)) Do you see anything in there that would be problematic? On Mon, Feb 13, 2017 at 2:41 PM, Jeff Janes wrote: > On Mon, Feb 13, 2017 at 11

Re: [GENERAL] Bad planning data resulting in OOM killing of postgres

2017-02-13 Thread David Hinkle
rsion ─── PostgreSQL 9.4.10 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2, 64-bit (1 row) On Mon, Feb 13, 2017 at 1:26 PM, Jeff Janes wrote: > On Mon, Feb 13, 2017 at 9:40 AM, David Hinkle wrote: >> >> I'm h

[GENERAL] Bad planning data resulting in OOM killing of postgres

2017-02-13 Thread David Hinkle
g wildly off. ALTER TABLE set n_distinct doesn't seem to be used by the planner as it doesn't change any of the plans I've generated or seem to be taken into account in the row estimates. I'm out of ideas. Anybody have any ideas? -- David Hinkle Senior Software Developer P