On 24/11/2014 15:50, Alan Hicks wrote:
Although I don't have an answer, I can confirm the slow query happens
on 3.2.1, though for me at least unrepeatable, subsequent runs of the
query are returned sub second.
The query planner shows nothing unusual, also table stats for each of
the query tables shows few table scans. Fortunately it hasn't
affected me enough to investigate further.
I haven't tried header_cache_readonly which may help.
Currently running PostgreSQL 9.2, previously saw more on MySQL 5.1
though no stats to compare fairly.
Regards,
Alan
This explanation may help to clarify why it might be useful to use
header_cache_readonly.
If delivery of a new message adds to the headername table and due to any
of complexity, size or server load takes longer to commit, the index may
be locked on the headername table until the transaction is complete.
Both MySQL and PostgreSQL offer excellent Multiversion Concurrency
Control (MVCC), nevertheless the query plan shows these are evaluated as
Hash indexes whose locks are held for the duration of the nested loop.
Lock contention on the index may be the cause of the slow query.
Using header_cache_readonly would not add any new records to the
headername table and so should not need any locks.
It might be useful to get feedback to see if anyone using
header_cache_readonly still sees those slow queries.
SELECT m.message_idnr, n.headername, ... AS seq
FROM dbmail_header h
LEFT JOIN dbmail_messages m ON h.physmessage_id=m.physmessage_id
LEFT JOIN dbmail_headername n ON h.headername_id=n.id
LEFT JOIN dbmail_headervalue v ON h.headervalue_id=v.id
WHERE m.mailbox_idnr = 170
AND m.message_idnr BETWEEN 5477613 AND 5479501
AND n.headername IN ('from','to',...)
ORDER BY message_idnr, seq
Sort (cost=6387.09..6387.62 rows=212 width=163)
Sort Key: m.message_idnr, (CASE WHEN ((n.headername)::text =
'from'::text) THEN 0 WHEN ((n.headername)::text = 'to'::text) THEN 1
WHEN ((n.headername)::text = 'cc'::text) THEN 2 WHEN
((n.headername)::text = 'bcc'::text) THEN 3 WHEN ((n.headername)::text
(...)
-> Nested Loop Left Join (cost=115.79..6378.90 rows=212 width=163)
-> Hash Join (cost=115.79..4857.12 rows=212 width=32)
Hash Cond: (h.headername_id = n.id)
-> Nested Loop (cost=54.82..4734.68 rows=15828 width=24)
-> Bitmap Heap Scan on dbmail_messages m
(cost=54.82..1520.85 rows=500 width=16)
Recheck Cond: (mailbox_idnr = 1851)
Filter: ((message_idnr >= 2779446) AND
(message_idnr <= 2834893))
-> Bitmap Index Scan on dbmail_messages_7
(cost=0.00..54.69 rows=1923 width=0)
Index Cond: (mailbox_idnr = 1851)
-> Index Only Scan using dbmail_header_pkey on
dbmail_header h (cost=0.00..6.04 rows=39 width=24)
Index Cond: (physmessage_id = m.physmessage_id)
-> Hash (cost=60.75..60.75 rows=18 width=24)
-> Seq Scan on dbmail_headername n
(cost=0.00..60.75 rows=18 width=24)
Filter: ((headername)::text = ANY
('{from,to,cc,bcc,subject,date,message-id,priority,x-priority,references,newsgroups,in-reply-to,content-type,reply-to,x-spam-flag,sender,reply-to,x-cron-env}'::text[]))
-> Index Scan using dbmail_headervalue_pkey on
dbmail_headervalue v (cost=0.00..7.12 rows=1 width=147)
Index Cond: (h.headervalue_id = id)
Further reading on Locking and Indexes
http://dev.mysql.com/doc/refman/5.6/en/innodb-locks-set.html
http://www.postgresql.org/docs/9.2/interactive/locking-indexes.html
Regards,
Alan
On 24/11/2014 10:51, Daniel Schütze wrote:
I’ve noticed in my slow query log there is a particular type of query
which is regularly showing up and I was wondering what was triggering it.
The most recent example is this (but it’s being going on since I
turned the slow query log on in July:
SELECT m.message_idnr, n.headername, v.headervalue, CASE WHEN
n.headername='from' THEN 0 WHEN n.headername='to' THEN 1 WHEN
n.headername='cc' THEN 2 WHEN n.headername='bcc' THEN 3 WHEN
n.headername='subject' THEN 4 WHEN n.headername='date' THEN 5 WHEN
n.headername='message-id' THEN 6 WHEN n.headername='priority' THEN 7
WHEN n.headername='x-priority' THEN 8 WHEN n.headername='references'
THEN 9 WHEN n.headername='newsgroups' THEN 10 WHEN
n.headername='in-reply-to' THEN 11 WHEN n.headername='content-type'
THEN 12 WHEN n.headername='reply-to' THEN 13 END AS seq FROM
dbmail_header h LEFT JOIN dbmail_messages m ON
h.physmessage_id=m.physmessage_id LEFT JOIN dbmail_headername n ON
h.headername_id=n.id LEFT JOIN dbmail_headervalue v ON
h.headervalue_id=v.id WHERE m.mailbox_idnr = 170 AND m.message_idnr
BETWEEN 5477613 AND 5479501 AND n.headername IN
('from','to','cc','bcc','subject','date','message-id','priority','x-priority','references','newsgroups','in-reply-to','content-type','reply-to')
ORDER BY message_idnr, seq;
Which slow reports as taking some 33 seconds
# Time: 141124 10:32:05
# User@Host: dbmail[dbmail] @ localhost []
# Query_time: 33.363653 Lock_time: 0.000014 Rows_sent: 0
Rows_examined: 20214893
I’m still using dbmail 3.1.15 so apologies if this is obsolete by the
current 3.2.1 release; but I’m wondering if this query/speed is
normal or a sign something is a miss.
Regards
Daniel Schütze
------------------------
CWA International
Balmoral House
9 John Street
London
WC1N 2ES
(t) + 44 (0)20 7242 8444
(e) d...@cwa.uk.com
(w) http://www.cwa.uk.com/
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
--
Persistent Objects Ltd
128 Lilleshall Road
Morden SM4 6DR
The Home of Lasting Solutions
Mobile: 079 3030 5004
Tel: 020 8544 5292
Web: p-o.co.uk
Skype: alan-hicks-london
Personal bloghttps://plus.google.com/+AlanHicksLondon
Company bloghttps://plus.google.com/+PoCoUkLondon/posts
LinkedInhttps://uk.linkedin.com/in/alanhickslondon/
GitHubhttps://github.com/alan-hicks
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
--
Persistent Objects Ltd
128 Lilleshall Road
Morden SM4 6DR
The Home of Lasting Solutions
Mobile: 079 3030 5004
Tel: 020 8544 5292
Web: p-o.co.uk
Skype: alan-hicks-london
Personal blog https://plus.google.com/+AlanHicksLondon
Company blog https://plus.google.com/+PoCoUkLondon/posts
LinkedIn https://uk.linkedin.com/in/alanhickslondon/
GitHub https://github.com/alan-hicks
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
