#3907: mutt-1.7.0: Mutt-hcache caches the display-charset-decoded header lines.
-------------------+--------------
Reporter: is@… | Owner:
Type: defect | Status: new
Priority: major |
-------------------+--------------
{{{
Package: mutt
Version: 1.7.0
Severity: normal
-- Please type your report below this line
Hi,
I'm not useing any charset lines in my personal or system muttrc,
but rely on LC_LANG for the display charset.
A. When changing between terminals with utf-8 encoding and terminals
with iso8859-1 encoding and back again, I noticed that a few greek
language subject lines where not displayed correctly at the end,
but consistently as ?????'s (with embedded iso-8859-1 characters being
displayed correctly - e.g. "25%"). This happened consistently whether
mutt was compiled to use slang or ncursesw. All the time, the
message itself was displayed correctly (as long as an utf-8
terminal was used) including the subject line.
I removed the hcache-directory/imaps:user@host.example:INBOX.hcache, and
restarted mutt, to find the subject line being displayed correctly
again, as similar had in the past.
A subject once parsed in an utf-8 terminal is stable stays that
way even if the status is touched (e.g. replied) in a charset not
sufficient; when displaying in utf-8 it's ok again.
B. More surprising to me is a test I did with an iso-8859-7 terminal.
I couldn't display a greek subject line that was encoded as
=?utf-8?B?... I'd expected that it would display all iso-8859-7
characters, that is the greek letters and the "25%" mentioned above,
bot only showed the latter.
This doesn't work when setting charset in .muttrc, either.
C. Related note, but obviously more difficult, and maybe not fixable
with in-terminal-editors at all: While using an iso8859-1 terminal,
replying to a message with Greek subject line results in a message
with literal ???? as the subject to be sent.
P. Proposals for fixing:
1. store the undecoded (raw) Subject lines in the cache, and decode them
when reading from the cache as mutt does when readin from the mailbox.
This would solve A and B.
No disk space overhead (other than the encodings'); slight cpu (but
not network) time overhead for the iconv() calls at least whenever
the summary window is moved (or at mailbox open time?)
2. store a seperate header cache per mailbox and display charset.
Overhead: N times disk space.
This is implementable as configuration directive, e.g.
set header_cache=/usr/home/is/.mutthc-$charset/
It should at least be recommended/mentioned (warning about the
disk space overhead) in muttrc(5).
3. tag the the header cache with the display charset, and remove
(thus rebuilding) it (possibly asking the user) when opening the
mailbox in a different charset terminal. This works like 2.,
doesn't have space overhead over a single header cache, but has
CPU (and network traffic) overhead when the character set is changed.
(I'd have a look myself, but as I'm not familiar with the code, I fear
t'd take a long while to find enough continuous time.)
-- System Information
System Version: NetBSD henrietta 7.1_RC1 NetBSD 7.1_RC1 (GENERIC) #5: Sun
Jan 15 14:03:08 CET 2017
is@henrietta:/var/itch/sources/7/obj.amd64/sys/arch/amd64/compile/GENERIC
amd64
-- Mutt Version Information
Mutt 1.7.0 (2016-08-17)
Copyright (C) 1996-2016 Michael R. Elkins and others.
Mutt comes with ABSOLUTELY NO WARRANTY; for details type `mutt -vv'.
Mutt is free software, and you are welcome to redistribute it
under certain conditions; type `mutt -vv' for details.
System: NetBSD 7.1_RC1 (amd64)
slang: 20204
hcache backend: Berkeley DB 4.8.30: (April 9, 2010)
Compiler:
Using built-in specs.
COLLECT_GCC=/pkg_comp/obj/pkgsrc/mail/mutt/default/.gcc/bin/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/lto-wrapper
Target: x86_64--netbsd
Configured with:
/usr/7/src/tools/gcc/../../external/gpl3/gcc/dist/configure
--target=x86_64--netbsd --enable-long-long --enable-threads --with-
bugurl=http://www.NetBSD.org/Misc/send-pr.html --with-pkgversion='NetBSD
nb2 20150115' --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-
threads --with-tune=nocona --enable-libstdcxx-time=rt --enable-lto --with-
mpc-
lib=/var/obj/mknative/amd64-x86_64/usr/7/src/external/lgpl3/mpc/lib/libmpc
--with-mpfr-
lib=/var/obj/mknative/amd64-x86_64/usr/7/src/external/lgpl3/mpfr/lib/libmpfr
--with-gmp-
lib=/var/obj/mknative/amd64-x86_64/usr/7/src/external/lgpl3/gmp/lib/libgmp
--with-mpc-include=/usr/7/src/external/lgpl3/mpc/dist/src --with-mpfr-
include=/usr/7/src/external/lgpl3/mpfr/dist/src --with-gmp-
include=/usr/7/src/external/lgpl3/gmp/lib/libgmp/arch/x86_64 --enable-tls
--disable-multilib --disable-symvers --disable-libstdcxx-pch
--build=x86_64-unknown-netbsd6.0. --host=x86_64--netbsd --with-
sysroot=/var/obj/mknative/amd64-x86_64/usr/7/src/destdir.amd64
Thread model: posix
gcc version 4.8.5 (nb2 20150115)
Configure options: '--sysconfdir=/usr/pkg/etc' '--with-
docdir=/usr/pkg/share/doc/mutt' '--without-included-gettext' '--enable-
external-dotlock' '--enable-pop' '--enable-imap' '--with-slang=/usr/pkg'
'--with-sasl=/usr/pkg' '--with-ssl=/usr' '--enable-smime' '--enable-
hcache' '--without-gdbm' '--enable-smtp' '--with-idn=no'
'ac_cv_path_GNUPG=/usr/pkg/bin/gpg' '--enable-gpgme' '--with-gpgme-
prefix=/usr/pkg' '--prefix=/usr/pkg' '--build=x86_64--netbsd' '--
host=x86_64--netbsd' '--mandir=/usr/pkg/man' 'build_alias=x86_64--netbsd'
'host_alias=x86_64--netbsd' 'CC=gcc' 'CFLAGS=-O2 -I/usr/pkg/include
-I/usr/include -I/usr/pkg/include/db4' 'LDFLAGS=-s -L/usr/pkg/lib
-Wl,-R/usr/pkg/lib -L/usr/lib -Wl,-R/usr/lib' 'LIBS='
'CPPFLAGS=-I/usr/pkg/include -I/usr/include -I/usr/pkg/include/db4'
Compilation CFLAGS: -Wall -pedantic -Wno-long-long -O2 -I/usr/pkg/include
-I/usr/include -I/usr/pkg/include/db4
Compile options:
-DOMAIN
-DEBUG
-HOMESPOOL -USE_SETGID +USE_DOTLOCK +DL_STANDALONE +USE_FCNTL
-USE_FLOCK
+USE_POP +USE_IMAP +USE_SMTP
+USE_SSL_OPENSSL -USE_SSL_GNUTLS +USE_SASL -USE_GSS +HAVE_GETADDRINFO
+HAVE_REGCOMP -USE_GNU_REGEX
+HAVE_COLOR -HAVE_START_COLOR -HAVE_TYPEAHEAD -HAVE_BKGDSET
-HAVE_CURS_SET -HAVE_META -HAVE_RESIZETERM
+CRYPT_BACKEND_CLASSIC_PGP +CRYPT_BACKEND_CLASSIC_SMIME
+CRYPT_BACKEND_GPGME
-EXACT_ADDRESS -SUN_ATTACHMENT
+ENABLE_NLS -LOCALES_HACK +HAVE_WC_FUNCS +HAVE_LANGINFO_CODESET
+HAVE_LANGINFO_YESEXPR
+HAVE_ICONV -ICONV_NONTRANS -HAVE_LIBIDN +HAVE_GETSID +USE_HCACHE
-USE_SIDEBAR
-ISPELL
SENDMAIL="/usr/sbin/sendmail"
MAILPATH="/var/mail"
PKGDATADIR="/usr/pkg/share/mutt"
SYSCONFDIR="/usr/pkg/etc"
EXECSHELL="/bin/sh"
-MIXMASTER
To contact the developers, please mail to <mutt-dev@mutt.org>.
To report a bug, please visit http://bugs.mutt.org/.
}}}
--
Ticket URL: <https://dev.mutt.org/trac/ticket/3907>
Mutt <http://www.mutt.org/>
The Mutt mail user agent
