Different encodings at index and pager views
Hi, I'm noticing different encoding behavior for headers displayed at the index view and those at the pager view for the same email. For example, at the index view I can see subjects like: "?Opin? sobre Bebidas Alcoh?licas y gan? una TV LCD o una Notebook!", while the right chars show instead of ? at the pager view. First, I want to discard a number of usual causes: * It's not a cache header issue. I deleted the cache after each related configuration change. * I'm accessing to gmail via imap but it's not a case of bad encoding by them. True, the headers I'm receiving are not 2047 encoded, they are just iso-8859-1 or utf-8 encoded, but in theory I'm forcing their charset by means of assumed_charset (more on this below). That said, I can't swear that they are not serving the subject with that awful qmarks themselves, but I don't think so. * I made some tests with my locale configured to en_US.ISO-8859-1 and then to en_US.UTF-8. I also tested disabling muttrc charset setting, and forcing it to my current locale, whatever it were. It did no difference at all. To make things weirder, some non 2047 encoded headers are shown correctly at both views. For example, I have a utf-8 email and a latin-1 email, both with their subject headers encoded in the respective charset (I verified this editing the raw emails with e). No matter what my locale is the utf-8 email subject is correctly displayed while the latin-1 one isn't. Also assumed_charset=iso-8859-1 doesn't fix the problem for the latin-1 message. This begins to feel like random behavior but there is another aspect that could be making the difference: the email that is looking bad is a multipart one, with no charset specified at the main Content-Type: multipart/alternative header; but the well behaved email is single part, with Content-Type: text/plain; charset=UTF-8. Maybe the index view is assuming the encoding given by Content-Type header for the subject and other header values, disregarding assumed_charset in case no encoding is given by Content-Type. What do you think? Ah, my mutt version is 1.5.19. Best regards -Carlos
Re: Different encodings at index and pager views
> This is probably a gmail bug: > http://dev.mutt.org/trac/ticket/2997 I knew about that bug, I've even experienced it, but I can't find the relation with the problem I'm describing now. The headers in question are correctly utf-8 or latin-1 encoded, there is no trace of rfc 2047 encoding, be it right o wrong done. If I edit the raw utf-8 email and my locale is also utf-8, the subject header looks as expected. If I edit the raw latin-1 email and my locale is also latin-1, the subject header looks as expected. I get the same results if I edit the raw mails from gmail and change the encoding of my browser. So my conclusion is that encoding is fine. Sure it's not rfc 2047 as it should, but it's perfectly legal utf-8 or latin-1 depending on case. And I remark that the effective encoding of both emails matches the one given by the Content-type header, in one case at the only part and in the other (the one that presents the problem) at one of its multiple parts (the text/plain one). Again, setting assumed_charset=iso-8859-1 and cleaning the header cache don't fix the issue. If it were to blame gmail, the only sensible explanation that comes to my mind is that their imap server has limitations to serve latin-1 encoded emails headers (only the headers, not the entire mail), and replacement with qmarks happens during some encoding conversion that takes place at their place. Sounds unlikely, but as I've not sniffed the actual contents of imap conversation I can't deny it for sure. Best regards -Carlos
Re: Different encodings at index and pager views
Hi Kyle, > You're not *forcing* the charset, you're *guessing* the charset. > There's a semantic difference (of course, we computer folk love to I didn't mean I was forcing the charset to be one that it wasn't, of course, but telling mutt to interpret the binary data as having one or another encoding. > > * I made some tests with my locale configured to en_US.ISO-8859-1 > > and then to en_US.UTF-8. > > I presume your terminal is capable of understanding UTF-8 characters? Yes it is. I'm using urxvt. I did the same thing viewing the raw mail in my browser and changing its enconding, anyway. > Maybe not, but generally speaking, it's a very VERY bad idea to set > the $charset manually (unless you really know what you're doing - I Sure. I did it with the only purpose of testing different scenarios. > Anyway - step 1 is to find out *exactly* what the conversation between > mutt and gmail looks like. I'm on it. Thanks for your reply, I'll post the results later. Regards -Carlos
Re: Different encodings at index and pager views
boundary="===3754908945113369111==" From: "Livra Encuestas" Subject: ?Opin? sobre Bebidas Alcoh?licas y gan? una TV LCD o una Notebook! Date: Thu, 26 Feb 2009 12:37:05 -0600 To: carlosjosep...@gmail.com Message-Id: <20090226183705.6152e1af0...@tweek.livra.com> Yeah, you're right, they're poorly reencoding at some point. Why would they do that? Does IMAP impose any restriction on the encoding? OTOH, they serve the entire mail in its original encoding and so they show it at their site. Thanks for the debugging tips! Best regards -Carlos >
Show my replies in thread
Hi all, I don't think this is supported but I'm going to ask it anyway. I would like to see my replies intercalated in threads. I use gmail imap server, and sent emails are correctly stored under "[Gmail]/Sent Mail" (this is the default gmail imap server behavior, no extra config is required, except that I set copy=no to avoid duplicated copies). "[Gmail]/Sent Mail" is a huge non-archivable (in the gmail sense) folder, which makes things even worse when I pretend to follow the natural flow of a thread without distractions. At least, is it a way to quickly open a mail referenced by the in-reply-to header? Best regards -Carlos
Re: Show my replies in thread
I configured Kyle's hook, which does a nice job, and will periodically enter gmail webmail and empty their Sent Mail label in a manual fashion. I can't see how to configure a filter at their site to drop all sent mail. As soon as it arrives the email seems not to be even labeled, so a condition like label:Sent Mail won't match. And afaik you can't schedule a filter to run periodically. Alternatively imapfilter or a combination of offlineimap+procmail could automate the job of removing unneeded sent copies, but I don't thinks it's worth the effort. Any enlightened ideas? Best regards -Carlos On Fri, Feb 27, 2009 at 05:01:26PM -0500, Patrick Shanahan wrote: > * Carlos Pita [01-01-70 12:34]: > > > > I don't think this is supported but I'm going to ask it anyway. I would > > like to see my replies intercalated in threads. I use gmail imap server, > > and sent emails are correctly stored under "[Gmail]/Sent Mail" (this is > > the default gmail imap server behavior, no extra config is required, > > except that I set copy=no to avoid duplicated copies). "[Gmail]/Sent > > Mail" is a huge non-archivable (in the gmail sense) folder, which makes > > things even worse when I pretend to follow the natural flow of a thread > > without distractions. > > > > Then you will need to change the way you *send* mail. I see on my > system exactly what you are wanting, but I *only* use gmail to > *receive* mail, not send. I use my provider to send and include a > gmail "from" address. This way you get gmail's storage and spam > filtering and see your own posts. > > -- > Patrick Shanahan Plainfield, Indiana, USAHOG # US1244711 > http://wahoo.no-ip.org Photo Album: http://wahoo.no-ip.org/gallery2 > Registered Linux User #207535@ http://counter.li.org >
Re: Show my replies in thread
Hi Patrick, > Then you have not considered/read my post which you fully quoted, Yes, I had read it before I posted my reply. Thank you very much for the suggestion. Currently I don't have a "provider's smtp". And I find my problem a minor annoyance, not worth switching to another smtp server. That I don't find your solution appealing doesn't mean that I consider it stupid or that I haven't read it at all. I'm sincerely surprised by your reaction. This kind of discussion doesn't mean to be personal. > Tell me if I am wasting my time! This decision is entirely up to you. Best regards -Carlos
Re: Show my replies in thread
> > folder-hook . 'set record=^' > > One thing that is interesting about the Gmail IMAP server is > that a message can exist in multiple folders at the same time. > This is actually a pretty powerful concept. In reality there's > a database with a single copy of each message, and IMAP That's true in general but I've confirmed that the hook above effectively creates two copies. If I enter gmail web interface, trash the copy at Sent Mail, then go to Thrash and permanently remove this trashed copy, the one at the original label where I've replied still exists by its own. > "[Gmail]/Sent Mail", so if you want to keep things consistent, > you should configure that as your "record" folder in mutt. No. If you want to play fair with gmail you have to set copy = no, and then the record folder becomes irrelevant. That is because they automatically save outgoing mails into [Gmail]/Sent Mail, no matter if they're send from the web interface or via their smtp server. They even give this guideline at their site: https://mail.google.com/support/bin/answer.py?answer=78892&topic=12815 Best regards -Carlos
Re: Show my replies in thread
> ps: your sig should begin with , the characters > not the words. Thanks for the great tip !!! Carlos
