Encoding switching from UTF-8 to Latin1 before being sent
[Using Mutt-1.5.21 with FreeBSD and Arch Linux] Hi there, I noticed a nasty behaviour I do not understand. It happens if the e-mail contains latin characters with acute, like 'é', but no unicode character not covered by latin1. The text editor -- Emacs, but I tested with others too -- will correctly set it to UTF-8. file /tmp/mutt-... will confirm this. When I close the editor, thus swithing back to Mutt, it sees the content as iso8859-1. This is on the "Mutt Compose" screen, right before actually sending the mail. Of course I can convert it at this very same point with the 'edit-type' function. But I do not want to do it manually. It would be nice if I could send all my mails in UTF-8 automatically. If I add a unicode character not covered by Latin1 (e.g. '€'), then the e-mail is correctly marked as being utf-8. All my locale variables are set to "en_US.UTF-8". And :set &charset ?charset returns 'utf-8'. So I guess it comes from Mutt that fails at guessing the proper character encoding. I do not know how it works internally, but I do not have so much time right now to crawl in the source code. To reproduce the issue: write a e-mail with the sole letter 'é' as content. Note that the e-mail is not screwed up. It gets properly converted from utf-8 to latin1, 'é' goes from a9c3 (utf-8) to e9 (latin1). Any clue? Cheers.
Re: Encoding switching from UTF-8 to Latin1 before being sent
On 12-09-26 20:59:51, Christian Ebert wrote: > * Ambrevar on Wednesday, September 26, 2012 at 21:43:18 +0200 > > [Using Mutt-1.5.21 with FreeBSD and Arch Linux] > > > > Hi there, > > > > I noticed a nasty behaviour I do not understand. It happens if the e-mail > > contains latin characters with acute, like 'é', but no unicode character not > > covered by latin1. The text editor -- Emacs, but I tested with others too -- > > will correctly set it to UTF-8. > > > > file /tmp/mutt-... > > > > will confirm this. > > > > When I close the editor, thus swithing back to Mutt, it sees the content as > > iso8859-1. This is on the "Mutt Compose" screen, right before actually > > sending > > the mail. Of course I can convert it at this very same point with the > > 'edit-type' function. But I do not want to do it manually. It would be nice > > if I > > could send all my mails in UTF-8 automatically. > > > > If I add a unicode character not covered by Latin1 (e.g. '€'), then the > > e-mail > > is correctly marked as being utf-8. > > > > All my locale variables are set to "en_US.UTF-8". And > > > > :set &charset ?charset > > > > returns 'utf-8'. > > > > So I guess it comes from Mutt that fails at guessing the proper character > > encoding. I do not know how it works internally, but I do not have so much > > time > > right now to crawl in the source code. > > > > To reproduce the issue: write a e-mail with the sole letter 'é' as content. > > > > Note that the e-mail is not screwed up. It gets properly converted from > > utf-8 to > > latin1, 'é' goes from a9c3 (utf-8) to e9 (latin1). > > > > Any clue? > > man 5 muttrc: > > send_charset > Type: string > Default: "us-ascii:iso-8859-1:utf-8" > > A colon-delimited list of character sets for outgoing messages. Mutt > will > use the first char- acter set into which the text can be converted > exactly. If your $charset is not "iso-8859-1" and recipients may > not > understand "UTF-8", it is advisable to include in the list an appropri- > ate widely used standard character set (such as "iso-8859-2", > "koi8-r" > or "iso-2022-jp") either instead of or after "iso-8859-1". > > In case the text cannot be converted into one of these exactly, mutt > uses > $charset as a fall- back. > > I have: > set send_charset="us-ascii:iso-8859-1:iso-8859-15:windows-1252:utf-8" > > And when I write €, this email will probably be iso-8859-15. > > It's all about saving old-fashionedly some bits over the wire. > > -- > theatre - books - texts - movies > Black Trash Productions at home: http://www.blacktrash.org > Black Trash Productions on Facebook: > http://www.facebook.com/blacktrashproductions Thank you so much for this quick answer! Well, it definitely works! I wasn't aware of the send_charset variable: I must have missed it in the not-so-short man page and in the wiki. The thing is, I was looking for 'encoding', not 'charset'. :/ Saving bits may be understandable, but in my humble opinion universality and simplicity is more important. UTF-8 saves us a lot of pain, that's why I'm a UTF-8 advocate -- and for a lot of other reasons too. Cheers.
Android MUA does not display mail correctly because of Content-Disposition in header
[Mutt 1.5.21 on FreeBSD and Arch Linux] Hi My fellows recently let me aware of the fact that all the e-mail I've sent them with Mutt could not be displayed correctly on their Android 2.* smartphone. The mails display an empty body, the actual body being displayed as an attachment (which Android may name 'Unknown.txt'). I've been investigating around for a while and I've come to the conclusion that it comes from the 'Content-Disposition' that Mutt includes in the header of every e-mail. This feature does not seem to be toggable by the end-user (even though you can change the value of the Content-Disposition, but this does not solve the issue). This header is standard but rarely used by MUA. I've patched my local Mutt to remove this line from the header and it works perfectly. I've reported the issue on the bugtracker. I just wanted to know if someone ever noticed a similar issue, since I'm quiet surprised I couldn't find any clue on the net. Cheers
Re: Android MUA does not display mail correctly because of Content-Disposition in header
On 12-09-27 17:53:43, Jim Graham wrote: > On Thu, Sep 27, 2012 at 12:07:51PM -0500, Luis Mochan wrote: > > On Wed, Sep 26, 2012 at 11:06:00PM +0200, Ambrevar wrote: > > I don't recall any problem reading with Android 2.1 mail sent by > > mutt. I use K-9 as my Android mail client in my phone. > > Best regards, > > > > My fellows recently let me aware of the fact that all the e-mail I've > > > sent them > > > with Mutt could not be displayed correctly on their Android 2.* > > > smartphone. The > > > mails display an empty body, the actual body being displayed as an > > > attachment > > > (which Android may name 'Unknown.txt'). > > First of all, you should try K-9 mail...or ANYTHING other than the stock > e-mail app (same goes for the browser...I use Dolphin (which was Dolphin > Browser HD for a little while, and now is back to just Dolphin), FWIW. > Back to e-mail, though, like the stock browser, the stock e-mail > basically isn't used by more than a relative few. Few, if any, people > that I'm aware of like either one. But that's all a matter of personal > preference, so I'll stop those comments here. > > Other than that, all I can say is I send mail to myself to/from my tablet > all the time (when it's too small to be worth using FTP or a SAMBA > mount). I also forward e-mail that would look better in K-9, or that > has links that I'd rather have on my tablet than on my Mac. I've never > had any problems. My current tablet runs ICS, but I also had one that > ran Froyo (an el-cheapo tablet Archos POS), and I also have a phone > running Froyo that I used for e-mail before I got my tablets, and still, > no problems (always using K-9 from day one with e-mail.). Thanks for pointing me out this application. Actually it's not for me, but for my fellows. I do not have an Android devices myself. I've suggested them using this application instead. But I must disagree: a *lot* of people are using the stock MUA. Simply because they don't care. That's why this issue shouldn't be neglected. -- Pierre Neidhardt O< ascii ribbon campaign - stop html mail - www.asciiribbon.org The early bird gets the coffee left over from the night before.
Re: Android MUA does not display mail correctly because of Content-Disposition in header
On 12-09-29 19:47:24, Sebastian Tramp wrote: > On Sat, Sep 29, 2012 at 11:46:55AM +0200, P. Mazart wrote: > > > > … [Content-Disposition-Header] … But I must disagree: a *lot* of people > > > are > > > using the stock MUA. Simply because they don't care. That's why this issue > > > shouldn't be neglected. > > > > I agree. But is this really a problem with mutt rather than android’s stock > > MUA? > > I have the same problem. Can you provide the patch somewhere? > > ST > > -- > WebID: http://sebastian.tramp.name 1. I made the patched version available for Arch Linux : https://aur.archlinux.org/packages.php?ID=63185 Tested on FreeBSD too. If you use any other distribution, it shouldn't be a problem since it uses Mutt devel 1.5.21 and only removes a small piece of code. Tell me if you're having any problem with it. Keep in mind that this patch is by no mean an "upgrade" of Mutt. Actually it's either downgrading it. It's a quick-and-dirty fix I made available for the time being, waiting for a real fix on either sides. Mutt's right fix should be an option in muttrc allowing to toggle this line. 2. Yes of course this should be reported to the Android dev team. I'll check this later. Still, this issue highlights a limitation from Mutt. In my opinion what users like about Mutt is the ability to have full control over the e-mail content, especially the header. That's clearly not the case here. Regards -- Pierre Neidhardt O< ascii ribbon campaign - stop html mail - www.asciiribbon.org volcano, n.: A mountain with hiccups.
Re: Android MUA does not display mail correctly because of Content-Disposition in header
On 12-09-29 19:47:24, Sebastian Tramp wrote: > On Sat, Sep 29, 2012 at 11:46:55AM +0200, P. Mazart wrote: > > > > … [Content-Disposition-Header] … But I must disagree: a *lot* of people > > > are > > > using the stock MUA. Simply because they don't care. That's why this issue > > > shouldn't be neglected. > > > > I agree. But is this really a problem with mutt rather than android’s stock > > MUA? > > I have the same problem. Can you provide the patch somewhere? > > ST > > -- > WebID: http://sebastian.tramp.name Hi there, For those using the patch I provided for compatibility with Android stock MUA, I just noticed a mistake of mine: attachments you send have no name! A quick look at the header and you will see that the "filename" variable is missing. This is because I short-circuited the whole process. Thanksfully I managed to fix this. I handled the issue differently: if a part of the e-mail has no file (e.g. e-mail body) then Content-Disposition line is not printed at all. Otherwise Content-Disposition is printed with filename="..." after it. So basically that's exactly Mutt's default behaviour for attachments. Only e-mail body will be affected, which is exactly what we want for Android compatibility. You can get my new (hopefully) fully-functional patch at the same location: https://aur.archlinux.org/packages.php?ID=63185 Cheers -- Pierre Neidhardt O< ascii ribbon campaign - stop html mail - www.asciiribbon.org As you will see, I told them, in no uncertain terms, to see Figure one. -- Dave "First Strike" Pare
