On Fri, Mar 29, 2024 at 03:03:54PM -0700, Bob Crochelt wrote: > Thank you Derek! > Bob Crochelt
No problem, glad I could help. I had to figure this stuff all out the hard way 20 years ago, when I moved to Korea and suddently needed to be able to type in both English and Korean... I've been answering it on this list ever since... seriously, see this post from 2006: https://marc.info/?l=mutt-users&m=114434601817142&w=2 Things were A LOT worse then--Unicode hadn't really been adopted very much in 2004. Some of the finer details of how (or rather where) to set those settings keep changing (in part because young developers can't leave well enough alone when a solution has been working for literally decades), and most people don't need to futz with fonts anymore since Unicode is the default everywhere now, and the modern font rendering libraries handle all this automatically--but the basics of the problem and its solution are still the same. In another such post I commented that this topic comes up often enough that it really ought to be a FAQ. Some day I'll take the time to collect the details in a bit better format and provide one... -- Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. > On Fri, Mar 29, 2024, at 13:47, Derek Martin wrote: > > On Sat, Mar 23, 2024 at 07:41:45PM +0800, Sadeep Madurange wrote: > >> Initially, LANG was unset and LC_CTYPE="C". The character encoding was > >> US-ASCII. I changed these variables (i.e., LANG, LC_CTYPE and locale > >> settings) to en_US.UTF-8. Then the ? changed to ?. So, looks like you > >> are on to something. I will check this with OpenBSD community as well. > >> > >> In Xdefaults, I have set XTerm*utf-8 setting to true as well. > > > > Your problem is that these settings are not consistent (and you still > > have this problem, because the "solution" proposed by Sirius is > > incorrect--even if it appears to have solved your issue). By having > > LANG unset, you've told your shell (and therefore everything started > > by it) to use ASCII, but you've explicitly told xterm to use Unicode. > > That's wrong. > > > > The TL;DR of this is: > > > > 1. You should NEVER need to set Mutt's charset explicitly. [*] > > 2. Your shell, Mutt, and X should all inherit what they need from your > > LANG environment variable, assuming it is set properly for your > > system and environment (it definitely isn't in your case). > > 3. Setting Mutt's charset may appear to "work" but it's not the > > correct solution, because your shell and terminal settings are > > still inconsistent. You'll have trouble with other things later if > > you don't fix this. > > ________________________________________________________________ > > [*] Except in extremely rare and completely esoteric cases that apply > > only to experts... and by now should really apply to no one. > > > > > > The unfortunately lengthy details: > > ---------------------------------- > > > > Displaying characters properly is actually tricky business on modern > > computers, because of the legacy methods by which we tried to > > accommodate different languages, and the (relatively) recent advent of > > Unicode to unify that mess. All of the following must be set > > consistently: Your shell, your terminal program (or your operating > > system's console), your font, all of your application programs, and > > when appropriate, the X window system. If any of these are not > > consistently set, you can, and eventually WILL, have trouble. Most > > modern systems have the concept of a default locale, which is > > typically set for you at install time, and which every process you > > start inherits, unless you configure your user environment > > differently. > > > > Fortunately, there is a very simple mechanism by which this happens, > > which is the LANG environment variable. There are additional > > ancillary environment variables which start with "LC_*" but you > > usually should not have to set any of these, because they inherit > > their value from LANG if they are not explicitly set. When you run > > the locale command, values enclosed in quotes are inherited from LANG, > > and values NOT enclosed in quotes have been set explicitly: > > > > $ locale > > LANG=en_US.UTF-8 > > LANGUAGE= > > LC_CTYPE="en_US.UTF-8" > > LC_NUMERIC="en_US.UTF-8" > > LC_TIME="en_US.UTF-8" > > LC_COLLATE=C > > LC_MONETARY="en_US.UTF-8" > > LC_MESSAGES="en_US.UTF-8" > > LC_PAPER="en_US.UTF-8" > > LC_NAME="en_US.UTF-8" > > LC_ADDRESS="en_US.UTF-8" > > LC_TELEPHONE="en_US.UTF-8" > > LC_MEASUREMENT="en_US.UTF-8" > > LC_IDENTIFICATION="en_US.UTF-8" > > LC_ALL= > > > > Here you can see that I explicitly set LC_COLLATE=C. The rest are > > inherited from LANG. Typically most users will want to leave all of > > the LC_* variables unset, and inherit from LANG. > > > > I haven't tried a *BSD in a really long while, but if it doesn't ask > > you for your default locale during install, or if you made a mistake > > setting it up, then you should add the settings manually to your login > > shell environment. If you're using UTF-8 (which you should be--by now > > every modern OS uses it by default), the value of LANG should reflect > > that. Pretty much no one should be using ASCII anymore (i.e. LANG > > should NEVER be unset). The most portable way to do that would be to > > include the following in BOTH .profile and .kshrc (or whatever file > > you've set ENV to): > > > > LANG=en_US.UTF-8 > > export LANG > > > > [See the Invocation section of the ksh man page for exact details of > > which files you should put this in, but in general it's the ones I > > said.] > > > > Of course if you are not using English, change en_US to whatever your > > default language is, but you'll want to retain the ".UTF-8" portion. > > > > That *should* be sufficient to handle everything... however, there may > > be additional places you'll need to add it for your X applications, > > depending on exactly what OpenBSD does to initialize users' X > > sessions. In general, the X startup stuff is supposed to make sure > > that it sources the user's environment so that you don't need to > > figure out which of the 17 different files you actually need to put > > this stuff in... but over the years most vendors have bastardized how > > X sessions start up, so you may have to look up or trace out how your > > system does it to make sure everything works correctly. So you might > > have to also add it to .xinitrc or .xsession or similar. Or, if it > > does not already, you could simply have your X init thingy source your > > .profile or .kshrc or whatever--and probably should. > > > > As for this: > > > > On Sun, Mar 24, 2024 at 08:11:29AM +0800, Sadeep Madurange wrote: > >> On 2024-03-23 12:52:40, Sirius Rayner-Karlsson via Mutt-users wrote: > >> > It may be that you just need to pop in the "set charset="utf-8"" in > >> > your mutt config and you are good to go. > > > > As I said, you should NEVER need to do this... if your environment is > > set up properly, Mutt will CORRECTLY inherit its charset and language > > settings from that. The only time you should ever need to mess with > > Mutt's idea of your charset is if you have some very esoteric edge > > case use that requires things to be different from your default > > language. Since the proliferation of Unicode, this is now EXTREMELY > > rare, and if you don't know what you're doing, you almost certainly > > shouldn't be doing this. > > > >> Thanks for sharing your config. You're right. I needed to add "set > >> charset=utf-8" to muttrc and set LC_CTYPE. The problem was I had set the > >> latter in my .kshrc when I needed to set it in .xsession. It's all good > >> now. > > > > Your settings may "work" but they are not correct. You might not have > > trouble with Mutt displaying ??? where it shouldn't anymore, but if > > you don't take care to set up your locale correctly, you will > > eventually continue to experience related issues elsewhere, inside and > > outside Mutt. > > > > You should really remove that charset setting in Mutt, to confirm that > > your other settings are now correct. You don't need it, and you > > shouldn't use it. It's just more likely to cause problems later. > > > > -- > > Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 > > -=-=-=-=- > > This message is posted from an invalid address. Replying to it will result > > in > > undeliverable mail due to spam prevention. Sorry for the inconvenience. > > > > > > Attachments: > > * signature.asc
signature.asc
Description: PGP signature
