* Colin Watson <cjwat...@debian.org>, 2009-06-22, 11:19:
There's no actual *problem* with adding a UTF-8 encoding declaration,
but it will not help with this bug, and once this bug is fixed it will
not be necessary for an encoding declaration to be present in order to
take advantage of the fix. manconv already has pretty reliable heuristic
detection of UTF-8; in fact, this is a large part of its purpose in
life.
In fact, adding UTF-8 encoding declaration could (minimally) help:
$ echo "[©] [≤]" > tmp1
$ man -l tmp1
[©] [â¤]
$ (echo "'\\\" -*- coding: UTF-8 -*-"; cat tmp1) > tmp2
$ man -l tmp2
[©] []
In the latter case, at least *some* characters are displayed properly.
This makes no difference for me (man-db 2.5.5-2, groff-base
1.18.1.1-22).
I have exactly the same versions installed.
Debugging output shows identical processing apart from the
insertion of tbl. I'm adamant that this *should* not cause a
difference,
I guess it has something to do with my locale settings:
$ locale
LANG=
LC_CTYPE=pl_PL.UTF-8
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=
$ man -l tmp1
[©] [â¤]
$ LC_ALL=pl_PL.utf8 man -l tmp1
[] []
$ LC_ALL=en_US.utf8 man -l tmp1
[©] []
That's weird. I believed that locale settings should make no difference,
as long as LC_CTYPE is *.UTF-8.
so I'm happy to figure out why this is different for you if you can show
me 'man -d' output.
See the attachments.
--
Jakub Wilk
ruid=500, euid=500
++priv_drop_count = 1
From the config file /etc/manpath.config:
Mandatory mandir `/usr/man'.
Mandatory mandir `/usr/share/man'.
Mandatory mandir `/usr/local/share/man'.
Path `/bin' mapped to mandir `/usr/share/man'.
Path `/usr/bin' mapped to mandir `/usr/share/man'.
Path `/sbin' mapped to mandir `/usr/share/man'.
Path `/usr/sbin' mapped to mandir `/usr/share/man'.
Path `/usr/local/bin' mapped to mandir `/usr/local/man'.
Path `/usr/local/bin' mapped to mandir `/usr/local/share/man'.
Path `/usr/local/sbin' mapped to mandir `/usr/local/man'.
Path `/usr/local/sbin' mapped to mandir `/usr/local/share/man'.
Path `/usr/games' mapped to mandir `/usr/share/man'.
Path `/opt/bin' mapped to mandir `/opt/man'.
Path `/opt/sbin' mapped to mandir `/opt/man'.
Global mandir `/usr/man', catdir `/var/cache/man/fsstnd'.
Global mandir `/usr/share/man', catdir `/var/cache/man'.
Global mandir `/usr/local/man', catdir `/var/cache/man/oldlocal'.
Global mandir `/usr/local/share/man', catdir `/var/cache/man/local'.
Global mandir `/opt/man', catdir `/var/cache/man/opt'.
Added section `1'.
Added section `n'.
Added section `l'.
Added section `8'.
Added section `3'.
Added section `2'.
Added section `3posix'.
Added section `3pm'.
Added section `3perl'.
Added section `5'.
Added section `4'.
Added section `9'.
Added section `6'.
Added section `7'.
`/usr/man' `' `1'
`/usr/share/man' `' `1'
`/usr/local/share/man' `' `1'
`/bin' `/usr/share/man' `0'
`/usr/bin' `/usr/share/man' `0'
`/sbin' `/usr/share/man' `0'
`/usr/sbin' `/usr/share/man' `0'
`/usr/local/bin' `/usr/local/man' `0'
`/usr/local/bin' `/usr/local/share/man' `0'
`/usr/local/sbin' `/usr/local/man' `0'
`/usr/local/sbin' `/usr/local/share/man' `0'
`/usr/games' `/usr/share/man' `0'
`/opt/bin' `/opt/man' `0'
`/opt/sbin' `/opt/man' `0'
`/usr/man' `/var/cache/man/fsstnd' `-1'
`/usr/share/man' `/var/cache/man' `-1'
`/usr/local/man' `/var/cache/man/oldlocal' `-1'
`/usr/local/share/man' `/var/cache/man/local' `-1'
`/opt/man' `/var/cache/man/opt' `-1'
`1' `' `-5'
`n' `' `-5'
`l' `' `-5'
`8' `' `-5'
`3' `' `-5'
`2' `' `-5'
`3posix' `' `-5'
`3pm' `' `-5'
`3perl' `' `-5'
`5' `' `-5'
`4' `' `-5'
`9' `' `-5'
`6' `' `-5'
`7' `' `-5'
is a tty
real user = 500; effective user = 500
using pager -s as pager
++priv_drop_count = 2
chdir /home/jwilk/trunk/bugs/man-db
Starting pipeline: [input: {0, tmp1}, output: {-1, NULL}]
pre-processors `t' from default
page_encoding = ISO-8859-1
source_encoding = ISO-8859-1
cat_charset = ANSI_X3.4-1968
locale_charset = UTF-8
roff_device (locale) = utf8
roff_encoding = ISO-8859-1
output_encoding = UTF-8
less_charset = utf-8
Terminal width 160
Terminal width 160 not within cat page range [80, 80]
Using 156-character lines
format: 1, save_cat: 0, found: 1
Setting LESS to -ix8RmPm Manual page tmp1 ?ltline %lt?L/%L.:byte %bB?s/%s..?e
(END):?pB %pB\%..$PM Manual page tmp1 ?ltline %lt?L/%L.:byte %bB?s/%s..?e
(END):?pB %pB\%..$
Setting MAN_PN to tmp1
++priv_drop_count = 3
Starting pipeline: /usr/bin/zsoelim | /usr/lib/man-db/manconv -f
UTF-8:ISO-8859-1 -t ISO-8859-1//IGNORE | tbl | nroff -mandoc -rLL=156n
-rLT=156n -Tutf8 [input: {-1, NULL}, output: {-1, NULL}]
Started "/usr/bin/zsoelim", pid 3421
Started "/usr/lib/man-db/manconv -f UTF-8:ISO-8859-1 -t ISO-8859-1//IGNORE",
pid 3423
Started "tbl", pid 3425
Started "nroff", pid 3426
Starting pipeline: pager -s [input: {-1, NULL}, output: {0, NULL}]
Started "pager", pid 3427
source pipeline 0 returned error or EOF
source pipeline 1 returned error or EOF
Waiting for pipeline: [input: {0, tmp1}, output: {-1, NULL}]
Waiting for pipeline: /usr/bin/zsoelim | /usr/lib/man-db/manconv -f
UTF-8:ISO-8859-1 -t ISO-8859-1//IGNORE | tbl | nroff -mandoc -rLL=156n
-rLT=156n -Tutf8 [input: {-1, NULL}, output: {-1, NULL}]
Active processes (4):
"/usr/bin/zsoelim" (3421) -> 0
"/usr/lib/man-db/manconv -f UTF-8:ISO-8859-1 -t ISO-8859-1//IGNORE" (3423) -> 0
"tbl" (3425) -> 0
"nroff" (3426) -> -1
Active processes (1):
"nroff" (3426) -> 0
Waiting for pipeline: pager -s [input: {-1, NULL}, output: {0, NULL}]
Active processes (1):
"pager" (3427) -> -1
Active processes (1):
"pager" (3427) -> 0
--priv_drop_count = 2
--priv_drop_count = 1
ruid=500, euid=500
++priv_drop_count = 1
From the config file /etc/manpath.config:
Mandatory mandir `/usr/man'.
Mandatory mandir `/usr/share/man'.
Mandatory mandir `/usr/local/share/man'.
Path `/bin' mapped to mandir `/usr/share/man'.
Path `/usr/bin' mapped to mandir `/usr/share/man'.
Path `/sbin' mapped to mandir `/usr/share/man'.
Path `/usr/sbin' mapped to mandir `/usr/share/man'.
Path `/usr/local/bin' mapped to mandir `/usr/local/man'.
Path `/usr/local/bin' mapped to mandir `/usr/local/share/man'.
Path `/usr/local/sbin' mapped to mandir `/usr/local/man'.
Path `/usr/local/sbin' mapped to mandir `/usr/local/share/man'.
Path `/usr/games' mapped to mandir `/usr/share/man'.
Path `/opt/bin' mapped to mandir `/opt/man'.
Path `/opt/sbin' mapped to mandir `/opt/man'.
Global mandir `/usr/man', catdir `/var/cache/man/fsstnd'.
Global mandir `/usr/share/man', catdir `/var/cache/man'.
Global mandir `/usr/local/man', catdir `/var/cache/man/oldlocal'.
Global mandir `/usr/local/share/man', catdir `/var/cache/man/local'.
Global mandir `/opt/man', catdir `/var/cache/man/opt'.
Added section `1'.
Added section `n'.
Added section `l'.
Added section `8'.
Added section `3'.
Added section `2'.
Added section `3posix'.
Added section `3pm'.
Added section `3perl'.
Added section `5'.
Added section `4'.
Added section `9'.
Added section `6'.
Added section `7'.
`/usr/man' `' `1'
`/usr/share/man' `' `1'
`/usr/local/share/man' `' `1'
`/bin' `/usr/share/man' `0'
`/usr/bin' `/usr/share/man' `0'
`/sbin' `/usr/share/man' `0'
`/usr/sbin' `/usr/share/man' `0'
`/usr/local/bin' `/usr/local/man' `0'
`/usr/local/bin' `/usr/local/share/man' `0'
`/usr/local/sbin' `/usr/local/man' `0'
`/usr/local/sbin' `/usr/local/share/man' `0'
`/usr/games' `/usr/share/man' `0'
`/opt/bin' `/opt/man' `0'
`/opt/sbin' `/opt/man' `0'
`/usr/man' `/var/cache/man/fsstnd' `-1'
`/usr/share/man' `/var/cache/man' `-1'
`/usr/local/man' `/var/cache/man/oldlocal' `-1'
`/usr/local/share/man' `/var/cache/man/local' `-1'
`/opt/man' `/var/cache/man/opt' `-1'
`1' `' `-5'
`n' `' `-5'
`l' `' `-5'
`8' `' `-5'
`3' `' `-5'
`2' `' `-5'
`3posix' `' `-5'
`3pm' `' `-5'
`3perl' `' `-5'
`5' `' `-5'
`4' `' `-5'
`9' `' `-5'
`6' `' `-5'
`7' `' `-5'
is a tty
real user = 500; effective user = 500
using pager -s as pager
++priv_drop_count = 2
chdir /home/jwilk/trunk/bugs/man-db
Starting pipeline: [input: {0, tmp2}, output: {-1, NULL}]
pre-processors `-*- coding: UTF-8 -*-' from file
preprocessor encoding: UTF-8
page_encoding = UTF-8
source_encoding = ISO-8859-1
cat_charset = ANSI_X3.4-1968
locale_charset = UTF-8
roff_device (locale) = utf8
roff_encoding = ISO-8859-1
output_encoding = UTF-8
less_charset = utf-8
Terminal width 160
Terminal width 160 not within cat page range [80, 80]
Using 156-character lines
format: 1, save_cat: 0, found: 1
Setting LESS to -ix8RmPm Manual page tmp2 ?ltline %lt?L/%L.:byte %bB?s/%s..?e
(END):?pB %pB\%..$PM Manual page tmp2 ?ltline %lt?L/%L.:byte %bB?s/%s..?e
(END):?pB %pB\%..$
Setting MAN_PN to tmp2
++priv_drop_count = 3
Starting pipeline: /usr/bin/zsoelim | /usr/lib/man-db/manconv -f UTF-8 -t
ISO-8859-1//IGNORE | nroff -mandoc -rLL=156n -rLT=156n -Tutf8 [input: {-1,
NULL}, output: {-1, NULL}]
Started "/usr/bin/zsoelim", pid 3434
Started "/usr/lib/man-db/manconv -f UTF-8 -t ISO-8859-1//IGNORE", pid 3435
Started "nroff", pid 3438
Starting pipeline: pager -s [input: {-1, NULL}, output: {0, NULL}]
Started "pager", pid 3439
source pipeline 0 returned error or EOF
source pipeline 1 returned error or EOF
Waiting for pipeline: [input: {0, tmp2}, output: {-1, NULL}]
Waiting for pipeline: /usr/bin/zsoelim | /usr/lib/man-db/manconv -f UTF-8 -t
ISO-8859-1//IGNORE | nroff -mandoc -rLL=156n -rLT=156n -Tutf8 [input: {-1,
NULL}, output: {-1, NULL}]
Active processes (3):
"/usr/bin/zsoelim" (3434) -> 0
"/usr/lib/man-db/manconv -f UTF-8 -t ISO-8859-1//IGNORE" (3435) -> 0
"nroff" (3438) -> -1
Active processes (1):
"nroff" (3438) -> 0
Waiting for pipeline: pager -s [input: {-1, NULL}, output: {0, NULL}]
Active processes (1):
"pager" (3439) -> -1
Active processes (1):
"pager" (3439) -> 0
--priv_drop_count = 2
--priv_drop_count = 1
ruid=500, euid=500
++priv_drop_count = 1
From the config file /etc/manpath.config:
Mandatory mandir `/usr/man'.
Mandatory mandir `/usr/share/man'.
Mandatory mandir `/usr/local/share/man'.
Path `/bin' mapped to mandir `/usr/share/man'.
Path `/usr/bin' mapped to mandir `/usr/share/man'.
Path `/sbin' mapped to mandir `/usr/share/man'.
Path `/usr/sbin' mapped to mandir `/usr/share/man'.
Path `/usr/local/bin' mapped to mandir `/usr/local/man'.
Path `/usr/local/bin' mapped to mandir `/usr/local/share/man'.
Path `/usr/local/sbin' mapped to mandir `/usr/local/man'.
Path `/usr/local/sbin' mapped to mandir `/usr/local/share/man'.
Path `/usr/games' mapped to mandir `/usr/share/man'.
Path `/opt/bin' mapped to mandir `/opt/man'.
Path `/opt/sbin' mapped to mandir `/opt/man'.
Global mandir `/usr/man', catdir `/var/cache/man/fsstnd'.
Global mandir `/usr/share/man', catdir `/var/cache/man'.
Global mandir `/usr/local/man', catdir `/var/cache/man/oldlocal'.
Global mandir `/usr/local/share/man', catdir `/var/cache/man/local'.
Global mandir `/opt/man', catdir `/var/cache/man/opt'.
Added section `1'.
Added section `n'.
Added section `l'.
Added section `8'.
Added section `3'.
Added section `2'.
Added section `3posix'.
Added section `3pm'.
Added section `3perl'.
Added section `5'.
Added section `4'.
Added section `9'.
Added section `6'.
Added section `7'.
`/usr/man' `' `1'
`/usr/share/man' `' `1'
`/usr/local/share/man' `' `1'
`/bin' `/usr/share/man' `0'
`/usr/bin' `/usr/share/man' `0'
`/sbin' `/usr/share/man' `0'
`/usr/sbin' `/usr/share/man' `0'
`/usr/local/bin' `/usr/local/man' `0'
`/usr/local/bin' `/usr/local/share/man' `0'
`/usr/local/sbin' `/usr/local/man' `0'
`/usr/local/sbin' `/usr/local/share/man' `0'
`/usr/games' `/usr/share/man' `0'
`/opt/bin' `/opt/man' `0'
`/opt/sbin' `/opt/man' `0'
`/usr/man' `/var/cache/man/fsstnd' `-1'
`/usr/share/man' `/var/cache/man' `-1'
`/usr/local/man' `/var/cache/man/oldlocal' `-1'
`/usr/local/share/man' `/var/cache/man/local' `-1'
`/opt/man' `/var/cache/man/opt' `-1'
`1' `' `-5'
`n' `' `-5'
`l' `' `-5'
`8' `' `-5'
`3' `' `-5'
`2' `' `-5'
`3posix' `' `-5'
`3pm' `' `-5'
`3perl' `' `-5'
`5' `' `-5'
`4' `' `-5'
`9' `' `-5'
`6' `' `-5'
`7' `' `-5'
is a tty
real user = 500; effective user = 500
using pager -s as pager
++priv_drop_count = 2
chdir /home/jwilk/trunk/bugs/man-db
Starting pipeline: [input: {0, tmp1}, output: {-1, NULL}]
pre-processors `t' from default
page_encoding = UTF-8
source_encoding = ISO-8859-1
cat_charset = UTF-8
locale_charset = UTF-8
roff_device (locale) = utf8
roff_encoding = ISO-8859-1
output_encoding = UTF-8
less_charset = utf-8
Terminal width 160
Terminal width 160 not within cat page range [80, 80]
Using 156-character lines
format: 1, save_cat: 0, found: 1
Setting LESS to -ix8RmPm Manual page tmp1 ?ltline %lt?L/%L.:byte %bB?s/%s..?e
(END):?pB %pB\%..$PM Manual page tmp1 ?ltline %lt?L/%L.:byte %bB?s/%s..?e
(END):?pB %pB\%..$
Setting MAN_PN to tmp1
++priv_drop_count = 3
Starting pipeline: /usr/bin/zsoelim | /usr/lib/man-db/manconv -f UTF-8 -t
ISO-8859-1//IGNORE | tbl | nroff -mandoc -rLL=156n -rLT=156n -Tutf8 [input:
{-1, NULL}, output: {-1, NULL}]
Started "/usr/bin/zsoelim", pid 3788
Started "/usr/lib/man-db/manconv -f UTF-8 -t ISO-8859-1//IGNORE", pid 3790
Started "tbl", pid 3792
Started "nroff", pid 3793
Starting pipeline: pager -s [input: {-1, NULL}, output: {0, NULL}]
Started "pager", pid 3794
source pipeline 0 returned error or EOF
source pipeline 1 returned error or EOF
Waiting for pipeline: [input: {0, tmp1}, output: {-1, NULL}]
Waiting for pipeline: /usr/bin/zsoelim | /usr/lib/man-db/manconv -f UTF-8 -t
ISO-8859-1//IGNORE | tbl | nroff -mandoc -rLL=156n -rLT=156n -Tutf8 [input:
{-1, NULL}, output: {-1, NULL}]
Active processes (4):
"/usr/bin/zsoelim" (3788) -> 0
"/usr/lib/man-db/manconv -f UTF-8 -t ISO-8859-1//IGNORE" (3790) -> 0
"tbl" (3792) -> 0
"nroff" (3793) -> -1
Active processes (1):
"nroff" (3793) -> 0
Waiting for pipeline: pager -s [input: {-1, NULL}, output: {0, NULL}]
Active processes (1):
"pager" (3794) -> -1
Active processes (1):
"pager" (3794) -> 0
--priv_drop_count = 2
--priv_drop_count = 1
ruid=500, euid=500
++priv_drop_count = 1
From the config file /etc/manpath.config:
Mandatory mandir `/usr/man'.
Mandatory mandir `/usr/share/man'.
Mandatory mandir `/usr/local/share/man'.
Path `/bin' mapped to mandir `/usr/share/man'.
Path `/usr/bin' mapped to mandir `/usr/share/man'.
Path `/sbin' mapped to mandir `/usr/share/man'.
Path `/usr/sbin' mapped to mandir `/usr/share/man'.
Path `/usr/local/bin' mapped to mandir `/usr/local/man'.
Path `/usr/local/bin' mapped to mandir `/usr/local/share/man'.
Path `/usr/local/sbin' mapped to mandir `/usr/local/man'.
Path `/usr/local/sbin' mapped to mandir `/usr/local/share/man'.
Path `/usr/games' mapped to mandir `/usr/share/man'.
Path `/opt/bin' mapped to mandir `/opt/man'.
Path `/opt/sbin' mapped to mandir `/opt/man'.
Global mandir `/usr/man', catdir `/var/cache/man/fsstnd'.
Global mandir `/usr/share/man', catdir `/var/cache/man'.
Global mandir `/usr/local/man', catdir `/var/cache/man/oldlocal'.
Global mandir `/usr/local/share/man', catdir `/var/cache/man/local'.
Global mandir `/opt/man', catdir `/var/cache/man/opt'.
Added section `1'.
Added section `n'.
Added section `l'.
Added section `8'.
Added section `3'.
Added section `2'.
Added section `3posix'.
Added section `3pm'.
Added section `3perl'.
Added section `5'.
Added section `4'.
Added section `9'.
Added section `6'.
Added section `7'.
`/usr/man' `' `1'
`/usr/share/man' `' `1'
`/usr/local/share/man' `' `1'
`/bin' `/usr/share/man' `0'
`/usr/bin' `/usr/share/man' `0'
`/sbin' `/usr/share/man' `0'
`/usr/sbin' `/usr/share/man' `0'
`/usr/local/bin' `/usr/local/man' `0'
`/usr/local/bin' `/usr/local/share/man' `0'
`/usr/local/sbin' `/usr/local/man' `0'
`/usr/local/sbin' `/usr/local/share/man' `0'
`/usr/games' `/usr/share/man' `0'
`/opt/bin' `/opt/man' `0'
`/opt/sbin' `/opt/man' `0'
`/usr/man' `/var/cache/man/fsstnd' `-1'
`/usr/share/man' `/var/cache/man' `-1'
`/usr/local/man' `/var/cache/man/oldlocal' `-1'
`/usr/local/share/man' `/var/cache/man/local' `-1'
`/opt/man' `/var/cache/man/opt' `-1'
`1' `' `-5'
`n' `' `-5'
`l' `' `-5'
`8' `' `-5'
`3' `' `-5'
`2' `' `-5'
`3posix' `' `-5'
`3pm' `' `-5'
`3perl' `' `-5'
`5' `' `-5'
`4' `' `-5'
`9' `' `-5'
`6' `' `-5'
`7' `' `-5'
is a tty
real user = 500; effective user = 500
using pager -s as pager
++priv_drop_count = 2
chdir /home/jwilk/trunk/bugs/man-db
Starting pipeline: [input: {0, tmp1}, output: {-1, NULL}]
pre-processors `t' from default
page_encoding = UTF-8
source_encoding = ISO-8859-2
cat_charset = UTF-8
locale_charset = UTF-8
roff_device (locale) = ascii8
roff_encoding = ISO-8859-2
output_encoding = ISO-8859-2
less_charset = utf-8
Terminal width 160
Terminal width 160 not within cat page range [80, 80]
Using 156-character lines
format: 1, save_cat: 0, found: 1
Setting LESS to -ix8RmPm Manual page tmp1 ?ltline %lt?L/%L.:byte %bB?s/%s..?e
(END):?pB %pB\%..$PM Manual page tmp1 ?ltline %lt?L/%L.:byte %bB?s/%s..?e
(END):?pB %pB\%..$
Setting MAN_PN to tmp1
++priv_drop_count = 3
Starting pipeline: /usr/bin/zsoelim | /usr/lib/man-db/manconv -f UTF-8 -t
ISO-8859-2//IGNORE | tbl | nroff -mandoc -rLL=156n -rLT=156n -Tascii8 | iconv
-c -f ISO-8859-2 -t UTF-8//TRANSLIT [input: {-1, NULL}, output: {-1, NULL}]
Started "/usr/bin/zsoelim", pid 3775
Started "/usr/lib/man-db/manconv -f UTF-8 -t ISO-8859-2//IGNORE", pid 3777
Started "tbl", pid 3779
Started "nroff", pid 3780
Started "iconv", pid 3781
Starting pipeline: pager -s [input: {-1, NULL}, output: {0, NULL}]
Started "pager", pid 3782
source pipeline 0 returned error or EOF
source pipeline 1 returned error or EOF
Waiting for pipeline: [input: {0, tmp1}, output: {-1, NULL}]
Waiting for pipeline: /usr/bin/zsoelim | /usr/lib/man-db/manconv -f UTF-8 -t
ISO-8859-2//IGNORE | tbl | nroff -mandoc -rLL=156n -rLT=156n -Tascii8 | iconv
-c -f ISO-8859-2 -t UTF-8//TRANSLIT [input: {-1, NULL}, output: {-1, NULL}]
Active processes (5):
"/usr/bin/zsoelim" (3775) -> 0
"/usr/lib/man-db/manconv -f UTF-8 -t ISO-8859-2//IGNORE" (3777) -> 0
"tbl" (3779) -> 0
"nroff" (3780) -> 0
"iconv" (3781) -> 0
Waiting for pipeline: pager -s [input: {-1, NULL}, output: {0, NULL}]
Active processes (1):
"pager" (3782) -> -1
Active processes (1):
"pager" (3782) -> 0
--priv_drop_count = 2
--priv_drop_count = 1
