On 10/24/18 4:13 PM, Chase wrote:
So the C docs work, the international docs don't, they all fail with a
multibyte character error. How would I go about debugging this (and the regex
patch earlier)? Also, when I noted that there were translation warnings being
thrown, this was only for the man pages, the other docs didn't throw this.
Yeah, thats what I was afraid of... instant is *very* delicate and seems
to have a variety of memory and other problems. I fixed several during
the tcl conversion.
During my recent attempt at converting to utf8, I ran into many more
problems with it crashing or corrupting memory. Some of them I could
fix by using the appropriate Tcl_Alloc/Free routines instead of
malloc/free (discovered via valgrind).
In the end, I'm thinking we cannot convert the docs to utf8 until we
revamp the whole doc generation system (using XML and a modern docbook).
Locale stuff (catalogs and the like) work fine with utf8 in all the
locales.
Debugging these issues will be difficult. You can run the dtdocbook
script with the -x and -v parameters. This outputs commands before
executing them, and leaves some of the artifacts around afterwards that
you can use for investigation.
I also used valgrind a couple of times to find issues in instant, though
the crashes were often deep in the Tcl libs - so, more memory corruption
somewhere that I could often fix with appropriate uses of
Tcl_Free/Alloc, especially when dealing with strings that are fed to the
interpreter.
Modern Tcl can handle utf8 fine, as long as they are proper utf8
sequences. It cannot handle other 8-bit encodings, like the upper chars
(128+) present in iso8859* locales. Instant has this really stupid
encode/decode functionality to get around that problem.
It is very poorly written software, and debugging it is hard due to it's
use in a pipeline and it's dependences on various ENV vars being set.
As for the man pages, yes those are all C so locale translation isn't
needed. Perhaps that warning/error could be disabled if a locale
translation DB wasn't specified as an option. If a translation db is
specified, it should definitely fail if it can't be opened (I noticed
you changed that from a fatal error to a warning).
I am seriously toying with the idea of making document building
optional. We would just ship pre-generated files and install those by
default.
Then some major rework could go on with the docs subsystem without
affecting mainline cde. Or, they could stay frozen in time until
someone with the urge decides they want to work on it :)
There has got to be a cleaner and better way to do this than the way
it's being done now.
Sorry I can't give you specifics on how to proceed with debugging. One
way is to start at the beginning (say with your regexp changes) and do
small commits/changes, one at a time until you find a change where it
starts to blow up. Then start debugging at that point.
I don't know your debugging skill level, but the CDE doc system will no
doubt test it well :)
-jon
Thank you for your time,
-Chase
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, October 23, 2018 9:22 PM, Jon Trulson <j...@radscan.com> wrote:
I don't have time to test this tonight, but in addition to man pages,
have you verified that all of the help text for all locales is also
generated properly? Ie: when building this stuff on linux, you should
definitely build all of the locales not just the C locale.
I'm thinking that building for all locales should be the default on
Linux again...
To make sure all of the locales are built, use:
make World IMAKE_DEFINES='-DDtLocalesToBuild="de_DE.ISO8859-1
es_ES.ISO8859-1 fr_FR.ISO8859-1 it_IT.ISO8859-1 en_US.UTF-8"'
Instead of just a 'make World'.
You need to make sure all of these build and install properly. You
should also make sure they actually work - login with a de_DE locale for
example.
The locale DB stuff is interesting and might be a problem. It's
probably not used for generating manpages, which are all C locale, but
it's definitely needed for help text.
-jon
On 10/23/18 7:45 PM, Chase via cdesktopenv-devel wrote:
This is a big patch, I have thoroughly tested it and man pages generate
perfectly. The only problem that I've encountered, is that it writes a
lot of nonsense to stderr, mostly warnings that it could not find a
translation database. I am leaving the (disabled) extra copy of instant
in dbtoman just in case anyone wants to see if there is anything else
that can be taken from it, but most of the remaining code is seemingly
redundant or useless, so if there are no qualms about it, I will remove
it in another patch.
Thank you for your time,
-Chase
cdesktopenv-devel mailing list
cdesktopenv-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/cdesktopenv-devel
--
Jon Trulson
"In the game of chess, you can never let your adversary see your pieces."
- Zapp Brannigan
cdesktopenv-devel mailing list
cdesktopenv-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/cdesktopenv-devel
--
Jon Trulson
"In the game of chess, you can never let your adversary see your pieces."
- Zapp Brannigan
_______________________________________________
cdesktopenv-devel mailing list
cdesktopenv-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/cdesktopenv-devel