On 10/24/18 4:13 PM, Chase wrote:
So the C docs work, the international docs don't, they all fail with a 
multibyte character error. How would I go about debugging this (and the regex 
patch earlier)? Also, when I noted that there were translation warnings being 
thrown, this was only for the man pages, the other docs didn't throw this.


Yeah, thats what I was afraid of... instant is *very* delicate and seems to have a variety of memory and other problems. I fixed several during the tcl conversion.

During my recent attempt at converting to utf8, I ran into many more problems with it crashing or corrupting memory. Some of them I could fix by using the appropriate Tcl_Alloc/Free routines instead of malloc/free (discovered via valgrind).

In the end, I'm thinking we cannot convert the docs to utf8 until we revamp the whole doc generation system (using XML and a modern docbook). Locale stuff (catalogs and the like) work fine with utf8 in all the locales.

Debugging these issues will be difficult. You can run the dtdocbook script with the -x and -v parameters. This outputs commands before executing them, and leaves some of the artifacts around afterwards that you can use for investigation.

I also used valgrind a couple of times to find issues in instant, though the crashes were often deep in the Tcl libs - so, more memory corruption somewhere that I could often fix with appropriate uses of Tcl_Free/Alloc, especially when dealing with strings that are fed to the interpreter.

Modern Tcl can handle utf8 fine, as long as they are proper utf8 sequences. It cannot handle other 8-bit encodings, like the upper chars (128+) present in iso8859* locales. Instant has this really stupid encode/decode functionality to get around that problem.

It is very poorly written software, and debugging it is hard due to it's use in a pipeline and it's dependences on various ENV vars being set.

As for the man pages, yes those are all C so locale translation isn't needed. Perhaps that warning/error could be disabled if a locale translation DB wasn't specified as an option. If a translation db is specified, it should definitely fail if it can't be opened (I noticed you changed that from a fatal error to a warning).

I am seriously toying with the idea of making document building optional. We would just ship pre-generated files and install those by default.

Then some major rework could go on with the docs subsystem without affecting mainline cde. Or, they could stay frozen in time until someone with the urge decides they want to work on it :)

There has got to be a cleaner and better way to do this than the way it's being done now.

Sorry I can't give you specifics on how to proceed with debugging. One way is to start at the beginning (say with your regexp changes) and do small commits/changes, one at a time until you find a change where it starts to blow up. Then start debugging at that point.

I don't know your debugging skill level, but the CDE doc system will no doubt test it well :)

-jon



Thank you for your time,
-Chase

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, October 23, 2018 9:22 PM, Jon Trulson <j...@radscan.com> wrote:

I don't have time to test this tonight, but in addition to man pages,
have you verified that all of the help text for all locales is also
generated properly? Ie: when building this stuff on linux, you should
definitely build all of the locales not just the C locale.

I'm thinking that building for all locales should be the default on
Linux again...

To make sure all of the locales are built, use:

make World IMAKE_DEFINES='-DDtLocalesToBuild="de_DE.ISO8859-1
es_ES.ISO8859-1 fr_FR.ISO8859-1 it_IT.ISO8859-1 en_US.UTF-8"'

Instead of just a 'make World'.

You need to make sure all of these build and install properly. You
should also make sure they actually work - login with a de_DE locale for
example.

The locale DB stuff is interesting and might be a problem. It's
probably not used for generating manpages, which are all C locale, but
it's definitely needed for help text.

-jon

On 10/23/18 7:45 PM, Chase via cdesktopenv-devel wrote:

This is a big patch, I have thoroughly tested it and man pages generate
perfectly. The only problem that I've encountered, is that it writes a
lot of nonsense to stderr, mostly warnings that it could not find a
translation database. I am leaving the (disabled) extra copy of instant
in dbtoman just in case anyone wants to see if there is anything else
that can be taken from it, but most of the remaining code is seemingly
redundant or useless, so if there are no qualms about it, I will remove
it in another patch.
Thank you for your time,
-Chase

cdesktopenv-devel mailing list
cdesktopenv-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/cdesktopenv-devel

--

Jon Trulson

"In the game of chess, you can never let your adversary see your pieces."

- Zapp Brannigan

cdesktopenv-devel mailing list
cdesktopenv-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/cdesktopenv-devel


--
Jon Trulson

"In the game of chess, you can never let your adversary see your pieces."

                              - Zapp Brannigan


_______________________________________________
cdesktopenv-devel mailing list
cdesktopenv-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/cdesktopenv-devel

Reply via email to