Re: SIGSEGV if iconv error from local layout or module

2024-04-08 Thread Richard Kimberly Heck
On 4/8/24 18:57, Scott Kostyshak wrote: If we have a UTF8 character in the user preamble, LyX gives a very user-friendly warning if the document encoding is not UTF8. However, if the same situation happens for the local layout (or in a module), I get a SIGSEGV when compiling. To reproduce, open

SIGSEGV if iconv error from local layout or module

2024-04-08 Thread Scott Kostyshak
If we have a UTF8 character in the user preamble, LyX gives a very user-friendly warning if the document encoding is not UTF8. However, if the same situation happens for the local layout (or in a module), I get a SIGSEGV when compiling. To reproduce, open the attached document and compile to PDF.

Re: iconv errors when opening IEEEtran-Conference.lyx

2022-12-10 Thread Scott Kostyshak
On Sat, Dec 10, 2022 at 01:02:15PM +0100, Jürgen Spitzmüller wrote: > Am Freitag, dem 09.12.2022 um 12:08 -0500 schrieb Scott Kostyshak: > > Changing \inputencoding from utf8 to auto-legacy leads to no iconv > > error. Is that a reasonable thing to do? I don't understan

Re: iconv errors when opening IEEEtran-Conference.lyx

2022-12-10 Thread Jürgen Spitzmüller
Am Freitag, dem 09.12.2022 um 12:08 -0500 schrieb Scott Kostyshak: > Changing \inputencoding from utf8 to auto-legacy leads to no iconv > error. Is that a reasonable thing to do? I don't understand encodings > well. This is one possibility. The other one, which makes more sense i

Re: iconv errors when opening IEEEtran-Conference.lyx

2022-12-10 Thread Jürgen Spitzmüller
Am Samstag, dem 10.12.2022 um 12:34 +0100 schrieb Kornel Benko: > The error shows while checking IEEEabrv.bib, which is latin1 encoded > (and not UTF-8 as > expected). > The problem is with the name > 'Nicolás Barabino'. (bibtex/bib/ieeetran/IEEEabrv.bib:24) Fixed. -- Jürgen signature.as

Re: iconv errors when opening IEEEtran-Conference.lyx

2022-12-10 Thread Kornel Benko
: > > > > > > > When opening IEEEtran-Conference.lyx I get iconv errors: > > > > > > > > Error 84 returned from iconv when converting from UTF-8 to UCS-4LE: > > > > Invalid or incomplete multibyte or wide character Converted input

Re: iconv errors when opening IEEEtran-Conference.lyx

2022-12-09 Thread Kornel Benko
Am Fri, 9 Dec 2022 14:41:03 -0500 schrieb Scott Kostyshak : > On Fri, Dec 09, 2022 at 07:28:03PM +0100, Kornel Benko wrote: > > Am Fri, 9 Dec 2022 12:08:27 -0500 > > schrieb Scott Kostyshak : > > > > > When opening IEEEtran-Conference.lyx I get iconv errors: &g

Re: iconv errors when opening IEEEtran-Conference.lyx

2022-12-09 Thread Scott Kostyshak
On Fri, Dec 09, 2022 at 07:28:03PM +0100, Kornel Benko wrote: > Am Fri, 9 Dec 2022 12:08:27 -0500 > schrieb Scott Kostyshak : > > > When opening IEEEtran-Conference.lyx I get iconv errors: > > > > Error 84 returned from iconv when converting from UTF-8 to UCS-4LE:

Re: iconv errors when opening IEEEtran-Conference.lyx

2022-12-09 Thread Kornel Benko
Am Fri, 9 Dec 2022 12:08:27 -0500 schrieb Scott Kostyshak : > When opening IEEEtran-Conference.lyx I get iconv errors: > > Error 84 returned from iconv when converting from UTF-8 to UCS-4LE: > Invalid or incomplete multibyte or wide character Converted input: > 0x0

iconv errors when opening IEEEtran-Conference.lyx

2022-12-09 Thread Scott Kostyshak
When opening IEEEtran-Conference.lyx I get iconv errors: Error 84 returned from iconv when converting from UTF-8 to UCS-4LE: Invalid or incomplete multibyte or wide character Converted input: 0x0a 0x49 ... [cut] Changing \inputencoding from utf8 to auto-legacy leads to no iconv error. Is

Re: Automake configuration with '--with-included-iconv'

2020-11-18 Thread Kornel Benko
Am Wed, 18 Nov 2020 16:45:34 +0100 schrieb Jean-Marc Lasgouttes : > Le 18/11/2020 à 16:00, Kornel Benko a écrit : > > Yes, compilable with cmake. For convenience of the automake users I tried > > to compile > > with automake and failed. > > Reading commit log of 08afc52c4cc, I see that it is n

Re: Automake configuration with '--with-included-iconv'

2020-11-18 Thread Jean-Marc Lasgouttes
Le 18/11/2020 à 16:00, Kornel Benko a écrit : Yes, compilable with cmake. For convenience of the automake users I tried to compile with automake and failed. Reading commit log of 08afc52c4cc, I see that it is not supposed to work with Linux. I am wary of tweaking it, because all this looks pr

Re: Automake configuration with '--with-included-iconv'

2020-11-18 Thread Jean-Marc Lasgouttes
Le 18/11/2020 à 16:00, Kornel Benko a écrit : Yes, compilable with cmake. For convenience of the automake users I tried to compile with automake and failed. Right, I will take a look. JMarc -- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel

Re: Automake configuration with '--with-included-iconv'

2020-11-18 Thread Kornel Benko
Am Wed, 18 Nov 2020 15:54:08 +0100 schrieb Jean-Marc Lasgouttes : > Le 18/11/2020 à 15:41, Scott Kostyshak a écrit : > > We have several tests set up that we can use to check whether other > > solutions work well. In fact, some of the tests fail when compiling LyX > > with o

Re: Automake configuration with '--with-included-iconv'

2020-11-18 Thread Jean-Marc Lasgouttes
Le 18/11/2020 à 15:41, Scott Kostyshak a écrit : We have several tests set up that we can use to check whether other solutions work well. In fact, some of the tests fail when compiling LyX with our included iconv but not when compiling LyX with a system iconv (on Ubuntu 20.04 and 19.10 at least

Re: Automake configuration with '--with-included-iconv'

2020-11-18 Thread Scott Kostyshak
ests set up that we can use to check whether other solutions work well. In fact, some of the tests fail when compiling LyX with our included iconv but not when compiling LyX with a system iconv (on Ubuntu 20.04 and 19.10 at least). The tests consist of exporting the files in autotests/export/latex/unicodesy

Re: Automake configuration with '--with-included-iconv'

2020-11-18 Thread Jean-Marc Lasgouttes
Le 18/11/2020 à 14:43, Scott Kostyshak a écrit : I think it is only useful for compiling with mingw32, but my memory is fuzzy. Removing it would be a good option otherwise. I think we use it for converting characters between encodings? I don't know anything about it, I just know that it has ye

Re: Automake configuration with '--with-included-iconv'

2020-11-18 Thread Scott Kostyshak
On Wed, Nov 18, 2020 at 10:58:14AM +0100, Jean-Marc Lasgouttes wrote: > Le 17/11/2020 à 23:16, Kornel Benko a écrit : > > $ ./autogen.sh > > $ cd bindir > > $ srcdir/configure ... --with-included-iconv ... > > ... > > config.status: creating 3rdparty/Mak

Re: Automake configuration with '--with-included-iconv'

2020-11-18 Thread Jean-Marc Lasgouttes
Le 17/11/2020 à 23:16, Kornel Benko a écrit : $ ./autogen.sh $ cd bindir $ srcdir/configure ... --with-included-iconv ... ... config.status: creating 3rdparty/Makefile config.status: creating 3rdparty/boost/Makefile config.status: creating 3rdparty/dtl/Makefile config.status: creating

Re: Automake configuration with '--with-included-iconv'

2020-11-17 Thread Scott Kostyshak
On Tue, Nov 17, 2020 at 11:16:51PM +0100, Kornel Benko wrote: > $ ./autogen.sh > $ cd bindir > $ srcdir/configure ... --with-included-iconv ... > ... > config.status: creating 3rdparty/Makefile > config.status: creating 3rdparty/boost/Makefile > config.status: creating

Automake configuration with '--with-included-iconv'

2020-11-17 Thread Kornel Benko
$ ./autogen.sh $ cd bindir $ srcdir/configure ... --with-included-iconv ... ... config.status: creating 3rdparty/Makefile config.status: creating 3rdparty/boost/Makefile config.status: creating 3rdparty/dtl/Makefile config.status: creating 3rdparty/hunspell/Makefile config.status: creating

Re: iconv warnings

2017-09-10 Thread Uwe Stöhr
El 10.09.2017 a las 14:18, Uwe Stöhr escribió: I just compiled the current 2.3.x branch to perform some final tests. I noted some avoidable compiler warnings in libiconv, see below I reported the warnings to the email address and got already a reply. They will analyze the signed/unsigned

iconv warnings

2017-09-10 Thread Uwe Stöhr
https://github.com/win-iconv/win-iconv/issues which seems to be inactive but also https://www.gnu.org/software/libiconv/ that suggest a mail address to report bugs. thanks and regards Uwe d:\lyxgit\2.3.x\3rdparty\libiconv\1.14\lib\utf7.h(162): warning C4018: '<': signed/unsigned mismatch

Re: attic/id_UserGuide iconv errors

2017-05-04 Thread Scott Kostyshak
On Thu, May 04, 2017 at 08:15:23PM +0200, Kornel Benko wrote: > Thanks for the testcase. Yes thanks for taking care of that. Scott signature.asc Description: PGP signature

Re: attic/id_UserGuide iconv errors

2017-05-04 Thread Kornel Benko
Am Donnerstag, 4. Mai 2017 um 16:08:41, schrieb Guenter Milde ... > I checked 0x2018, it's the LEFT SINGLE QUOTATION MARK ‘. > > This appears 2 times in id_UserGuide.lyx: > > 28091: description "The quote sign is output by writing ‘ \"\"\"\" '" > 43702: It is recommended that you always

Re: attic/id_UserGuide iconv errors

2017-05-04 Thread Guenter Milde
On 2017-05-03, Kornel Benko wrote: > Am Mittwoch, 3. Mai 2017 um 21:20:11, schrieb Guenter Milde > >> On 2017-04-30, Scott Kostyshak wrote: >> > On current master if I do: >> > ctest -R "id_UserGuide_pdf2" >> > I get a failed export, and ico

Re: attic/id_UserGuide iconv errors

2017-05-03 Thread Kornel Benko
Am Mittwoch, 3. Mai 2017 um 21:20:11, schrieb Guenter Milde > On 2017-04-30, Scott Kostyshak wrote: > > > [-- Type: text/plain, Encoding: --] > > > On current master if I do: > > > ctest -R "id_UserGuide_pdf2" > > > I get a failed export, an

Re: attic/id_UserGuide iconv errors

2017-05-03 Thread Guenter Milde
On 2017-04-30, Scott Kostyshak wrote: > [-- Type: text/plain, Encoding: --] > On current master if I do: > ctest -R "id_UserGuide_pdf2" > I get a failed export, and iconv errors such as: > Error 84 returned from iconv when converting from UCS-4LE to > ISO-88

Re: attic/id_UserGuide iconv errors

2017-04-30 Thread Scott Kostyshak
On Mon, May 01, 2017 at 12:19:24AM +0200, Uwe Stöhr wrote: > El 30.04.2017 a las 23:55, Scott Kostyshak escribió: > > On current master if I do: > > > >ctest -R "id_UserGuide_pdf2" > > It is an attic file so it is not under review for a long time. > I would say please ignore attic files. Yes

Re: attic/id_UserGuide iconv errors

2017-04-30 Thread Uwe Stöhr
El 30.04.2017 a las 23:55, Scott Kostyshak escribió: On current master if I do: ctest -R "id_UserGuide_pdf2" It is an attic file so it is not under review for a long time. I would say please ignore attic files. regards Uwe

attic/id_UserGuide iconv errors

2017-04-30 Thread Scott Kostyshak
On current master if I do: ctest -R "id_UserGuide_pdf2" I get a failed export, and iconv errors such as: Error 84 returned from iconv when converting from UCS-4LE to ISO-8859-15: Invalid or incomplete multibyte or wide character I do not get the error when exporting in the G

Re: iconv error with listing + Spanish

2017-03-01 Thread Scott Kostyshak
On Wed, Mar 01, 2017 at 05:35:45PM +, Guenter Milde wrote: > This confirms it as an instance of #9871 (or related to it). Good, I added the information to the ticket. Scott signature.asc Description: PGP signature

Re: iconv error with listing + Spanish

2017-03-01 Thread Guenter Milde
On 2017-03-01, Kornel Benko wrote: > Am Mittwoch, 1. März 2017 um 11:35:45, schrieb Guenter Milde > >> On 2017-03-01, Scott Kostyshak wrote: >> >> > Note that this affects lib/examples/es/linguistics.lyx >> >> > >> >> > Can anyone else reproduce? Is the problem obvious? >> >> > >> >> > I can r

Re: iconv error with listing + Spanish

2017-03-01 Thread Scott Kostyshak
On Wed, Mar 01, 2017 at 11:35:45AM +, Guenter Milde wrote: > 36b9645b is part of the fix to #9740 (XeTeX with TeX fonts). > (BTW: how do you get the content/description of a change-set given this > strange number only?) git show 36b9645b Does that do what you want? The number is referred to

Re: iconv error with listing + Spanish

2017-03-01 Thread Scott Kostyshak
On Wed, Mar 01, 2017 at 11:41:50AM +, Guenter Milde wrote: > On 2017-02-28, Scott Kostyshak wrote: > > On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote: > > >> Note that this affects lib/examples/es/linguistics.lyx > > >> Can anyone else reproduce? Is the problem obvious? > >

Re: iconv error with listing + Spanish

2017-03-01 Thread Kornel Benko
Am Mittwoch, 1. März 2017 um 11:35:45, schrieb Guenter Milde > On 2017-03-01, Scott Kostyshak wrote: > > On Tue, Feb 28, 2017 at 06:24:51PM -0500, Scott Kostyshak wrote: > >> On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote: > > >> > Note that this affects lib/examples/es/linguist

Re: iconv error with listing + Spanish

2017-03-01 Thread Guenter Milde
On 2017-02-28, Scott Kostyshak wrote: > On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote: >> Note that this affects lib/examples/es/linguistics.lyx >> Can anyone else reproduce? Is the problem obvious? >> I can reproduce the issue on 2.2.0 and on master. I cannot reproduce on >> 2

Re: iconv error with listing + Spanish

2017-03-01 Thread Guenter Milde
On 2017-03-01, Scott Kostyshak wrote: > On Tue, Feb 28, 2017 at 06:24:51PM -0500, Scott Kostyshak wrote: >> On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote: >> > Note that this affects lib/examples/es/linguistics.lyx >> > >> > Can anyone else reproduce? Is the problem obvious? >>

Re: iconv error with listing + Spanish

2017-02-28 Thread Scott Kostyshak
On Tue, Feb 28, 2017 at 06:24:51PM -0500, Scott Kostyshak wrote: > On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote: > > > Note that this affects lib/examples/es/linguistics.lyx > > > > Can anyone else reproduce? Is the problem obvious? > > > > I can reproduce the issue on 2.2.0 a

Re: iconv error with listing + Spanish

2017-02-28 Thread Scott Kostyshak
On Tue, Feb 28, 2017 at 02:07:38PM -0500, Scott Kostyshak wrote: > Note that this affects lib/examples/es/linguistics.lyx > > Can anyone else reproduce? Is the problem obvious? > > I can reproduce the issue on 2.2.0 and on master. I cannot reproduce on > 2.1.0. I can do a git bisect if someone t

iconv error with listing + Spanish

2017-02-28 Thread Scott Kostyshak
l: Error 84 returned from iconv when converting from UCS-4LE to ascii: Invalid or incomplete multibyte or wide character Converted input: 0x005c 0x0062 0x0061 0x0074 0x0063 0x0068 0x006d 0x006f 0x0064 0x0065 0x000a 0x005c 0x006d 0x0061 0x006b 0x0065 0x0061 0x0074 0x006c 0x0065 0x0074 0x0074 0x0065 0

iconv failure (was: ctests for input encoding)

2016-11-17 Thread Guenter Milde
gs for all ASCII chars! >> # DOS code page, Western European: >> export/export/latex/unicodesymbols/*_cp858_pdf2 >> # Cyrillic >> export/export/latex/unicodesymbols/*_pt254_pdf2 Indeed, these encodings are not supported by iconv (at least not the version installed with Debian/testing). Thanks a lot, Günter

Re: [LyX/master] Ensure that iconv and zlib are always found

2016-07-05 Thread Kornel Benko
Am Dienstag, 5. Juli 2016 um 21:53:49, schrieb Georg Baum > Kornel Benko wrote: > > > Did it at a51847f. > > > > By the way, I get these linkage error independently if I select both (z + > > iconv) as local or only one of them. Tested each time on empty build

Re: [LyX/master] Ensure that iconv and zlib are always found

2016-07-05 Thread Georg Baum
Kornel Benko wrote: > Did it at a51847f. > > By the way, I get these linkage error independently if I select both (z + > iconv) as local or only one of them. Tested each time on empty build build > tree. Thank you very much. I am sorry for the trouble and might have a look with

Re: [LyX/master] Ensure that iconv and zlib are always found

2016-07-05 Thread Kornel Benko
mpilation is OK too, but if it comes to bind I have many > > unsatisfied references. > > > > Will try to find out whats wrong, but I remember having problems before. > > That's why hunspel and iconv/zlib were separated. (I mean > > 'if(LYX_3RDPARTY_BUILD)'

Re: [LyX/master] Ensure that iconv and zlib are always found

2016-07-03 Thread Kornel Benko
autotools? > Yes /usr/src/lyx/lyx-git/configure --bindir=/usr/local/bin --datarootdir=/usr/local/share/lyx2.3 --without-included-boost --with-included-iconv --with-included-zlib --with-included-hunspell --with-version-suffix=2.3 ... Making all in libiconv make[3]: Entering direct

Re: [LyX/master] Ensure that iconv and zlib are always found

2016-07-03 Thread Georg Baum
Kornel Benko wrote: > No, sorry. Looks like the same quadrillion error messages. I do not understand how these errors could be related to my commit. Do they go away if you call cmake without LYX_3RDPARTY_BUILD? > Trying now with clean build tree: > ... > Same result. As an example, this is the

Re: [LyX/master] Ensure that iconv and zlib are always found

2016-07-03 Thread Kornel Benko
mpilation is OK too, but if it comes to bind I have many > > unsatisfied references. > > > > Will try to find out whats wrong, but I remember having problems before. > > That's why hunspel and iconv/zlib were separated. (I mean > > 'if(LYX_3RDPARTY_BUILD)'

Re: [LyX/master] Ensure that iconv and zlib are always found

2016-07-03 Thread Georg Baum
> Will try to find out whats wrong, but I remember having problems before. > That's why hunspel and iconv/zlib were separated. (I mean > 'if(LYX_3RDPARTY_BUILD)' on separate places in CMakeLists.txt) If you want to use the included hunspell and not the included zlib a

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jean-Marc Lasgouttes
11/04/2014 12:01, Stephan Witt: Is there a way to avoid this "feature"? I don't know. It's a black box. It's a OS service. Perhaps it can be configured somewhere, via System Preferences or API. With LyX you can use hunspell as the spell checker backend instead. Stephan It looks liike there

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Stephan Witt
Am 11.04.2014 um 11:56 schrieb Jean-Marc Lasgouttes : > 11/04/2014 11:23, Stephan Witt: >>> So do you mean that if I write in an English text "somme" instead >>> of "some", if will be considered an OK work because "somme" exists >>> in French? Is that supposed to be a feature? >> >> Indeed. Follo

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jean-Marc Lasgouttes
11/04/2014 11:23, Stephan Witt: So do you mean that if I write in an English text "somme" instead of "some", if will be considered an OK work because "somme" exists in French? Is that supposed to be a feature? Indeed. Following is the debug output of text input while instant-spellchecking is en

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Stephan Witt
Am 11.04.2014 um 10:23 schrieb Jean-Marc Lasgouttes : > 11/04/2014 08:23, Stephan Witt: >> Am 11.04.2014 um 01:36 schrieb Cyrille Artho : >> >>> I agree that it would be good to have all dictionaries in utf-8, but I'm >>> not sure if this is feasible for a typical user/installation. >>> >>> Ano

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jean-Marc Lasgouttes
11/04/2014 11:06, Cyrille Artho: I always thought that the French did this on purpose to not surrender to the fact that the English language dominates the world. Vincent They did this to the keyboard layout, too. If you ever tried to use a computer in a French Internet cafe, good luck typing y

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Cyrille Artho
I always thought that the French did this on purpose to not surrender to the fact that the English language dominates the world. Vincent They did this to the keyboard layout, too. If you ever tried to use a computer in a French Internet cafe, good luck typing your password (even your username

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jean-Marc Lasgouttes
11/04/2014 10:31, Vincent van Ravesteijn: Recently I did some proofreading of a paper written with TeXShop (LaTeX editor for Mac). It turned out that the text was peppered with french words. From what I understand, this horror was a joint work of automatic correction and automatic language detec

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Vincent van Ravesteijn
> So do you mean that if I write in an English text "somme" instead of "some", > if will be considered an OK work because "somme" exists in French? Is that > supposed to be a feature? > I guess that the guessing is done for the whole paragraph > Recently I did some proofreading of a paper wri

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jean-Marc Lasgouttes
11/04/2014 08:23, Stephan Witt: Am 11.04.2014 um 01:36 schrieb Cyrille Artho : I agree that it would be good to have all dictionaries in utf-8, but I'm not sure if this is feasible for a typical user/installation. Another option would be for LyX to tokenize the text and forward it word by wo

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jürgen Spitzmüller
2014-04-10 22:30 GMT+02:00 Stephan Witt: > Am 10.04.2014 um 20:43 schrieb Georg Baum: > > It is probably not difficult to implement sensible behaviour for > "ignore" > > and "ignore all" for these words: HunspellChecker has already a member > > variable ignored_ which tracks ignored words, so if

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jürgen Spitzmüller
2014-04-10 20:43 GMT+02:00 Georg Baum: > Does this mean that we need to maintain our own versions? If not then it is > probably the best solution, if yes then I'd rather not do it. > The Chromium project does maintain utf8 versions (or "deltas", for that matter): http://www.chromium.org/developer

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Stephan Witt
Am 11.04.2014 um 01:36 schrieb Cyrille Artho : > I agree that it would be good to have all dictionaries in utf-8, but I'm not > sure if this is feasible for a typical user/installation. > > Another option would be for LyX to tokenize the text and forward it word by > word to the spell checker.

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Vincent van Ravesteijn
On Fri, Apr 11, 2014 at 1:40 AM, Cyrille Artho wrote: > Regarding the idea I just mentioned before, there is a major flaw. > > Asian languages do not have spaces. Tokenizing a text into words requires a > dictionary and is a non-trivial problem (due to inflection: different verb > forms need to be

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Cyrille Artho
Regarding the idea I just mentioned before, there is a major flaw. Asian languages do not have spaces. Tokenizing a text into words requires a dictionary and is a non-trivial problem (due to inflection: different verb forms need to be recognized, etc.). We can therefore not just scan for white

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Cyrille Artho
I agree that it would be good to have all dictionaries in utf-8, but I'm not sure if this is feasible for a typical user/installation. Another option would be for LyX to tokenize the text and forward it word by word to the spell checker. This way, we could handle "Ignore All" in LyX itself ra

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Stephan Witt
Am 10.04.2014 um 20:43 schrieb Georg Baum : > Jürgen Spitzmüller wrote: > >> The point is that users cannot do something sensible with such marked >> words (except for adding them into the personal dictionary). > > It is probably not difficult to implement sensible behaviour for "ignore" > and

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Georg Baum
Jürgen Spitzmüller wrote: > The point is that users cannot do something sensible with such marked > words (except for adding them into the personal dictionary). It is probably not difficult to implement sensible behaviour for "ignore" and "ignore all" for these words: HunspellChecker has already

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Jean-Marc Lasgouttes
10/04/2014 14:29, Jürgen Spitzmüller: No, if the encoding fits, I can hit "Ignore all" and only ignore you (or your name's spelling, for that matter) in the current document (which is what I do for names usually, except for very recurrent names). If the encoding does not fit, hitting "Ignore all"

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Jürgen Spitzmüller
2014-04-10 14:18 GMT+02:00 Jean-Marc Lasgouttes : > > The point is that users cannot do something sensible with such marked >> words (except for adding them into the personal dictionary). >> > > Sure, but the same holds for "Lasgouttes", doesn't it? > No, if the encoding fits, I can hit "Ignore a

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Jean-Marc Lasgouttes
10/04/2014 14:14, Jürgen Spitzmüller: There is something that I do not understand. If the word is not representable in the German dictionary, presumably it is not part of the language. "Lasgoittes" is perfectly representable in any latin encoding and yet the spell-checker will mar

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Jürgen Spitzmüller
2014-04-10 10:11 GMT+02:00 JeanMarc Lasgouttes : > There is something that I do not understand. If the word is not > representable in the German dictionary, presumably it is not part of the > language. "Lasgoittes" is perfectly representable in any latin encoding and > yet the spell-checker will m

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread JeanMarc Lasgouttes
There is something that I do not understand. If the word is not representable in the German dictionary, presumably it is not part of the language. "Lasgoittes" is perfectly representable in any latin encoding and yet the spell-checker will mark it as misspelled. Why should it be different for a

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Stephan Witt
Am 10.04.2014 um 08:57 schrieb Cyrille Artho : > How is the call to iconv implemented? On the application level, the interface > is probably not flexible enough; it is easy to ignore non-convertible > characters, but they are just removed from the output. > > The C libra

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Cyrille Artho
How is the call to iconv implemented? On the application level, the interface is probably not flexible enough; it is easy to ignore non-convertible characters, but they are just removed from the output. The C library interface is probably richer. Is it possible to convert text word by word

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Jürgen Spitzmüller
2014-04-10 1:02 GMT+02:00 Cyrille Artho : > I think we have to use Unicode for all the given operations and (a) either > risk a mismatch for each word that is not learned/ignored, or (b) > up-convert words in the dictionary before they are matched. The latter > solution implies that the dictionary

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Cyrille Artho
to use UTF-8 to look up words. When down-converting text to the character set of the target language, we can ignore non-convertible characters silently, but echo 'István' | iconv -c -f utf-8 -t ascii yields "Istvn", which is not very useful. I think we have to use U

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Stephan Witt
Am 09.04.2014 um 11:12 schrieb Jürgen Spitzmüller : > 2014-04-09 10:59 GMT+02:00 Stephan Witt : > > That's a good example. So, my parents are from Hungarian and named me István. > Let's assume the á isn't valid in german iso encoding. Then > * I can change my name to Stephan - to avoid to spell m

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Georg Baum
Jürgen Spitzmüller wrote: > 2014-04-09 9:52 GMT+02:00 Jean-Marc Lasgouttes : > >> I think the lyxerr message could be rewritten to be at least useful. Who >> found a use for the use hex dump anyways? >> > > Agreed. Me too. I think this output is still unchanged from the time when we had bugs i

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Jürgen Spitzmüller
2014-04-09 10:59 GMT+02:00 Stephan Witt : > > That's a good example. So, my parents are from Hungarian and named me > István. > Let's assume the á isn't valid in german iso encoding. Then > * I can change my name to Stephan - to avoid to spell my name on every > formal occasion > * if I don't like

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Stephan Witt
Am 09.04.2014 um 08:53 schrieb Jürgen Spitzmüller : > 2014-04-09 8:40 GMT+02:00 Stephan Witt : > Can you provide an example, please? If the word cannot be converted to > Hunspell dictionary encoding > the dictionary is broken or the language is not correct, isn't it? > > Depends on how you defin

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Jürgen Spitzmüller
2014-04-09 9:52 GMT+02:00 Jean-Marc Lasgouttes : > I think the lyxerr message could be rewritten to be at least useful. Who > found a use for the use hex dump anyways? > Agreed. Jürgen > > JMarc > >

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Jean-Marc Lasgouttes
09/04/2014 08:14, Jürgen Spitzmüller: 2014-04-08 22:42 GMT+02:00 Georg Baum: The change in src/support/unicode.cpp is problematic: It disables all error handling, not only the lyxerr output. Also, if you now throw an exception there, you need to make sure that all callers can

Re: [patch for 2.2] silence iconv warnings

2014-04-08 Thread Jürgen Spitzmüller
2014-04-09 8:40 GMT+02:00 Stephan Witt : > Can you provide an example, please? If the word cannot be converted to > Hunspell dictionary encoding > the dictionary is broken or the language is not correct, isn't it? > Depends on how you define "language". Think of names. > You're right, the user

Re: [patch for 2.2] silence iconv warnings

2014-04-08 Thread Stephan Witt
Am 09.04.2014 um 08:14 schrieb Jürgen Spitzmüller : > 2014-04-08 22:42 GMT+02:00 Georg Baum: > The change in src/support/unicode.cpp is problematic: It disables all error > handling, not only the lyxerr output. Also, if you now throw an exception > there, you need to make sure that all callers ca

Re: [patch for 2.2] silence iconv warnings

2014-04-08 Thread Jürgen Spitzmüller
2014-04-08 22:42 GMT+02:00 Georg Baum: > The change in src/support/unicode.cpp is problematic: It disables all error > handling, not only the lyxerr output. Also, if you now throw an exception > there, you need to make sure that all callers can cope with that. Maybe one > solution would be to thro

Re: [patch for 2.2] silence iconv warnings

2014-04-08 Thread Georg Baum
Jürgen Spitzmüller wrote: > I am not very familiar with iconv/unicode and exception handling, so I > would appreciate a critical review. The change in src/support/unicode.cpp is problematic: It disables all error handling, not only the lyxerr output. Also, if you now throw an exception

[patch for 2.2] silence iconv warnings

2014-04-06 Thread Jürgen Spitzmüller
When scrolling through a document while instant-spellchecking is enabled and Hunspell used, LyX spits out iconv errors if a word appears which is not in the Hunspell dictionary's encoding. E.g.: Error returned from iconv EILSEQ An invalid multibyte sequence has been encountered in the input.

iconv fails on Unicode dashes

2009-08-03 Thread Jean-Pierre Chrétien
Hello, I recently discovered that I could not convert some docs from UTF to latin1 with iconv, which stops on message: iconv: illegal input sequence at position 851 The reason is the Unicode encoding of dashes, e.g. emdash (exported all right in LaTeX as \textemdash), iconv does not seem to be

Re: [patch] dynamic iconv outbuffer

2009-05-19 Thread Jürgen Spitzmüller
Jean-Marc Lasgouttes wrote: > >> Could someone please join in reviewing? > > > > Seems quite straightforward to me. > > +1 Thank you both. I'm committing, then. Jürgen

Re: [patch] dynamic iconv outbuffer

2009-05-19 Thread Jean-Marc Lasgouttes
Enrico Forestieri writes: > On Tue, May 19, 2009 at 07:48:43AM +0200, Jürgen Spitzmüller wrote: > >> Could someone please join in reviewing? > > Seems quite straightforward to me. +1 JMarc

Re: [patch] dynamic iconv outbuffer

2009-05-19 Thread Enrico Forestieri
On Tue, May 19, 2009 at 07:48:43AM +0200, Jürgen Spitzmüller wrote: > Could someone please join in reviewing? Seems quite straightforward to me. -- Enrico

[patch] dynamic iconv outbuffer

2009-05-18 Thread Jürgen Spitzmüller
http://www.lyx.org/trac/ticket/5951 The attached patch, contributed by Georg, fixes a dataloss issue. With a larger chunk of text in a non-Western encoding, it can happen that the iconv outbuffer (which is currently fixed-size) is too small and the conversion just bails out. In the case

Re: iconv errors and deurl.py

2009-03-31 Thread Guenter Milde
On 2009-03-27, John McCabe-Dansted wrote: > On Fri, Mar 27, 2009 at 8:43 PM, Guenter Milde wrote: >> Setting Document>Settings>Language>Encoding to UTF8 (ucs enhanced/utf8x) >> solves this particular problem. > Would this be a sensible default? No, because a) it does not solve all cases b) the u

Re: iconv errors and deurl.py

2009-03-27 Thread John McCabe-Dansted
>> You had to be difficult didn't you! :P > > No, just generic. While this might be a quick fix for you, there are just > too many unicode characters that are regularely used by users around the > world to make this a LyX internal. For the record, the proposal was to put this s

Re: iconv errors and deurl.py

2009-03-27 Thread Guenter Milde
On 2009-03-27, John McCabe-Dansted wrote: > On Thu, Mar 26, 2009 at 5:44 PM, Guenter Milde wrote: >> On 2009-03-26, John McCabe-Dansted wrote: > Well, I agree that this isn't a real solution, but for the sake of > argument: >>> I get iconv errors quite regularly. I

Re: iconv errors and deurl.py

2009-03-27 Thread Guenter Milde
On 2009-03-27, John McCabe-Dansted wrote: > On Thu, Mar 26, 2009 at 6:54 PM, Jean-Marc Lasgouttes > wrote: >> John McCabe-Dansted writes: >>> In the long run I don't think users should be expected to spend time >>> manually fixing iconv errors. >> No, b

Re: iconv errors and deurl.py

2009-03-27 Thread John McCabe-Dansted
Well, I agree that this isn't a real solution, but for the sake of argument: On Thu, Mar 26, 2009 at 5:44 PM, Guenter Milde wrote: > On 2009-03-26, John McCabe-Dansted wrote: >> I get iconv errors quite regularly. I have written a python script >> (filter) to remove all UTF

Re: iconv errors and deurl.py

2009-03-27 Thread John McCabe-Dansted
On Thu, Mar 26, 2009 at 6:54 PM, Jean-Marc Lasgouttes wrote: > John McCabe-Dansted writes: >> In the long run I don't think users should be expected to spend time >> manually fixing iconv errors. At worst LyX could pop up a dialog box: >> "LyX cannot handle the f

Re: iconv errors and deurl.py

2009-03-26 Thread Jean-Marc Lasgouttes
John McCabe-Dansted writes: > In the long run I don't think users should be expected to spend time > manually fixing iconv errors. At worst LyX could pop up a dialog box: > "LyX cannot handle the following characters ... [Remove] [Replace] > [Cancel]". No, but they sh

Re: iconv errors and deurl.py

2009-03-26 Thread Guenter Milde
On 2009-03-26, John McCabe-Dansted wrote: > I get iconv errors quite regularly. I have written a python script > (filter) to remove all UTF characters. This is sufficient for my > purposes since I hardly ever use UTF However, this is no solution for us umlaut-, accent-, Greek-, and

iconv errors and deurl.py

2009-03-26 Thread John McCabe-Dansted
I get iconv errors quite regularly. I have written a python script (filter) to remove all UTF characters. This is sufficient for my purposes since I hardly ever use UTF, and I can always use diff to see if there are any UTF characters I want to preserve. I find it much faster than manually

  1   2   >