Re: iconv warnings

2017-09-10 Thread Uwe Stöhr
El 10.09.2017 a las 14:18, Uwe Stöhr escribió: I just compiled the current 2.3.x branch to perform some final tests. I noted some avoidable compiler warnings in libiconv, see below I reported the warnings to the email address and got already a reply. They will analyze the signed/unsigned

iconv warnings

2017-09-10 Thread Uwe Stöhr
I just compiled the current 2.3.x branch to perform some final tests. I noted some avoidable compiler warnings in libiconv, see below. The question is if we should or could fix this? If we don't do this, does anybody know where I can or should report this? There is a github page: https://github

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jean-Marc Lasgouttes
11/04/2014 12:01, Stephan Witt: Is there a way to avoid this "feature"? I don't know. It's a black box. It's a OS service. Perhaps it can be configured somewhere, via System Preferences or API. With LyX you can use hunspell as the spell checker backend instead. Stephan It looks liike there

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Stephan Witt
Am 11.04.2014 um 11:56 schrieb Jean-Marc Lasgouttes : > 11/04/2014 11:23, Stephan Witt: >>> So do you mean that if I write in an English text "somme" instead >>> of "some", if will be considered an OK work because "somme" exists >>> in French? Is that supposed to be a feature? >> >> Indeed. Follo

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jean-Marc Lasgouttes
11/04/2014 11:23, Stephan Witt: So do you mean that if I write in an English text "somme" instead of "some", if will be considered an OK work because "somme" exists in French? Is that supposed to be a feature? Indeed. Following is the debug output of text input while instant-spellchecking is en

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Stephan Witt
Am 11.04.2014 um 10:23 schrieb Jean-Marc Lasgouttes : > 11/04/2014 08:23, Stephan Witt: >> Am 11.04.2014 um 01:36 schrieb Cyrille Artho : >> >>> I agree that it would be good to have all dictionaries in utf-8, but I'm >>> not sure if this is feasible for a typical user/installation. >>> >>> Ano

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jean-Marc Lasgouttes
11/04/2014 11:06, Cyrille Artho: I always thought that the French did this on purpose to not surrender to the fact that the English language dominates the world. Vincent They did this to the keyboard layout, too. If you ever tried to use a computer in a French Internet cafe, good luck typing y

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Cyrille Artho
I always thought that the French did this on purpose to not surrender to the fact that the English language dominates the world. Vincent They did this to the keyboard layout, too. If you ever tried to use a computer in a French Internet cafe, good luck typing your password (even your username

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jean-Marc Lasgouttes
11/04/2014 10:31, Vincent van Ravesteijn: Recently I did some proofreading of a paper written with TeXShop (LaTeX editor for Mac). It turned out that the text was peppered with french words. From what I understand, this horror was a joint work of automatic correction and automatic language detec

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Vincent van Ravesteijn
> So do you mean that if I write in an English text "somme" instead of "some", > if will be considered an OK work because "somme" exists in French? Is that > supposed to be a feature? > I guess that the guessing is done for the whole paragraph > Recently I did some proofreading of a paper wri

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jean-Marc Lasgouttes
11/04/2014 08:23, Stephan Witt: Am 11.04.2014 um 01:36 schrieb Cyrille Artho : I agree that it would be good to have all dictionaries in utf-8, but I'm not sure if this is feasible for a typical user/installation. Another option would be for LyX to tokenize the text and forward it word by wo

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jürgen Spitzmüller
2014-04-10 22:30 GMT+02:00 Stephan Witt: > Am 10.04.2014 um 20:43 schrieb Georg Baum: > > It is probably not difficult to implement sensible behaviour for > "ignore" > > and "ignore all" for these words: HunspellChecker has already a member > > variable ignored_ which tracks ignored words, so if

Re: [patch for 2.2] silence iconv warnings

2014-04-11 Thread Jürgen Spitzmüller
2014-04-10 20:43 GMT+02:00 Georg Baum: > Does this mean that we need to maintain our own versions? If not then it is > probably the best solution, if yes then I'd rather not do it. > The Chromium project does maintain utf8 versions (or "deltas", for that matter): http://www.chromium.org/developer

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Stephan Witt
Am 11.04.2014 um 01:36 schrieb Cyrille Artho : > I agree that it would be good to have all dictionaries in utf-8, but I'm not > sure if this is feasible for a typical user/installation. > > Another option would be for LyX to tokenize the text and forward it word by > word to the spell checker.

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Vincent van Ravesteijn
On Fri, Apr 11, 2014 at 1:40 AM, Cyrille Artho wrote: > Regarding the idea I just mentioned before, there is a major flaw. > > Asian languages do not have spaces. Tokenizing a text into words requires a > dictionary and is a non-trivial problem (due to inflection: different verb > forms need to be

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Cyrille Artho
Regarding the idea I just mentioned before, there is a major flaw. Asian languages do not have spaces. Tokenizing a text into words requires a dictionary and is a non-trivial problem (due to inflection: different verb forms need to be recognized, etc.). We can therefore not just scan for white

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Cyrille Artho
I agree that it would be good to have all dictionaries in utf-8, but I'm not sure if this is feasible for a typical user/installation. Another option would be for LyX to tokenize the text and forward it word by word to the spell checker. This way, we could handle "Ignore All" in LyX itself ra

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Stephan Witt
Am 10.04.2014 um 20:43 schrieb Georg Baum : > Jürgen Spitzmüller wrote: > >> The point is that users cannot do something sensible with such marked >> words (except for adding them into the personal dictionary). > > It is probably not difficult to implement sensible behaviour for "ignore" > and

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Georg Baum
Jürgen Spitzmüller wrote: > The point is that users cannot do something sensible with such marked > words (except for adding them into the personal dictionary). It is probably not difficult to implement sensible behaviour for "ignore" and "ignore all" for these words: HunspellChecker has already

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Jean-Marc Lasgouttes
10/04/2014 14:29, Jürgen Spitzmüller: No, if the encoding fits, I can hit "Ignore all" and only ignore you (or your name's spelling, for that matter) in the current document (which is what I do for names usually, except for very recurrent names). If the encoding does not fit, hitting "Ignore all"

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Jürgen Spitzmüller
2014-04-10 14:18 GMT+02:00 Jean-Marc Lasgouttes : > > The point is that users cannot do something sensible with such marked >> words (except for adding them into the personal dictionary). >> > > Sure, but the same holds for "Lasgouttes", doesn't it? > No, if the encoding fits, I can hit "Ignore a

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Jean-Marc Lasgouttes
10/04/2014 14:14, Jürgen Spitzmüller: There is something that I do not understand. If the word is not representable in the German dictionary, presumably it is not part of the language. "Lasgoittes" is perfectly representable in any latin encoding and yet the spell-checker will mar

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Jürgen Spitzmüller
2014-04-10 10:11 GMT+02:00 JeanMarc Lasgouttes : > There is something that I do not understand. If the word is not > representable in the German dictionary, presumably it is not part of the > language. "Lasgoittes" is perfectly representable in any latin encoding and > yet the spell-checker will m

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread JeanMarc Lasgouttes
There is something that I do not understand. If the word is not representable in the German dictionary, presumably it is not part of the language. "Lasgoittes" is perfectly representable in any latin encoding and yet the spell-checker will mark it as misspelled. Why should it be different for a

Re: [patch for 2.2] silence iconv warnings

2014-04-10 Thread Stephan Witt
Am 10.04.2014 um 08:57 schrieb Cyrille Artho : > How is the call to iconv implemented? On the application level, the interface > is probably not flexible enough; it is easy to ignore non-convertible > characters, but they are just removed from the output. > > The C library interface is probably

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Cyrille Artho
How is the call to iconv implemented? On the application level, the interface is probably not flexible enough; it is easy to ignore non-convertible characters, but they are just removed from the output. The C library interface is probably richer. Is it possible to convert text word by word, fi

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Jürgen Spitzmüller
2014-04-10 1:02 GMT+02:00 Cyrille Artho : > I think we have to use Unicode for all the given operations and (a) either > risk a mismatch for each word that is not learned/ignored, or (b) > up-convert words in the dictionary before they are matched. The latter > solution implies that the dictionary

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Cyrille Artho
Usually given names are not in a language dictionary, although many (translation) services have separate dictionaries for proper/given names. We have two problems here: (1) Language: I think most users are OK with proper names not being accepted by the spell checker (before learning them). How

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Stephan Witt
Am 09.04.2014 um 11:12 schrieb Jürgen Spitzmüller : > 2014-04-09 10:59 GMT+02:00 Stephan Witt : > > That's a good example. So, my parents are from Hungarian and named me István. > Let's assume the á isn't valid in german iso encoding. Then > * I can change my name to Stephan - to avoid to spell m

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Georg Baum
Jürgen Spitzmüller wrote: > 2014-04-09 9:52 GMT+02:00 Jean-Marc Lasgouttes : > >> I think the lyxerr message could be rewritten to be at least useful. Who >> found a use for the use hex dump anyways? >> > > Agreed. Me too. I think this output is still unchanged from the time when we had bugs i

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Jürgen Spitzmüller
2014-04-09 10:59 GMT+02:00 Stephan Witt : > > That's a good example. So, my parents are from Hungarian and named me > István. > Let's assume the á isn't valid in german iso encoding. Then > * I can change my name to Stephan - to avoid to spell my name on every > formal occasion > * if I don't like

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Stephan Witt
Am 09.04.2014 um 08:53 schrieb Jürgen Spitzmüller : > 2014-04-09 8:40 GMT+02:00 Stephan Witt : > Can you provide an example, please? If the word cannot be converted to > Hunspell dictionary encoding > the dictionary is broken or the language is not correct, isn't it? > > Depends on how you defin

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Jürgen Spitzmüller
2014-04-09 9:52 GMT+02:00 Jean-Marc Lasgouttes : > I think the lyxerr message could be rewritten to be at least useful. Who > found a use for the use hex dump anyways? > Agreed. Jürgen > > JMarc > >

Re: [patch for 2.2] silence iconv warnings

2014-04-09 Thread Jean-Marc Lasgouttes
09/04/2014 08:14, Jürgen Spitzmüller: 2014-04-08 22:42 GMT+02:00 Georg Baum: The change in src/support/unicode.cpp is problematic: It disables all error handling, not only the lyxerr output. Also, if you now throw an exception there, you need to make sure that all callers can

Re: [patch for 2.2] silence iconv warnings

2014-04-08 Thread Jürgen Spitzmüller
2014-04-09 8:40 GMT+02:00 Stephan Witt : > Can you provide an example, please? If the word cannot be converted to > Hunspell dictionary encoding > the dictionary is broken or the language is not correct, isn't it? > Depends on how you define "language". Think of names. > You're right, the user

Re: [patch for 2.2] silence iconv warnings

2014-04-08 Thread Stephan Witt
Am 09.04.2014 um 08:14 schrieb Jürgen Spitzmüller : > 2014-04-08 22:42 GMT+02:00 Georg Baum: > The change in src/support/unicode.cpp is problematic: It disables all error > handling, not only the lyxerr output. Also, if you now throw an exception > there, you need to make sure that all callers ca

Re: [patch for 2.2] silence iconv warnings

2014-04-08 Thread Jürgen Spitzmüller
2014-04-08 22:42 GMT+02:00 Georg Baum: > The change in src/support/unicode.cpp is problematic: It disables all error > handling, not only the lyxerr output. Also, if you now throw an exception > there, you need to make sure that all callers can cope with that. Maybe one > solution would be to thro

Re: [patch for 2.2] silence iconv warnings

2014-04-08 Thread Georg Baum
Jürgen Spitzmüller wrote: > I am not very familiar with iconv/unicode and exception handling, so I > would appreciate a critical review. The change in src/support/unicode.cpp is problematic: It disables all error handling, not only the lyxerr output. Also, if you now throw an exception there, y

[patch for 2.2] silence iconv warnings

2014-04-06 Thread Jürgen Spitzmüller
When scrolling through a document while instant-spellchecking is enabled and Hunspell used, LyX spits out iconv errors if a word appears which is not in the Hunspell dictionary's encoding. E.g.: Error returned from iconv EILSEQ An invalid multibyte sequence has been encountered in the input. When