Stephan Witt wrote:
> > So I propose to profile your current code and see whether there are
> > things that seem wrong (typing in a paragraph that is very long, for
> > example). And go for the more complicated version only if the profiler
> > shows something.
>
> You were right that long paragraphs slows down the editing in a unacceptable
> manner (I didn't need the profiler to state that).
> So I had to go the way with the "(2) more complicated thing".
> I implemented the real "spell check as you type" feature now.
FYI this is the last thing before we can release alpha 6,
so if somebody having understanding of the spellcheck area
can give it 10 mins review, we can move on.
pavel
>
> On 07.09.2010 at 17:41 I wrote:
>
> > In general I'd like to save CPU cycles - if possible.
> > So the current approach to check the spelling of the displayed again and
> > again is something I normally would avoid.
> > But I have to admit that it's difficult to do the thing right. Another very
> > important goal. :-)
> >
> > I tried to cache the results of the spell checker (as positions) and
> > succeeded. (It doesn't matter here if in font or paragraph.)
> > Now the problem is to invalidate the cache when it is appropriate. And that
> > would be the problem with every cache implementation here.
> >
> > Imagine the user switches the spellchecker or the buffer language or adds a
> > word to the personal dictionary.
> > All these operations effectively invalidate the position cache.
> > I have only a small problem to implement the invalidation for the cases
> > with constant spell checker engine.
> > But the switchable engine feature I can only address with a global state
> > variable...
> > either in the LyX singleton or as static POD.
>
> Furthermore this patch solves the issues with switching the speller engine
> and changing the personal dictionary.
> See attached long patch.
>
> The case of the change of the buffer language is no problem since it affects
> new entered text only.
>
> ===========
>
> Nevertheless I used the profiler to verify the possible solution with
> the word-wise check plus caching the check results.
> (The paragraph changes to cache the result positions at paragraph level
> included.)
>
> The use-case is:
> * Enable "Continuously spell check"
> * Open Tutorial and scroll down to the end, do the same with Users Guide
>
> The profiler numbers for AppleSpellChecker::check() sum up to:
> * Word-wise+word cache: 40% (cache cost is 23%) CPU
> * Paragraph-wise: 6.2% CPU
>
> The cache implementation I attach as diff between two patches.
> Perhaps there is a better implementation - I don't know.
>
> > What about replacing each character coming from the inset output by a
> > discretionary hyphen (0x00ad)? Will the spellchecker correctly ignore
> > it? In this case, this is a nice way of having strings of the same size.
>
> I cannot answer the question what happens with discretionary hyphens.
> The same problem exists for ligature breaks and possibly other cases.
> I have no idea how to solve that for these cases and will stick with my
> "softbreaks" solution for now.
>
> Stephan
>
> P.S. With enabled change tracking you cannot see spell errors. That's
> the case already now - without any patch applied. Maybe the blue line
> and the dotted red line overlap... and the blue line is the winner.
>
> Index: src/EnchantChecker.cpp
> ===================================================================
> --- src/EnchantChecker.cpp (Revision 35325)
> +++ src/EnchantChecker.cpp (Arbeitskopie)
> @@ -124,19 +124,29 @@
> }
>
>
> +void EnchantChecker::advanceChangeNumber()
> +{
> + nextChangeNumber();
> +}
> +
> +
> void EnchantChecker::insert(WordLangTuple const & word)
> {
> Spellers::iterator it = d->spellers_.find(word.lang()->code());
> - if (it != d->spellers_.end())
> + if (it != d->spellers_.end()) {
> it->second.speller->add(to_utf8(word.word()));
> + advanceChangeNumber();
> + }
> }
>
>
> void EnchantChecker::accept(WordLangTuple const & word)
> {
> Spellers::iterator it = d->spellers_.find(word.lang()->code());
> - if (it != d->spellers_.end())
> + if (it != d->spellers_.end()) {
> it->second.speller->add_to_session(to_utf8(word.word()));
> + advanceChangeNumber();
> + }
> }
>
>
> Index: src/HunspellChecker.cpp
> ===================================================================
> --- src/HunspellChecker.cpp (Revision 35325)
> +++ src/HunspellChecker.cpp (Arbeitskopie)
> @@ -233,6 +233,12 @@
> }
>
>
> +void HunspellChecker::advanceChangeNumber()
> +{
> + nextChangeNumber();
> +}
> +
> +
> void HunspellChecker::insert(WordLangTuple const & wl)
> {
> string const word_to_check = to_utf8(wl.word());
> @@ -240,12 +246,14 @@
> if (!h)
> return;
> h->add(word_to_check.c_str());
> + advanceChangeNumber();
> }
>
>
> void HunspellChecker::accept(WordLangTuple const & wl)
> {
> d->ignored_.push_back(wl);
> + advanceChangeNumber();
> }
>
>
> Index: src/AppleSpellChecker.cpp
> ===================================================================
> --- src/AppleSpellChecker.cpp (Revision 35325)
> +++ src/AppleSpellChecker.cpp (Arbeitskopie)
> @@ -88,12 +88,19 @@
> }
>
>
> +void AppleSpellChecker::advanceChangeNumber()
> +{
> + nextChangeNumber();
> +}
> +
> +
> // add to personal dictionary
> void AppleSpellChecker::insert(WordLangTuple const & word)
> {
> string const word_str = to_utf8(word.word());
> AppleSpeller_learn(d->speller, word_str.c_str());
> LYXERR(Debug::GUI, "learn word: \"" << word.word() << "\"") ;
> + advanceChangeNumber();
> }
>
>
> @@ -103,6 +110,7 @@
> string const word_str = to_utf8(word.word());
> AppleSpeller_unlearn(d->speller, word_str.c_str());
> LYXERR(Debug::GUI, "unlearn word: \"" << word.word() << "\"") ;
> + advanceChangeNumber();
> }
>
>
> @@ -111,6 +119,7 @@
> {
> string const word_str = to_utf8(word.word());
> AppleSpeller_ignore(d->speller, word_str.c_str());
> + advanceChangeNumber();
> }
>
>
> Index: src/Text2.cpp
> ===================================================================
> --- src/Text2.cpp (Revision 35325)
> +++ src/Text2.cpp (Arbeitskopie)
> @@ -356,6 +356,9 @@
> Font f = tm.displayFont(pit, pos);
> f.update(font, language, toggleall);
> setCharFont(pit, pos, f, tm.font_);
> + // font change may change language...
> + // spell checker has to know that
> + pars_[pit].requestSpellCheck(pos);
> }
> }
>
> Index: src/LyX.cpp
> ===================================================================
> --- src/LyX.cpp (Revision 35325)
> +++ src/LyX.cpp (Arbeitskopie)
> @@ -1354,39 +1354,48 @@
>
> void setSpellChecker()
> {
> + SpellChecker::ChangeNumber speller_change_number
> =singleton_->pimpl_->spell_checker_ ?
> + singleton_->pimpl_->spell_checker_->changeNumber() : 0;
> +
> + if (lyxrc.spellchecker == "native") {
> #if defined(USE_MACOSX_PACKAGING)
> - if (lyxrc.spellchecker == "native") {
> if (!singleton_->pimpl_->apple_spell_checker_)
> singleton_->pimpl_->apple_spell_checker_ = new
> AppleSpellChecker();
> singleton_->pimpl_->spell_checker_ =
> singleton_->pimpl_->apple_spell_checker_;
> - return;
> - }
> +#else
> + singleton_->pimpl_->spell_checker_ = 0;
> #endif
> + } else if (lyxrc.spellchecker == "aspell") {
> #if defined(USE_ASPELL)
> - if (lyxrc.spellchecker == "aspell") {
> if (!singleton_->pimpl_->aspell_checker_)
> singleton_->pimpl_->aspell_checker_ = new
> AspellChecker();
> singleton_->pimpl_->spell_checker_ =
> singleton_->pimpl_->aspell_checker_;
> - return;
> - }
> +#else
> + singleton_->pimpl_->spell_checker_ = 0;
> #endif
> + } else if (lyxrc.spellchecker == "enchant") {
> #if defined(USE_ENCHANT)
> - if (lyxrc.spellchecker == "enchant") {
> if (!singleton_->pimpl_->enchant_checker_)
> singleton_->pimpl_->enchant_checker_ = new
> EnchantChecker();
> singleton_->pimpl_->spell_checker_ =
> singleton_->pimpl_->enchant_checker_;
> - return;
> - }
> +#else
> + singleton_->pimpl_->spell_checker_ = 0;
> #endif
> + } else if (lyxrc.spellchecker == "hunspell") {
> #if defined(USE_HUNSPELL)
> - if (lyxrc.spellchecker == "hunspell") {
> if (!singleton_->pimpl_->hunspell_checker_)
> singleton_->pimpl_->hunspell_checker_ = new
> HunspellChecker();
> singleton_->pimpl_->spell_checker_ =
> singleton_->pimpl_->hunspell_checker_;
> - return;
> +#else
> + singleton_->pimpl_->spell_checker_ = 0;
> +#endif
> + } else {
> + singleton_->pimpl_->spell_checker_ = 0;
> }
> -#endif
> - singleton_->pimpl_->spell_checker_ = 0;
> + if (singleton_->pimpl_->spell_checker_) {
> +
> singleton_->pimpl_->spell_checker_->changeNumber(speller_change_number);
> + singleton_->pimpl_->spell_checker_->advanceChangeNumber();
> + }
> }
>
> } // namespace lyx
> Index: src/AppleSpellChecker.h
> ===================================================================
> --- src/AppleSpellChecker.h (Revision 35325)
> +++ src/AppleSpellChecker.h (Arbeitskopie)
> @@ -34,6 +34,7 @@
> int numMisspelledWords() const;
> void misspelledWord(int index, int & start, int & length) const;
> docstring const error();
> + void advanceChangeNumber();
> //@}
>
> private:
> Index: src/Font.cpp
> ===================================================================
> --- src/Font.cpp (Revision 35325)
> +++ src/Font.cpp (Arbeitskopie)
> @@ -95,7 +95,7 @@
>
>
> Font::Font(FontInfo bits, Language const * l)
> - : bits_(bits), lang_(l), misspelled_(false), open_encoding_(false)
> + : bits_(bits), lang_(l), open_encoding_(false)
> {
> if (!lang_)
> lang_ = default_language;
> Index: src/TextMetrics.cpp
> ===================================================================
> --- src/TextMetrics.cpp (Revision 35325)
> +++ src/TextMetrics.cpp (Arbeitskopie)
> @@ -2132,15 +2132,7 @@
>
> // Take this opportunity to spellcheck the row contents.
> if (row_has_changed && lyxrc.spellcheck_continuously) {
> - WordLangTuple wl;
> - // dummy variable, not used.
> - static docstring_list suggestions;
> - pos_type from = row.pos();
> - pos_type to = row.endpos();
> - while (from < row.endpos()) {
> - text_->getPar(pit).spellCheck(from, to, wl,
> suggestions, false);
> - from = to + 1;
> - }
> + text_->getPar(pit).spellCheck();
> }
>
> // Don't paint the row if a full repaint has not been requested
> Index: src/Text.cpp
> ===================================================================
> --- src/Text.cpp (Revision 35325)
> +++ src/Text.cpp (Arbeitskopie)
> @@ -519,6 +519,9 @@
>
> // Initialize begin_of_body_ on load; redoParagraph maintains
> par.setBeginOfBody();
> +
> + // mark paragraph for spell checking on load
> + // par.requestSpellCheck();
> }
>
>
> Index: src/EnchantChecker.h
> ===================================================================
> --- src/EnchantChecker.h (Revision 35325)
> +++ src/EnchantChecker.h (Arbeitskopie)
> @@ -38,6 +38,7 @@
> void accept(WordLangTuple const &);
> bool hasDictionary(Language const * lang) const;
> docstring const error();
> + void advanceChangeNumber();
> ///@}
>
> private:
> Index: src/Font.h
> ===================================================================
> --- src/Font.h (Revision 35325)
> +++ src/Font.h (Arbeitskopie)
> @@ -42,10 +42,6 @@
> ///
> Language const * language() const { return lang_; }
> ///
> - void setMisspelled(bool misspelled) { misspelled_ = misspelled; }
> - ///
> - bool isMisspelled() const { return misspelled_; }
> - ///
> bool isRightToLeft() const;
> ///
> bool isVisibleRightToLeft() const;
> @@ -116,9 +112,6 @@
> FontInfo bits_;
> ///
> Language const * lang_;
> - ///
> - bool misspelled_;
> -
> /// Did latexWriteStartChanges open an encoding environment?
> mutable bool open_encoding_;
> };
> @@ -128,8 +121,7 @@
> inline
> bool operator==(Font const & font1, Font const & font2)
> {
> - return font1.bits_ == font2.bits_ && font1.lang_ == font2.lang_
> - && font1.misspelled_ == font2.misspelled_;
> + return font1.bits_ == font2.bits_ && font1.lang_ == font2.lang_;
> }
>
> ///
> Index: src/frontends/qt4/Menus.cpp
> ===================================================================
> --- src/frontends/qt4/Menus.cpp (Revision 35325)
> +++ src/frontends/qt4/Menus.cpp (Arbeitskopie)
> @@ -738,7 +738,7 @@
> pos_type from = bv->cursor().pos();
> pos_type to = from;
> Paragraph const & par = bv->cursor().paragraph();
> - SpellChecker::Result res = par.spellCheck(from, to, wl, suggestions);
> + SpellChecker::Result res = par.spellCheck(from, to, wl, suggestions,
> true, true);
> switch (res) {
> case SpellChecker::UNKNOWN_WORD:
> if (lyxrc.spellcheck_continuously) {
> Index: src/FontList.cpp
> ===================================================================
> --- src/FontList.cpp (Revision 35325)
> +++ src/FontList.cpp (Arbeitskopie)
> @@ -181,20 +181,6 @@
> }
>
>
> -void FontList::setMisspelled(pos_type startpos, pos_type endpos,
> - bool misspelled)
> -{
> - // FIXME: move misspelled state out of font!?
> - for (pos_type p = startpos; p <= endpos; ++p) {
> - Font f = fontIterator(p)->font();
> - if (f.isMisspelled() != misspelled) {
> - f.setMisspelled(misspelled);
> - set(p, f);
> - }
> - }
> -}
> -
> -
> FontSize FontList::highestInRange(pos_type startpos, pos_type endpos,
> FontSize def_size) const
> {
> Index: src/Paragraph.cpp
> ===================================================================
> --- src/Paragraph.cpp (Revision 35325)
> +++ src/Paragraph.cpp (Arbeitskopie)
> @@ -73,8 +73,142 @@
> char_type const META_INSET = 0x200001;
> };
>
> +
> /////////////////////////////////////////////////////////////////////
> //
> +// SpellCheckerState
> +//
> +/////////////////////////////////////////////////////////////////////
> +
> +class SpellCheckerState {
> +public:
> + SpellCheckerState() {
> + needs_refresh_ = true;
> + current_change_number_ = 0;
> + }
> +
> + void setRange(FontSpan const fp, SpellChecker::Result state)
> + {
> + eraseCoveredRanges(fp);
> + if (state != SpellChecker::WORD_OK)
> + ranges_[fp] = state;
> + }
> +
> + void increasePosAfterPos(pos_type pos)
> + {
> + correctRangesAfterPos(pos, 1);
> + needsRefresh(pos);
> + }
> +
> + void decreasePosAfterPos(pos_type pos)
> + {
> + correctRangesAfterPos(pos, -1);
> + needsRefresh(pos);
> + }
> +
> + SpellChecker::Result getState(pos_type pos) const
> + {
> + SpellChecker::Result result = SpellChecker::WORD_OK;
> + RangesIterator et = ranges_.end();
> + RangesIterator it = ranges_.begin();
> + for (; it != et; ++it) {
> + FontSpan fc = it->first;
> + if(fc.first <= pos && pos <= fc.last) {
> + result = it->second;
> + break;
> + }
> + }
> + return result;
> + }
> +
> + bool needsRefresh() const {
> + return needs_refresh_;
> + }
> +
> + SpellChecker::ChangeNumber currentChangeNumber() const {
> + return current_change_number_;
> + }
> +
> + void refreshRange(pos_type & first, pos_type & last) const {
> + first = refresh_.first;
> + last = refresh_.last;
> + }
> +
> + void needsRefresh(pos_type pos) {
> + if (needs_refresh_ && pos != -1) {
> + if (pos < refresh_.first)
> + refresh_.first = pos;
> + if (pos > refresh_.last)
> + refresh_.last = pos;
> + } else if (pos != -1) {
> + refresh_.first = pos;
> + refresh_.last = pos;
> + }
> + needs_refresh_ = pos != -1;
> + }
> +
> + void needsCompleteRefresh(SpellChecker::ChangeNumber change_number) {
> + needs_refresh_ = true;
> + refresh_.first = 0;
> + refresh_.last = -1;
> + current_change_number_ = change_number;
> + }
> +
> +private:
> + /// store the ranges as map of FontSpan and spell result pairs
> + typedef map<FontSpan, SpellChecker::Result> Ranges;
> + typedef Ranges::const_iterator RangesIterator;
> + Ranges ranges_;
> + ///
> + FontSpan refresh_;
> + bool needs_refresh_;
> + SpellChecker::ChangeNumber current_change_number_;
> +
> + void eraseCoveredRanges(FontSpan const fp)
> + {
> + Ranges result;
> + RangesIterator et = ranges_.end();
> + RangesIterator it = ranges_.begin();
> + for (; it != et; ++it) {
> + FontSpan fc = it->first;
> + // 1. first of new range inside current range or
> + // 2. last of new range inside current range or
> + // 3. first of current range inside new range or
> + // 4. last of current range inside new range or
> + if ((fc.first <= fp.first && fp.first <= fc.last) ||
> + (fc.first <= fp.last && fp.last <= fc.last) ||
> + (fp.first <= fc.first && fc.first <= fp.last) ||
> + (fp.first <= fc.last && fc.last <= fp.last))
> + {
> + continue;
> + }
> + result[fc] = it->second;
> + }
> + ranges_ = result;
> + }
> +
> + void correctRangesAfterPos(pos_type pos, int offset)
> + {
> + Ranges result;
> + RangesIterator et = ranges_.end();
> + RangesIterator it = ranges_.begin();
> + for (; it != et; ++it) {
> + FontSpan m = it->first;
> + if (m.first > pos) {
> + m.first += offset;
> + m.last += offset;
> + } else if (m.last > pos) {
> + m.last += offset;
> + }
> + result[m] = it->second;
> + }
> + ranges_ = result;
> + }
> +
> +};
> +
> +/////////////////////////////////////////////////////////////////////
> +//
> // Paragraph::Private
> //
> /////////////////////////////////////////////////////////////////////
> @@ -173,18 +307,69 @@
> /// match a string against a particular point in the paragraph
> bool isTextAt(string const & str, pos_type pos) const;
>
> - void setMisspelled(pos_type from, pos_type to, bool misspelled)
> + /// a vector of inset positions
> + typedef vector<pos_type> Positions;
> + typedef Positions::const_iterator PositionsIterator;
> +
> + Language * getSpellLanguage(pos_type const from) const;
> +
> + Language * locateSpellRange(pos_type & from, pos_type & to,
> + Positions &
> softbreaks) const;
> +
> + bool hasSpellerChange() const {
> + SpellChecker::ChangeNumber speller_change_number = 0;
> + if (theSpellChecker())
> + speller_change_number =
> theSpellChecker()->changeNumber();
> + return speller_change_number >
> speller_state_.currentChangeNumber();
> + }
> +
> + void setMisspelled(pos_type from, pos_type to, SpellChecker::Result
> state)
> {
> pos_type textsize = owner_->size();
> // check for sane arguments
> - if (to < from || from >= textsize) return;
> + if (to < from || from >= textsize)
> + return;
> + FontSpan fp = FontSpan(from, to);
> // don't mark end of paragraph
> - if (to >= textsize)
> - to = textsize - 1;
> - fontlist_.setMisspelled(from, to, misspelled);
> + if (fp.last >= textsize)
> + fp.last = textsize - 1;
> + speller_state_.setRange(fp, state);
> }
> +
> + void requestSpellCheck(pos_type pos) {
> + speller_state_.needsRefresh(pos);
> + }
>
> -
> + void readySpellCheck() {
> + speller_state_.needsRefresh(-1);
> + }
> +
> + bool needsSpellCheck() const
> + {
> + return speller_state_.needsRefresh();
> + }
> +
> + void rangeOfSpellCheck(pos_type & first, pos_type & last) const
> + {
> + speller_state_.refreshRange(first, last);
> + if (last == -1) {
> + last = owner_->size();
> + return;
> + }
> + pos_type endpos = last;
> + owner_->locateWord(first, endpos, WHOLE_WORD);
> + if (endpos < last) {
> + endpos = last;
> + owner_->locateWord(last, endpos, WHOLE_WORD);
> + }
> + last = endpos;
> + }
> +
> + void markMisspelledWords(pos_type const & first, pos_type const & last,
> + SpellChecker::Result
> result,
> + docstring const & word,
> + Positions const &
> softbreaks);
> +
> InsetCode ownerCode() const
> {
> return inset_owner_ ? inset_owner_->lyxCode() : NO_CODE;
> @@ -224,6 +409,8 @@
> LangWordsMap words_;
> ///
> Layout const * layout_;
> + ///
> + SpellCheckerState speller_state_;
> };
>
>
> @@ -265,9 +452,10 @@
> : owner_(owner), inset_owner_(p.inset_owner_), fontlist_(p.fontlist_),
> params_(p.params_), changes_(p.changes_), insetlist_(p.insetlist_),
> begin_of_body_(p.begin_of_body_), text_(p.text_), words_(p.words_),
> - layout_(p.layout_)
> + layout_(p.layout_), speller_state_(p.speller_state_)
> {
> id_ = ++paragraph_id;
> + requestSpellCheck(p.text_.size());
> }
>
>
> @@ -277,7 +465,7 @@
> params_(p.params_), changes_(p.changes_),
> insetlist_(p.insetlist_, beg, end),
> begin_of_body_(p.begin_of_body_), words_(p.words_),
> - layout_(p.layout_)
> + layout_(p.layout_), speller_state_(p.speller_state_)
> {
> id_ = ++paragraph_id;
> if (beg >= pos_type(p.text_.size()))
> @@ -297,6 +485,7 @@
> // Add a new entry in the fontlist_.
> fontlist_.set(fcit->pos() - beg, fcit->font());
> }
> + requestSpellCheck(p.text_.size());
> }
>
>
> @@ -464,6 +653,8 @@
> if (pos == pos_type(text_.size())) {
> // when appending characters, no need to update tables
> text_.push_back(c);
> + // but we want spell checking
> + requestSpellCheck(pos);
> return;
> }
>
> @@ -474,6 +665,9 @@
>
> // Update the insets
> insetlist_.increasePosAfterPos(pos);
> +
> + // Update list of misspelled positions
> + speller_state_.increasePosAfterPos(pos);
> }
>
>
> @@ -493,6 +687,9 @@
>
> // Add a new entry in the insetlist_.
> d->insetlist_.insert(inset, pos);
> +
> + // Some insets require run of spell checker
> + requestSpellCheck(pos);
> return true;
> }
>
> @@ -542,6 +739,9 @@
> // Update the insetlist_
> d->insetlist_.decreasePosAfterPos(pos);
>
> + // Update list of misspelled positions
> + d->speller_state_.decreasePosAfterPos(pos);
> +
> return true;
> }
>
> @@ -1249,9 +1449,8 @@
> if (i == size())
> break;
>
> - // Write font changes (ignore spelling markers)
> + // Write font changes
> Font font2 = getFontSettings(bparams, i);
> - font2.setMisspelled(false);
> if (font2 != font1) {
> flushString(os, write_buffer);
> font2.lyxWriteChanges(font1, os);
> @@ -2605,6 +2804,7 @@
> setFont(i, font);
> }
> }
> + d->requestSpellCheck(size());
> }
>
>
> @@ -3172,61 +3372,223 @@
> }
>
>
> -SpellChecker::Result Paragraph::spellCheck(pos_type & from, pos_type & to,
> WordLangTuple & wl,
> - docstring_list & suggestions, bool do_suggestion) const
> +Language * Paragraph::Private::locateSpellRange(
> + pos_type & from, pos_type & to,
> + Positions & softbreaks) const
> {
> + // skip leading white space
> + while (from < to && owner_->isWordSeparator(from))
> + ++from;
> + // don't check empty range
> + if (from >= to)
> + return 0;
> + // get current language
> + Language * lang = getSpellLanguage(from);
> + pos_type last = from;
> + bool samelang = true;
> + bool sameinset = true;
> + while (last < to && samelang && sameinset) {
> + // hop to end of word
> + while (last < to && !owner_->isWordSeparator(last)) {
> + if (owner_->getInset(last)) {
> + softbreaks.insert(softbreaks.end(), last);
> + }
> + ++last;
> + }
> + // hop to next word while checking for insets
> + while (sameinset && last < to && owner_->isWordSeparator(last))
> {
> + if (Inset const * inset = owner_->getInset(last))
> + sameinset = inset->isChar() &&
> inset->isLetter();
> + if (sameinset)
> + last++;
> + }
> + if (sameinset && last < to) {
> + // now check for language change
> + samelang = lang == getSpellLanguage(last);
> + }
> + }
> + // if language change detected backstep is needed
> + if (!samelang)
> + --last;
> + to = last;
> + return lang;
> +}
> +
> +
> +Language * Paragraph::Private::getSpellLanguage(pos_type const from) const
> +{
> + Language * lang =
> + const_cast<Language *>(owner_->getFontSettings(
> + inset_owner_->buffer().params(), from).language());
> + if (lang == inset_owner_->buffer().params().language
> + && !lyxrc.spellchecker_alt_lang.empty()) {
> + string lang_code;
> + string const lang_variety =
> + split(lyxrc.spellchecker_alt_lang, lang_code, '-');
> + lang->setCode(lang_code);
> + lang->setVariety(lang_variety);
> + }
> + return lang;
> +}
> +
> +
> +void Paragraph::requestSpellCheck(pos_type pos)
> +{
> + d->requestSpellCheck(pos == -1 ? size() : pos);
> +}
> +
> +
> +bool Paragraph::needsSpellCheck() const
> +{
> + SpellChecker::ChangeNumber speller_change_number = 0;
> + if (theSpellChecker())
> + speller_change_number = theSpellChecker()->changeNumber();
> + if (speller_change_number > d->speller_state_.currentChangeNumber()) {
> + d->speller_state_.needsCompleteRefresh(speller_change_number);
> + }
> + return d->needsSpellCheck();
> +}
> +
> +
> +SpellChecker::Result Paragraph::spellCheck(pos_type & from, pos_type & to,
> + WordLangTuple & wl, docstring_list & suggestions,
> + bool do_suggestion, bool check_learned) const
> +{
> + SpellChecker::Result result = SpellChecker::WORD_OK;
> SpellChecker * speller = theSpellChecker();
> if (!speller)
> - return SpellChecker::WORD_OK;
> + return result;
>
> if (!d->layout_->spellcheck || !inInset().allowSpellCheck())
> - return SpellChecker::WORD_OK;
> + return result;
>
> locateWord(from, to, WHOLE_WORD);
> if (from == to || from >= pos_type(d->text_.size()))
> - return SpellChecker::WORD_OK;
> + return result;
>
> docstring word = asString(from, to, AS_STR_INSETS);
> - // Ignore words with digits
> - // FIXME: make this customizable
> - // (note that hunspell ignores words with digits by default)
> - bool const ignored = hasDigit(word);
> - Language * lang = const_cast<Language *>(getFontSettings(
> - d->inset_owner_->buffer().params(), from).language());
> - if (lang == d->inset_owner_->buffer().params().language
> - && !lyxrc.spellchecker_alt_lang.empty()) {
> - string lang_code;
> - string const lang_variety =
> - split(lyxrc.spellchecker_alt_lang, lang_code, '-');
> - lang->setCode(lang_code);
> - lang->setVariety(lang_variety);
> - }
> + Language * lang = d->getSpellLanguage(from);
> +
> wl = WordLangTuple(word, lang);
> - SpellChecker::Result res = ignored ?
> - SpellChecker::WORD_OK : speller->check(wl);
> +
> + if (!word.size())
> + return result;
>
> - bool const misspelled_ = SpellChecker::misspelled(res) ;
> -
> - if (lyxrc.spellcheck_continuously)
> - d->setMisspelled(from, to, misspelled_);
> -
> + if (needsSpellCheck() || check_learned) {
> + // Ignore words with digits
> + // FIXME: make this customizable
> + // (note that some checkers ignore words with digits by default)
> + if (!hasDigit(word))
> + result = speller->check(wl);
> + d->setMisspelled(from, to, result);
> + } else {
> + result = d->speller_state_.getState(from);
> + }
> +
> + bool const misspelled_ = SpellChecker::misspelled(result) ;
> if (misspelled_ && do_suggestion)
> speller->suggest(wl, suggestions);
> + else if (misspelled_)
> + LYXERR(Debug::GUI, "misspelled word: \"" <<
> + word << "\" [" <<
> + from << ".." << to << "]");
> else
> suggestions.clear();
>
> - return res;
> + return result;
> }
>
>
> +void Paragraph::Private::markMisspelledWords(
> + pos_type const & first, pos_type const & last,
> + SpellChecker::Result result,
> + docstring const & word,
> + Positions const & softbreaks)
> +{
> + pos_type snext = first;
> + if (SpellChecker::misspelled(result)) {
> + SpellChecker * speller = theSpellChecker();
> + // locate and enumerate the error positions
> + int nerrors = speller->numMisspelledWords();
> + int numbreaks = 0;
> + PositionsIterator it = softbreaks.begin();
> + PositionsIterator et = softbreaks.end();
> + for (int index = 0; index < nerrors; ++index) {
> + int wstart;
> + int wlen = 0;
> + speller->misspelledWord(index, wstart, wlen);
> + if (wlen) {
> + docstring const misspelled =
> word.substr(wstart, wlen);
> + wstart += first + numbreaks;
> + if (snext < wstart) {
> + while (it != et && *it < wstart) {
> + ++wstart;
> + ++numbreaks;
> + ++it;
> + }
> + setMisspelled(snext,
> + wstart - 1,
> SpellChecker::WORD_OK);
> + }
> + snext = wstart + wlen;
> + while (it != et && *it < snext) {
> + ++snext;
> + ++numbreaks;
> + ++it;
> + }
> + setMisspelled(wstart, snext, result);
> + LYXERR(Debug::GUI, "misspelled word: \"" <<
> + misspelled << "\" [" <<
> + wstart << ".." << (snext-1) << "]");
> + ++snext;
> + }
> + }
> + }
> + if (snext <= last) {
> + setMisspelled(snext, last, SpellChecker::WORD_OK);
> + }
> +}
> +
> +
> +void Paragraph::spellCheck() const
> +{
> + SpellChecker * speller = theSpellChecker();
> + if (!speller || !size() ||!needsSpellCheck())
> + return;
> + pos_type start;
> + pos_type endpos;
> + d->rangeOfSpellCheck(start, endpos);
> + if (speller->canCheckParagraph()) {
> + // loop until we leave the range argument
> + for (pos_type first = start; first < endpos; ) {
> + pos_type last = endpos;
> + Private::Positions softbreaks;
> + Language * lang = d->locateSpellRange(first, last,
> softbreaks);
> + if (first >= endpos)
> + break;
> + // start the spell checker on the unit of meaning
> + docstring word = asString(first, last, AS_STR_INSETS);
> + WordLangTuple wl = WordLangTuple(word, lang);
> + SpellChecker::Result result = word.size() ?
> + speller->check(wl) : SpellChecker::WORD_OK;
> + d->markMisspelledWords(first, last, result, word,
> softbreaks);
> + first = ++last;
> + }
> + } else {
> + static docstring_list suggestions;
> + pos_type to = endpos;
> + while (start < endpos) {
> + WordLangTuple wl;
> + spellCheck(start, to, wl, suggestions, false);
> + start = to + 1;
> + }
> + }
> + d->readySpellCheck();
> +}
> +
> +
> bool Paragraph::isMisspelled(pos_type pos) const
> {
> - pos_type from = pos;
> - pos_type to = pos;
> - WordLangTuple wl;
> - docstring_list suggestions;
> - SpellChecker::Result res = spellCheck(from, to, wl, suggestions, false);
> - return SpellChecker::misspelled(res) ;
> + return SpellChecker::misspelled(d->speller_state_.getState(pos));
> }
>
>
> Index: src/AspellChecker.cpp
> ===================================================================
> --- src/AspellChecker.cpp (Revision 35325)
> +++ src/AspellChecker.cpp (Arbeitskopie)
> @@ -280,6 +280,12 @@
> }
>
>
> +void AspellChecker::advanceChangeNumber()
> +{
> + nextChangeNumber();
> +}
> +
> +
> void AspellChecker::insert(WordLangTuple const & word)
> {
> Spellers::iterator it = d->spellers_.find(
> @@ -287,6 +293,7 @@
> if (it != d->spellers_.end()) {
> AspellSpeller * speller =
> to_aspell_speller(it->second.e_speller);
> aspell_speller_add_to_personal(speller,
> to_utf8(word.word()).c_str(), -1);
> + advanceChangeNumber();
> }
> }
>
> @@ -298,6 +305,7 @@
> if (it != d->spellers_.end()) {
> AspellSpeller * speller =
> to_aspell_speller(it->second.e_speller);
> aspell_speller_add_to_session(speller,
> to_utf8(word.word()).c_str(), -1);
> + advanceChangeNumber();
> }
> }
>
> Index: src/rowpainter.cpp
> ===================================================================
> --- src/rowpainter.cpp (Revision 35325)
> +++ src/rowpainter.cpp (Arbeitskopie)
> @@ -236,6 +236,10 @@
> bool const selection = (pos >= row_.sel_beg && pos < row_.sel_end)
> || pi_.selected;
>
> + // spelling correct?
> + bool const spell_state =
> + lyxrc.spellcheck_continuously && par_.isMisspelled(pos);
> +
> char_type prev_char = ' ';
> // collect as much similar chars as we can
> for (++vpos ; vpos < end ; ++vpos) {
> @@ -248,6 +252,12 @@
> // Selection ends or starts here.
> break;
>
> + bool const new_spell_state =
> + lyxrc.spellcheck_continuously && par_.isMisspelled(pos);
> + if (new_spell_state != spell_state)
> + // Spell checker state changed here.
> + break;
> +
> Change const & change = par_.lookupChange(pos);
> if (!change_running.isSimilarTo(change))
> // Track change type or author has changed.
> @@ -359,6 +369,10 @@
> bool const arabic = lang == "arabic_arabtex" || lang == "arabic_arabi"
> ||
> lang == "farsi";
>
> + // spelling correct?
> + bool const misspelled_ =
> + lyxrc.spellcheck_continuously && par_.isMisspelled(pos);
> +
> // draw as many chars as we can
> if ((!hebrew && !arabic)
> || (hebrew && !Encodings::isHebrewComposeChar(c))
> @@ -372,8 +386,9 @@
>
> paintForeignMark(orig_x, orig_font.language());
>
> - if (lyxrc.spellcheck_continuously && orig_font.isMisspelled())
> + if (lyxrc.spellcheck_continuously && misspelled_) {
> paintMisspelledMark(orig_x, 2);
> + }
> }
>
>
> Index: src/FontList.h
> ===================================================================
> --- src/FontList.h (Revision 35325)
> +++ src/FontList.h (Arbeitskopie)
> @@ -105,12 +105,6 @@
> ///
> void decreasePosAfterPos(pos_type pos);
>
> - ///
> - void setMisspelled(
> - pos_type startpos,
> - pos_type endpos,
> - bool misspelled);
> -
> /// Returns the height of the highest font in range
> FontSize highestInRange(
> pos_type startpos,
> Index: src/Cursor.cpp
> ===================================================================
> --- src/Cursor.cpp (Revision 35325)
> +++ src/Cursor.cpp (Arbeitskopie)
> @@ -2265,9 +2265,7 @@
> // get font
> BufferParams const & bufparams = buffer()->params();
> current_font = par.getFontSettings(bufparams, cpos);
> - current_font.setMisspelled(false);
> real_current_font = tm.displayFont(cpit, cpos);
> - real_current_font.setMisspelled(false);
>
> // special case for paragraph end
> if (cs.pos() == lastpos()
> Index: src/SpellChecker.h
> ===================================================================
> --- src/SpellChecker.h (Revision 35325)
> +++ src/SpellChecker.h (Arbeitskopie)
> @@ -14,6 +14,7 @@
> #define SPELL_BASE_H
>
> #include "support/strfwd.h"
> +#include "support/lyxtime.h"
>
>
> namespace lyx {
> @@ -89,6 +90,16 @@
>
> /// give an error message on messy exit
> virtual docstring const error() = 0;
> +
> + /// spell checker state versioning support
> + typedef unsigned long int ChangeNumber ;
> + ChangeNumber changeNumber() const { return change_number_; }
> + void changeNumber(ChangeNumber value) { change_number_ = value; }
> + void nextChangeNumber() { ++change_number_; }
> + virtual void advanceChangeNumber() = 0;
> +
> +private:
> + ChangeNumber change_number_;
> };
>
> /// Access to the singleton SpellChecker.
> Index: src/Buffer.cpp
> ===================================================================
> --- src/Buffer.cpp (Revision 35325)
> +++ src/Buffer.cpp (Arbeitskopie)
> @@ -4033,6 +4033,7 @@
> if (from == end)
> break;
> to = from;
> + from.paragraph().spellCheck();
> SpellChecker::Result res =
> from.paragraph().spellCheck(from.pos(), to.pos(), wl, suggestions);
> if (SpellChecker::misspelled(res)) {
> word_lang = wl;
> Index: src/AspellChecker.h
> ===================================================================
> --- src/AspellChecker.h (Revision 35325)
> +++ src/AspellChecker.h (Arbeitskopie)
> @@ -32,6 +32,7 @@
> void accept(WordLangTuple const &);
> bool hasDictionary(Language const * lang) const;
> docstring const error();
> + void advanceChangeNumber();
> //@}
>
> private:
> Index: src/Paragraph.h
> ===================================================================
> --- src/Paragraph.h (Revision 35325)
> +++ src/Paragraph.h (Arbeitskopie)
> @@ -66,6 +66,18 @@
> public:
> /// Range including first and last.
> pos_type first, last;
> +
> + inline bool operator<(FontSpan const & s) const
> + {
> + return first < s.first;
> + }
> +
> + inline bool operator==(FontSpan const & s) const
> + {
> + return first == s.first && last == s.last;
> + }
> +
> +
> };
>
> ///
> @@ -147,7 +159,7 @@
> /// \param force means: output even if layout.inpreamble is true.
> void latex(BufferParams const &, Font const & outerfont, odocstream &,
> TexRow & texrow, OutputParams const &,
> - int start_pos = 0, int end_pos = -1, bool force =
> false) const;
> + int start_pos = 0, int end_pos = -1, bool force = false)
> const;
>
> /// Can we drop the standard paragraph wrapper?
> bool emptyTag() const;
> @@ -425,11 +437,23 @@
> /// and \p suggestions if \p do_suggestion is true.
> /// \return result from spell checker, SpellChecker::UNKNOWN_WORD when
> misspelled.
> SpellChecker::Result spellCheck(pos_type & from, pos_type & to,
> WordLangTuple & wl,
> - docstring_list & suggestions, bool do_suggestion = true) const;
> + docstring_list & suggestions, bool do_suggestion = true,
> + bool check_learned = false) const;
>
> - /// Spellcheck word at position \p pos.
> - /// \return true if pointed word is misspelled.
> + /// Spell checker status at position \p pos.
> + /// \return true if pointed position is misspelled.
> bool isMisspelled(pos_type pos) const;
> +
> + /// spell check of whole paragraph
> + /// remember results until call of requestSpellCheck()
> + void spellCheck() const;
> +
> + /// query state of spell checker results
> + bool needsSpellCheck() const;
> + /// mark position of text manipulation to inform the spell checker
> + /// default value -1 marks the whole paragraph to be checked (again)
> + void requestSpellCheck(pos_type pos = -1);
> +
> /// an automatically generated identifying label for this paragraph.
> /// presently used only in the XHTML output routines.
> std::string magicLabel() const;
> Index: src/HunspellChecker.h
> ===================================================================
> --- src/HunspellChecker.h (Revision 35325)
> +++ src/HunspellChecker.h (Arbeitskopie)
> @@ -32,6 +32,7 @@
> void accept(WordLangTuple const &);
> bool hasDictionary(Language const * lang) const;
> docstring const error();
> + void advanceChangeNumber();
> ///@}
>
> private:
>
> 30a31,77
> > Index: src/AppleSpellChecker.cpp
> > ===================================================================
> > --- src/AppleSpellChecker.cpp (Revision 35292)
> > +++ src/AppleSpellChecker.cpp (Arbeitskopie)
> > @@ -34,6 +34,12 @@
> >
> > /// the speller
> > AppleSpeller speller;
> > + // the cache
> > + typedef map<string, SpellCheckResult> ResultsMap;
> > + typedef ResultsMap::const_iterator ResultsIterator;
> > + typedef map<string, ResultsMap> LangResultsMap;
> > + typedef LangResultsMap::const_iterator LangResultsIterator;
> > + LangResultsMap results_;
> > };
> >
> >
> > @@ -78,9 +84,26 @@
> > SpellChecker::Result AppleSpellChecker::check(WordLangTuple const & word)
> > {
> > string const word_str = to_utf8(word.word());
> > - SpellCheckResult result =
> > - AppleSpeller_check(d->speller,
> > - word_str.c_str(), word.lang()->code().c_str());
> > + string const lang_str = word.lang()->code();
> > + SpellCheckResult result ;
> > + Private::LangResultsIterator lang_cache = d->results_.find(lang_str);
> > + if (lang_cache == d->results_.end()) {
> > + result = AppleSpeller_check(d->speller,
> > +
> > word_str.c_str(), lang_str.c_str());
> > + Private::ResultsMap map;
> > + map[word_str] = result;
> > + d->results_[lang_str] = map;
> > + } else {
> > + Private::ResultsIterator cached_result =
> > lang_cache->second.find(word_str);
> > + if (cached_result == lang_cache->second.end()) {
> > + result = AppleSpeller_check(d->speller,
> > +
> > word_str.c_str(), lang_str.c_str());
> > + Private::ResultsMap map = lang_cache->second;
> > + map[word_str] = result;
> > + } else {
> > + result = cached_result->second ;
> > + }
> > + }
> > LYXERR(Debug::GUI, "spellCheck: \"" <<
> > word.word() << "\" = " << d->toString(result) <<
> > ", lang = " << word.lang()->code()) ;
> 44a92,104
> > Index: src/AppleSpellChecker.h
> > ===================================================================
> > --- src/AppleSpellChecker.h (Revision 35292)
> > +++ src/AppleSpellChecker.h (Arbeitskopie)
> > @@ -30,7 +30,7 @@
> > void remove(WordLangTuple const &);
> > void accept(WordLangTuple const &);
> > bool hasDictionary(Language const * lang) const;
> > - bool canCheckParagraph() const { return true; }
> > + bool canCheckParagraph() const { return false; }
> > int numMisspelledWords() const;
> > void misspelledWord(int index, int & start, int & length) const;
> > docstring const error();
