[Libreoffice-commits] core.git: sw/inc sw/source

[László Németh (via logerrit)]

 sw/inc/shellio.hxx                |    8 ++++++++
 sw/source/filter/ascii/parasc.cxx |   11 +++++++++++
 2 files changed, 19 insertions(+)

New commits:
commit 13e34393e4564ef67d990c6dbe1991a0a6b288dd
Author:     László Németh <nem...@numbertext.org>
AuthorDate: Thu Mar 2 11:50:17 2023 +0100
Commit:     László Németh <nem...@numbertext.org>
CommitDate: Mon Mar 6 09:38:30 2023 +0000

    tdf#154000 tdf#70423 sw: fix crash/freezing with huge text files
    
    It was not possible to load or edit text files with very long
    lines any more, because removed wrapping of the lines resulted
    freezing and crash.
    
    This reverts commit 745898eb2af2686ffbdfdc0e44984db67b172a59
    "tdf#70423 Remove txtimport break in 10k chars line",
    keeping only its unit test.
    
    Fix also tdf#70423 by increasing the wrap limit from 10000
    to 250000 characters, which still permit to load e.g. 50 MB
    character or longer text lines, and to edit 5 MB or longer
    text lines without crash or freezing.
    
    Details: trying to load a relatively short 76 MB character
    length text line shows a non-linear slowing down during the
    load, resulting program crash after freezing 10 minutes.
    
    Now this file is opened within seconds again, allowing to modify,
    select and copy of the text before the full load of the document
    (with linear speed). Loading a much shorter 5 MB character
    length text line resulted an unusably slow editing previously:
    waiting half minutes for changing, selecting or copying text
    parts.
    
    Manual test
    
    Create a text file with a 76 MB character long line, and
    open it:
    
    $ seq 10000000 | tr '\n' ' '  >numbers.txt
    $ soffice numbers.txt
    
    Note: the reverted commit had also a paragraph limit exceeded
    error using unlimited paragraph length, because that is limited
    in 4 GiB characters since version 4.3, see tdf#30668 and commit
    ba27366f3d6bc6b209ecd5c5cb79a9ee5315316a "Resolves: #i17171#
    Writer paragraph cannot be longer than 65534 characters".
    
    Change-Id: If91e340a8aac4d4ebc4097b19cfa854b5659da3b
    Reviewed-on: https://gerrit.libreoffice.org/c/core/+/148169
    Tested-by: László Németh <nem...@numbertext.org>
    Reviewed-by: László Németh <nem...@numbertext.org>

diff --git a/sw/inc/shellio.hxx b/sw/inc/shellio.hxx
index e666442d19a9..b3a18e65f510 100644
--- a/sw/inc/shellio.hxx
+++ b/sw/inc/shellio.hxx
@@ -51,6 +51,14 @@ struct Writer_Impl;
 namespace sw::mark { class IMark; }
 namespace com::sun::star::embed { class XStorage; }
 
+// Defines the count of chars at which a paragraph read via ASCII/W4W-Reader
+// is forced to wrap. It has to be always greater than 200!!!
+// Note: since version 4.3, maximum character count changed to 4 GiB from 64 
KiB
+// in a paragraph, but because of the other limitations, we set a lower wrap 
value
+// to get a working text editor e.g. without freezing and crash during loading 
of
+// a 50 MB text line, or unusably slow editing of a 5 MB text line.
+#define MAX_ASCII_PARA 250000
+
 class SW_DLLPUBLIC SwAsciiOptions
 {
     OUString m_sFont;
diff --git a/sw/source/filter/ascii/parasc.cxx 
b/sw/source/filter/ascii/parasc.cxx
index eab7a59e898b..b29251bcbd8b 100644
--- a/sw/source/filter/ascii/parasc.cxx
+++ b/sw/source/filter/ascii/parasc.cxx
@@ -474,6 +474,17 @@ ErrCode SwASCIIParser::ReadChars()
 
         if( bIns )
         {
+            if( ( nLineLen >= MAX_ASCII_PARA - 100 ) &&
+                ( ( *pStt == ' ' ) || ( nLineLen >= MAX_ASCII_PARA - 1 ) ) )
+            {
+                sal_Unicode c = *pStt;
+                *pStt = 0;
+                InsertText( OUString( pLastStt ));
+                
m_rDoc.getIDocumentContentOperations().SplitNode(*m_oPam->GetPoint(), false);
+                pLastStt = pStt;
+                nLineLen = 0;
+                *pStt = c;
+            }
             ++pStt;
             ++nLineLen;
         }