Hi A long and painful bug with CSV files has been that OpenOffice has a row (line) limit of 64 KiB (in characters), as well as a field limit of 64 KiB. While individual lines of >= 64 KiB are rare, quoted CSV fields can span multiple lines.
Now, at least in Base, this limitation is finally gone!!! With my commit 7b2bc0e6bba2fbc38d078306fe10d875115d6c86: - New member functions were added to the main/tools SvStream class to work with 32 bit OUString and OStringBuilder when reading lines. - The helper class QuotedString had to be upgraded from using the 16 bit String to the 32 bit OUString. - The CSV database driver was patched to use OUString and 32 bit indexes in various places. - Luckily, little other work was needed, as the ORowSetValue class already uses 32 bit OUString, and was previously converting 16 bit String to 32 bit OUString internally anyway. And it works now, and works well! CSV files load fully. A field over 64 KiB no longer corrupts reading of further data from the file. Cells copy to Calc, copy out to other applications (admittedly truncated when over 64 KiB, and with line feeds removed, but that's due to other bugs in the clipboard, because pasting to Calc doesn't remove line feeds). SQL queries on CSV files work and give correct results. Oh and the new storage limit imposed by OUString is the signed 32 bit limit, about 2 GiB. This should be seen as phase 1. In phase 2 I want to unify CSV parsing between Base and Calc, reducing the amount of code we need. And at least, Calc should read 32 bit OUString rows, and then possibly impose 64 KiB field size limits, instead of the current 16 bit String sized lines and corruption of further data (https://bz.apache.org/ooo/show_bug.cgi?id=91028 ). Cherry-picked to AOO42X in 556cbcf7b90911f7cbf5dcdaffd2767ad2b2e230. Cherry-picked to AOO41X in f13410cf5cd4d68144c38af7cf9e805599c0d5cf (and 3 previous backported CSV patches). Regards Damjan
