i just re-created the situation, and here is a callstack (made in windows),
when i save_as an ods file.
it writes (still uncomressed) data to a stream, but that stream is a file on my
harddrive:
C:\\Users\\Szucsi\\AppData\\Local\\Temp\\lu99163n3gde.tmp\\lu9916{8AA24C62-674C-4CFF-BB49-581F27614405}.tmp
> mergedlo.dll!utl::OSeekableInputStreamWrapper::seek(__int64 _nLocation)
>Line 179 C++
mergedlo.dll!OWriteStream::writeBytes(const
com::sun::star::uno::Sequence<signed char> & aData) Line 2057 C++
expwraplo.dll!`anonymous namespace'::SaxWriterHelper::writeSequence() Line
231 C++
expwraplo.dll!`anonymous namespace'::SaxWriterHelper::convertToXML(const
char16_t * pStr, long nStrLen, bool bDoNormalization, bool
bNormalizeWhitespace, char * pTarget, unsigned long & rPos) Line 527 C++
expwraplo.dll!`anonymous namespace'::SaxWriterHelper::writeString(const
rtl::OUString & rWriteOutString, bool bDoNormalization, bool
bNormalizeWhitespace) Line 601 C++
expwraplo.dll!`anonymous namespace'::SaxWriterHelper::startElement(const
rtl::OUString & rName, const
com::sun::star::uno::Reference<com::sun::star::xml::sax::XAttributeList> &
xAttribs) Line 711 C++
expwraplo.dll!`anonymous namespace'::SAXWriter::startElement(const
rtl::OUString & aName, const
com::sun::star::uno::Reference<com::sun::star::xml::sax::XAttributeList> &
xAttribs) Line 1207 C++
mergedlo.dll!SvXMLExport::StartElement(const rtl::OUString & rName, bool
bIgnWSOutside) Line 2111 C++
mergedlo.dll!SvXMLElementExport::SvXMLElementExport(SvXMLExport & rExp,
const rtl::OUString & rQName, bool bIWSOutside, bool bIWSInside) Line 2460
C++
sclo.dll!ScXMLExport::WriteCell(ScMyCell & aCell, long nEqualCellCount)
Line 3268 C++
sclo.dll!ScXMLExport::WriteTable(long nTable, const
com::sun::star::uno::Reference<com::sun::star::sheet::XSpreadsheet> & xTable)
Line 2962 C++
sclo.dll!ScXMLExport::ExportContent_() Line 1943 C++
mergedlo.dll!SvXMLExport::ImplExportContent() Line 1159 C++
mergedlo.dll!SvXMLExport::exportDoc(xmloff::token::XMLTokenEnum eClass)
Line 1384 C++
sclo.dll!ScXMLExport::exportDoc(xmloff::token::XMLTokenEnum eClass) Line
5393 C++
mergedlo.dll!SvXMLExport::filter(const
com::sun::star::uno::Sequence<com::sun::star::beans::PropertyValue> &
aDescriptor) Line 812 C++
sclo.dll!ScXMLExport::filter(const
com::sun::star::uno::Sequence<com::sun::star::beans::PropertyValue> &
aDescriptor) Line 5431 C++
sclo.dll!ScXMLImportWrapper::ExportToComponent(const
com::sun::star::uno::Reference<com::sun::star::uno::XComponentContext> &
xContext, const com::sun::star::uno::Reference<com::sun::star::frame::XModel> &
xModel, const com::sun::star::uno::Reference<com::sun::star::xml::sax::XWriter>
& xWriter, const
com::sun::star::uno::Sequence<com::sun::star::beans::PropertyValue> &
aDescriptor, const rtl::OUString & sName, const rtl::OUString & sMediaType,
const rtl::OUString & sComponentName, const
com::sun::star::uno::Sequence<com::sun::star::uno::Any> & aArgs,
std::unique_ptr<ScMySharedData,std::default_delete<ScMySharedData>> &
pSharedData) Line 711 C++
sclo.dll!ScXMLImportWrapper::Export(bool bStylesOnly) Line 947 C++
sclo.dll!ScDocShell::SaveXML(SfxMedium * pSaveMedium, const
com::sun::star::uno::Reference<com::sun::star::embed::XStorage> & xStor) Line
569 C++
sclo.dll!ScDocShell::SaveAs(SfxMedium & rMedium) Line 1861 C++
...
and here is the value of 'this' in (OSeekableInputStreamWrapper::seek):
-this 0x00000272b1388610 {...} utl::OSeekableInputStreamWrapper *
{utl::OStreamWrapper}
- [utl::OStreamWrapper] {...} utl::OStreamWrapper
-
cppu::ImplInheritanceHelper<utl::OSeekableInputStreamWrapper,com::sun::star::io::XStream,com::sun::star::io::XOutputStream,com::sun::star::io::XTruncate>
{...}
cppu::ImplInheritanceHelper<utl::OSeekableInputStreamWrapper,com::sun::star::io::XStream,com::sun::star::io::XOutputStream,com::sun::star::io::XTruncate>
- utl::OSeekableInputStreamWrapper {...}
utl::OSeekableInputStreamWrapper
-
cppu::ImplInheritanceHelper<utl::OInputStreamWrapper,com::sun::star::io::XSeekable>
{...}
cppu::ImplInheritanceHelper<utl::OInputStreamWrapper,com::sun::star::io::XSeekable>
- utl::OInputStreamWrapper {m_aMutex={...}
m_pSvStream=0x00000272a992ff10 {mxFileHandle=0x00000000000049b0 nLockCounter=0
aFilename=...} ...} utl::OInputStreamWrapper
+ utl::OInputStreamWrapper_Base {...}
utl::OInputStreamWrapper_Base
+ comphelper::ByteReader {...}
comphelper::ByteReader
+ m_aMutex {...} std::mutex
- m_pSvStream 0x00000272a992ff10
{mxFileHandle=0x00000000000049b0 nLockCounter=0
aFilename=u"C:\\Users\\Szucsi\\AppData\\Local\\Temp\\lu99163n3gde.tmp\\lu9916{8AA24C62-674C-4CFF-BB49-581F27614405}.tmp"
...} SvStream * {SvFileStream}
- [SvFileStream] {mxFileHandle=0x00000000000049b0
nLockCounter=0
aFilename=u"C:\\Users\\Szucsi\\AppData\\Local\\Temp\\lu99163n3gde.tmp\\lu9916{8AA24C62-674C-4CFF-BB49-581F27614405}.tmp"
...} SvFileStream
+ SvStream {m_xLockBytes=nullptr m_nActPos=0
m_pRWBuf=unique_ptr 205 'Í' ...} SvStream
mxFileHandle 0x00000000000049b0 void *
nLockCounter 0 unsigned short
+ aFilename
u"C:\\Users\\Szucsi\\AppData\\Local\\Temp\\lu99163n3gde.tmp\\lu9916{8AA24C62-674C-4CFF-BB49-581F27614405}.tmp"
rtl::OUString
bIsOpen true bool
...
I can see this file on my hard drive, so it is really here.
i can view its content and it is a half made xml file
content.xml (the whole file) ~110mb big, but the compressed ods is only ~670kb.
so if it would be stored only in memory, we could avoid a lot of disk writing.
but, maybe this filestream cration (before zipping) designed to make sure we
dont run out of memory during the save.. ? ..
i can understand that in most cases this extra file is not a big problem... and
when the file is big enought to be a problem, than maybe the memory could be a
bigger problem.. :)
On Sunday, May 14, 2023 12:46 BST, Noel Grandin <noelgran...@gmail.com> wrote:
On Tue, 14 Feb 2023 at 17:40, Noel Grandin <noel.gran...@collabora.co.uk>
wrote:On 2/13/2023 4:34 PM, Michael Meeks wrote:
>
> Noel might be interested in this. Noel, Attila suggests that every zip
>file we write we have to stream the full
> un-compressed data to /tmp files before compressing it. And these can be huge
> =)
>
I couldn't find any of this specific thing happening. Perhaps Attila has a
codepointer or a stack trace? What I can see is that we save to a temporary
file and then copy the temporary file to its final location.That part however
would require major surgery to change since it starts in sfx and goes through
the dreaded UCB layer.
