On Mon, Oct 16, 2023 at 07:58:00PM +0200, Enrico Forestieri wrote:
> On Mon, Oct 16, 2023 at 03:41:59PM +0200, Pavel Sanda wrote:
> >
> >On Sun, Oct 15, 2023 at 09:07:24PM +0200, Enrico Forestieri wrote:
> >>Pavel, in commit eef0c8e8 you say:
> >>
> >>>xHTML export: change filenames of exported images.
> >>>
> >>>This patch aims at:
> >>>1. replacing absolute paths by their hashes (do not leak directory
> >>>structures)
> >>>2. not using counters anymore so that changing figures order in the
> >>>document
> >>> does not lead to large number of obsolete images in export directory.
> >>>
> >>>Other changes than in xHTML export of images are unintended.
> >>
> >>I think that there are unintented changes because now the mangled names of
> >>graphics in the temporary dir do not contain the mangled path and thus it is
> >>difficult/impossible to tell from where they are coming.
> >
> >I'll try to fix that in the next days, as this should not appear for latex
> >export.
> >But you are OK with having it obfuscated in temp directory when xHTML output
> >is used, right?
>
> Sure.
This patch should salvage the situation (I separated the tables for xhtml() and
latex() routes).
There is one point I'm not sure about - is it really true that windows still
set default limit for max path length to ~260 chars?
If so it might be good use shorter hash for windows. I previously used sha-256
resulting in +65 chars, but we could swicth to sha-1 (41 chars) or turn it off
altogether for windows. Any opinion about that?
Pavel
diff --git a/src/insets/InsetGraphics.cpp b/src/insets/InsetGraphics.cpp
index dac61ee449..62577f07db 100644
--- a/src/insets/InsetGraphics.cpp
+++ b/src/insets/InsetGraphics.cpp
@@ -575,14 +575,14 @@ copyFileIfNeeded(FileName const & file_in, FileName const
& file_out)
pair<GraphicsCopyStatus, FileName> const
-copyToDirIfNeeded(DocFileName const & file, string const & dir)
+copyToDirIfNeeded(DocFileName const & file, string const & dir, bool
encrypt_path)
{
string const file_in = file.absFileName();
string const only_path = onlyPath(file_in);
if (rtrim(only_path, "/") == rtrim(dir, "/"))
return make_pair(IDENTICAL_PATHS, FileName(file_in));
- string mangled = file.mangledFileName(empty_string(), false, true);
+ string mangled = file.mangledFileName(empty_string(), encrypt_path);
if (theFormats().isZippedFile(file)) {
// We need to change _eps.gz to .eps.gz. The mangled name is
// still unique because of the counter in mangledFileName().
@@ -680,7 +680,7 @@ string InsetGraphics::prepareFile(OutputParams const &
runparams) const
// we move it to a temp dir or uncompress it.
FileName temp_file;
GraphicsCopyStatus status;
- tie(status, temp_file) = copyToDirIfNeeded(params().filename,
temp_path);
+ tie(status, temp_file) = copyToDirIfNeeded(params().filename,
temp_path, false);
if (status == FAILURE)
return orig_file;
@@ -997,7 +997,7 @@ string InsetGraphics::prepareHTMLFile(OutputParams const &
runparams) const
// Copy to temporary directory.
FileName temp_file;
GraphicsCopyStatus status;
- tie(status, temp_file) = copyToDirIfNeeded(params().filename,
temp_path);
+ tie(status, temp_file) = copyToDirIfNeeded(params().filename,
temp_path, true);
if (status == FAILURE)
return string();
diff --git a/src/support/FileName.cpp b/src/support/FileName.cpp
index 32a5862fdb..d2819f9d86 100644
--- a/src/support/FileName.cpp
+++ b/src/support/FileName.cpp
@@ -958,68 +958,72 @@ string DocFileName::outputFileName(string const & path)
const
}
-string DocFileName::mangledFileName(string const & dir) const
-{
- return mangledFileName(dir, true, false);
-}
-
-string DocFileName::mangledFileName(string const & dir, bool use_counter, bool
encrypt_path) const
+string DocFileName::mangledFileName(string const & dir, bool encrypt_path)
const
{
// Concurrent access to these variables is possible.
// We need to make sure that every DocFileName instance for a given
// filename returns the same mangled name.
typedef map<string, string> MangledMap;
- static MangledMap mangledNames;
+ static MangledMap mangledPlainNames;
+ static MangledMap mangledEncryptNames; //separate table for xhtml() route
static Mutex mangledMutex;
// this locks both access to mangledNames and counter below
Mutex::Locker lock(&mangledMutex);
- MangledMap::const_iterator const it = mangledNames.find(absFileName());
- if (it != mangledNames.end())
+ MangledMap * mangledNames = encrypt_path ? &mangledEncryptNames :
&mangledPlainNames;
+ MangledMap::const_iterator const it = mangledNames->find(absFileName());
+ if (it != mangledNames->end())
return (*it).second;
string const name = absFileName();
// Now the real work. Remove the extension.
string mname = support::changeExtension(name, string());
- if (encrypt_path)
- mname = "export_" + onlyFileName() + "_" + toHexHash(mname);
+ if (encrypt_path) {
+ string enc_name = "export_" + onlyFileName() + "_" + toHexHash(mname);
+ if (enc_name.length() > 250) //various filesystems have filename limit
around 2^8
+ enc_name = "export_" + toHexHash(mname);
+ mname = enc_name;
+ }
// The mangled name must be a valid LaTeX name.
mname = sanitizeFileName(mname);
// Add the extension back on
mname = support::changeExtension(mname, getExtension(name));
- // Prepend a counter to the filename. This is necessary to make
- // the mangled name unique.
- static int counter = 0;
+ // Counter trick for MikTeX/YAP not needed for xHTML
+ if (!encrypt_path) {
+ // Prepend a counter to the filename. This is necessary to make
+ // the mangled name unique, see truncation below.
+ //
+ static int counter = 0;
- if (use_counter) {
ostringstream s;
s << counter++ << mname;
mname = s.str();
- }
- // MiKTeX's YAP (version 2.4.1803) crashes if the file name
- // is longer than about 160 characters. MiKTeX's pdflatex
- // is even pickier. A maximum length of 100 has been proven to work.
- // If dir.size() > max length, all bets are off for YAP. We truncate
- // the filename nevertheless, keeping a minimum of 10 chars.
- string::size_type max_length = max(100 - ((int)dir.size() + 1), 10);
+ // MiKTeX's YAP (version 2.4.1803) crashes if the file name
+ // is longer than about 160 characters. MiKTeX's pdflatex
+ // is even pickier. A maximum length of 100 has been proven to
work.
+ // If dir.size() > max length, all bets are off for YAP. We
truncate
+ // the filename nevertheless, keeping a minimum of 10 chars.
- // If the mangled file name is too long, hack it to fit.
- // We know we're guaranteed to have a unique file name because
- // of the counter.
- if (mname.size() > max_length) {
- int const half = (int(max_length) / 2) - 2;
- if (half > 0) {
- mname = mname.substr(0, half) + "___" +
- mname.substr(mname.size() - half);
+ string::size_type max_length = max(100 - ((int)dir.size() + 1),
10);
+
+ // If the mangled file name is too long, hack it to fit.
+ // We know we're guaranteed to have a unique file name because
+ // of the counter.
+ if (mname.size() > max_length) {
+ int const half = (int(max_length) / 2) - 2;
+ if (half > 0) {
+ mname = mname.substr(0, half) + "___" +
+ mname.substr(mname.size() - half);
+ }
}
}
- mangledNames[absFileName()] = mname;
+ (*mangledNames)[absFileName()] = mname;
return mname;
}
diff --git a/src/support/FileName.h b/src/support/FileName.h
index 95940a9235..43904420d5 100644
--- a/src/support/FileName.h
+++ b/src/support/FileName.h
@@ -288,19 +288,30 @@ public:
* - two FileName instances with the same filename have identical
* mangled names.
*
+ * @param encrypt_path will use hash (SHA-256) instead of path
+ * in the mangled name if set to true. Useful for xHTML export
+ * so we do not leak e.g. user names contained in the paths.
+ * Filename itself is stripped if the resulting filename would
+ * become too long (~250 chars).
+ * Prefix counter is not used because
+ * 1) it's hack useful for MikTeX/YAP only
+ * 2) causes many duplicates within the export directory across
different
+ * LyX sessions as (unlike in LaTeX export) we use mangled names in
+ * final xHTML export directory.
+ * An example of hashed mangled case:
+ * C:/foo bar/baz.png - >
export_baz_png_9666c406ce4533c7c63ef16541a93eacaf9ed41f026fa14a642c34611be36e35.png
+ *
+ * It is guaranteed that
+ * - two different filenames have different mangled names (modulo hash
collision)
+ * - two FileName instances with the same filename have identical
hashed
+ * mangled names.
+ *
+ *
* Only the mangled file name is returned. It is not prepended
* with @c dir.
*/
std::string
- mangledFileName(std::string const & dir = empty_string()) const;
-
- /** Identical to mangledFileName, with the following additions:
- *
- * @encrypt_path allows using hash (SHA-256) instead of full path.
- * @use_counter allows disabling the counter in the filename.
- */
- std::string
- mangledFileName(std::string const & dir, bool use_counter, bool
encrypt_path) const;
+ mangledFileName(std::string const & dir = empty_string(), bool
encrypt_path=false) const;
/// \return the absolute file name without its .gz, .z, .Z extension
std::string unzippedFileName() const;
--
lyx-devel mailing list
lyx-devel@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-devel
