Package: shared-mime-info Version: 1.3-1 Severity: wishlist Tags: upstream patch
This was filed upstream here: https://bugs.freedesktop.org/show_bug.cgi?id=82711 In update-mime-database, first write all files, then sync all filesystems, then move all files. The original behavior was very slow for two reasons: - After every file creation, we had to wait for the harddisk to complete the write. - We told the kernel and the harddisk in which order writes should happen. This reduces the time needed on my harddisk from 44-45s to 0.9-1.4s. More than an order of magnitude speedup! :) The only downside I can see is that if update-mime-database is interrupted for some reason, there are more *.new files in the FS. I don't think that that's a big problem. -- System Information: Debian Release: jessie/sid APT prefers testing APT policy: (500, 'testing') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 3.15.8 (SMP w/8 CPU cores) Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages shared-mime-info depends on: ii libc6 2.19-7 ii libglib2.0-0 2.40.0-3 ii libxml2 2.9.1+dfsg1-4 shared-mime-info recommends no packages. shared-mime-info suggests no packages. -- no debconf information
>From e00ead777e91e6203458aaf8c5acb141918b153d Mon Sep 17 00:00:00 2001 From: Jann Horn <[email protected]> Date: Sat, 16 Aug 2014 22:57:46 +0200 Subject: [PATCH] ~factor 40 speedup First write all files, then sync all filesystems, then move all files. The original behavior was very slow for two reasons: - After every file creation, we had to wait for the harddisk to complete the write. - We told the kernel and the harddisk in which order writes should happen. This reduces the time needed on my harddisk from 44-45s to 0.9-1.4s. --- update-mime-database.c | 52 ++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 44 insertions(+), 8 deletions(-) diff --git a/update-mime-database.c b/update-mime-database.c index be4aba2..43ccc82 100644 --- a/update-mime-database.c +++ b/update-mime-database.c @@ -85,6 +85,13 @@ struct _Type { char *media; char *subtype; + /* Contains the path under which the temporary file describing the + * type has been written. Only set to a meaningful value between + * file creation (write_out_type_tmpfile) and the file getting + * moved to its final name (move_type_file). + */ + char *filename; + /* Contains xmlNodes for elements that are being copied to the output. * That is, <comment>, <sub-class-of> and <alias> nodes, and anything * with an unknown namespace. @@ -1014,11 +1021,11 @@ out: } /* Write out an XML file for one type */ -static void write_out_type(gpointer key, gpointer value, gpointer data) +static void write_out_type_tmpfile(gpointer key, gpointer value, gpointer data) { Type *type = (Type *) value; const char *mime_dir = (char *) data; - char *media, *filename; + char *media; GError *local_error = NULL; char *lower; @@ -1032,18 +1039,45 @@ static void write_out_type(gpointer key, gpointer value, gpointer data) #endif lower = g_ascii_strdown(type->subtype, -1); - filename = g_strconcat(media, "/", lower, ".xml.new", NULL); + type->filename = g_strconcat(media, "/", lower, ".xml.new", NULL); g_free(lower); g_free(media); media = NULL; - if (!save_xml_file(type->output, filename, &local_error)) + if (!save_xml_file(type->output, type->filename, &local_error)) fatal_gerror(local_error); +} - if (!atomic_update(filename, &local_error)) - fatal_gerror(local_error); +/* Move the XML file to its correct name */ +static void move_type_file(gpointer key, gpointer value, gpointer data) +{ + Type *type = (Type *) value; + gchar *new_name = NULL; + int len; + +/*start*/ + + len = strlen(type->filename); - g_free(filename); + if (strcmp(type->filename + len - 4, ".new") != 0) + fatal_gerror(NULL); + + new_name = g_strndup(type->filename, len - 4); + +#ifdef _WIN32 + /* we need to remove the old file first! */ + remove(new_name); +#endif + if (rename(type->filename, new_name) == -1) + { + g_printerr("Failed to rename %s as %s: %s\n", type->filename, new_name, + g_strerror(errno)); + exit (EXIT_FAILURE); + } + + g_free(new_name); + g_free(type->filename); + type->filename = NULL; } /* Comparison function to get the magic rules in priority order */ @@ -3697,7 +3731,9 @@ int main(int argc, char **argv) delete_old_types(mime_dir); - g_hash_table_foreach(types, write_out_type, (gpointer) mime_dir); + g_hash_table_foreach(types, write_out_type_tmpfile, (gpointer) mime_dir); + sync(); + g_hash_table_foreach(types, move_type_file, NULL); { FILE *globs; -- 2.0.1
