Hi all,
I felt that it would be nice when mutt instantly notifies about new
mail. Think about mails like "There is on piece of cake left!".
Monitoring files is kind of "state of the art" nowadays, but
unfortunately not portable. So my goal was to limit interference with
existing functionality, and only give a trigger when a relevant file
changed, followed by the established logic for checking mailboxes doing
exactly the same as before.
As I am using linux, "my" mechanism is inotify,
This approach could be implemented pretty good by replacing getch() by
poll() as central point of waiting for input. Although at the heart of
mutt, it is less invasive as it sounds.
The implementation in (new) monitor.c is very specific to inotify, but
was coded having addition of more interfaces in mind, structured that
for top-level functions switches/ifdefs pointing other to stuff can be
done.
Using
When the attached patch is applied, configure checks whether inotify is
available for compilation (supported by glibc on Linux) and if possible
uses it (currently no extra --enable option).
The CONTEXT and BUFFY folders are equipped with monitors. Right now
the lifetime is limited to files existing on start of mutt. Creating
specified mailboxes later is not recognized, and replacing files ends
with monitors for these removed.
Failures in the inotify area are hidden to the user. With -d2 they
appear in the debugfile, -d3 adds regular events and -d5 the bits from
the events.
Supported mbox types
All local: mbox, mmdf (not tested by me), maildir, mh
For maildir, currently only the new directory is checked: When old
mails are copied from another folder, these will be recognized only
later according to Timeout/user input. (cur directories can easily be
added, but would double the number of watches.)
For NFS all this is not really useful in most cases. It shouldn't break,
but AFAIK there is no way to get change events from remote machines.
Tested
- Mailbox types: see above
- General compilation: on Linux with/without **1 inotify
- Sidebar compilation: breaks with same message in original and patched
source with my configuration/system (current git master HEAD).
**1 I tested by intentionally braking a configure for a required method.
Technical notes
Maildir:
According the specification the expected event is IN_MOVED_TO . This
works when Exim delivers the mail, however when mutt copies messages
around the only usable event is IN_ATTRIB .
MH:
Mutt replaces file .mh_sequences, so this is the only case where a
monitor will be "refreshed".
Identify files:
Comparing st_dev and st_ino should be sufficient, correct? Why does
compare_stat() (which I couldn't use here anyway) care about st_rdev?
#ifdef:
More of these ;-) I consider it not a bad thing, because on the event
that mutt's architecture is modernized it helps to understand
dependencies.
So, have fun!
Kind regards,
Gero
commit aefde7a91eba442e021c8d1c6cfb00ad93b4c4dc
Author: Gero Treuner <gero-m...@innocircle.com>
Date: Mon Apr 2 15:05:56 2018 +0200
Add file monitoring, using inotify (available on Linux)
diff --git a/Makefile.am b/Makefile.am
index 7418a0b8..79d52c0e 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -56,7 +56,7 @@ AM_CPPFLAGS=-I. -I$(top_srcdir) $(IMAP_INCLUDES) $(GPGME_CFLAGS) -Iintl
EXTRA_mutt_SOURCES = account.c bcache.c compress.c crypt-gpgme.c crypt-mod-pgp-classic.c \
crypt-mod-pgp-gpgme.c crypt-mod-smime-classic.c \
- crypt-mod-smime-gpgme.c dotlock.c gnupgparse.c hcache.c md5.c \
+ crypt-mod-smime-gpgme.c dotlock.c gnupgparse.c hcache.c md5.c monitor.c \
mutt_idna.c mutt_sasl.c mutt_socket.c mutt_ssl.c mutt_ssl_gnutls.c \
mutt_tunnel.c pgp.c pgpinvoke.c pgpkey.c pgplib.c pgpmicalg.c \
pgppacket.c pop.c pop_auth.c pop_lib.c remailer.c resize.c sha1.c \
diff --git a/buffy.c b/buffy.c
index 728f819d..c58e75f3 100644
--- a/buffy.c
+++ b/buffy.c
@@ -37,6 +37,10 @@
#include "imap.h"
#endif
+#ifdef USE_INOTIFY
+#include "monitor.h"
+#endif
+
#include <string.h>
#include <sys/stat.h>
#include <dirent.h>
@@ -218,6 +222,9 @@ static BUFFY *buffy_new (const char *path)
static void buffy_free (BUFFY **mailbox)
{
+#ifdef USE_INOTIFY
+ mutt_monitor_remove (*mailbox);
+#endif
FREE (mailbox); /* __FREE_CHECKED__ */
}
@@ -547,6 +554,11 @@ int mutt_buffy_check (int force)
}
}
+#ifdef USE_INOTIFY
+ if (force)
+ mutt_monitor_add (tmp);
+#endif
+
/* check to see if the folder is the currently selected folder
* before polling */
if (!Context || !Context->path ||
diff --git a/configure.ac b/configure.ac
index 7034f869..debf337b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1404,6 +1404,17 @@ else
MUTT_LIB_OBJECTS="$MUTT_LIB_OBJECTS utf8.o wcwidth.o"
fi
+have_inotify=yes
+AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch,,
+[
+ have_inotify=no
+])
+if test x$have_inotify = xyes; then
+ AC_DEFINE(USE_INOTIFY,1,[ Define if want to use inotify for filesystem monitoring (available in Linux only). ])
+ AC_CHECK_HEADERS(sys/inotify.h)
+ MUTT_LIB_OBJECTS="$MUTT_LIB_OBJECTS monitor.o"
+fi
+
AC_CACHE_CHECK([for nl_langinfo and CODESET], mutt_cv_langinfo_codeset,
[AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <langinfo.h>]], [[char* cs = nl_langinfo(CODESET);]])],[mutt_cv_langinfo_codeset=yes],[mutt_cv_langinfo_codeset=no])])
if test $mutt_cv_langinfo_codeset = yes; then
diff --git a/curs_lib.c b/curs_lib.c
index ecac6d95..160b841b 100644
--- a/curs_lib.c
+++ b/curs_lib.c
@@ -26,6 +26,9 @@
#include "mutt_curses.h"
#include "pager.h"
#include "mbyte.h"
+#ifdef USE_INOTIFY
+#include "monitor.h"
+#endif
#include <termios.h>
#include <sys/types.h>
@@ -120,7 +123,12 @@ event_t mutt_getch (void)
ch = KEY_RESIZE;
while (ch == KEY_RESIZE)
#endif /* KEY_RESIZE */
- ch = getch ();
+#ifdef USE_INOTIFY
+ if (mutt_monitor_poll () != 0)
+ ch = ERR;
+ else
+#endif
+ ch = getch ();
mutt_allow_interrupt (0);
if (SigInt)
diff --git a/curs_main.c b/curs_main.c
index b2c18cad..07ce83df 100644
--- a/curs_main.c
+++ b/curs_main.c
@@ -41,6 +41,10 @@
#include "imap_private.h"
#endif
+#ifdef USE_INOTIFY
+#include "monitor.h"
+#endif
+
#include "mutt_crypt.h"
@@ -577,6 +581,9 @@ int mutt_index_menu (void)
if (!attach_msg)
mutt_buffy_check(1); /* force the buffy check after we enter the folder */
+#ifdef USE_INOTIFY
+ mutt_monitor_add (NULL);
+#endif
FOREVER
{
@@ -1265,6 +1272,9 @@ int mutt_index_menu (void)
int check;
char *new_last_folder;
+#ifdef USE_INOTIFY
+ mutt_monitor_remove (NULL);
+#endif
#ifdef USE_COMPRESSED
if (Context->compress_info && Context->realpath)
new_last_folder = safe_strdup (Context->realpath);
@@ -1306,6 +1316,9 @@ int mutt_index_menu (void)
MUTT_READONLY : 0, NULL)) != NULL)
{
menu->current = ci_first_message ();
+#ifdef USE_INOTIFY
+ mutt_monitor_add (NULL);
+#endif
}
else
menu->current = 0;
diff --git a/keymap.c b/keymap.c
index 6a290413..83f5a195 100644
--- a/keymap.c
+++ b/keymap.c
@@ -29,6 +29,9 @@
#ifdef USE_IMAP
#include "imap/imap.h"
#endif
+#ifdef USE_INOTIFY
+#include "monitor.h"
+#endif
#include <stdlib.h>
#include <string.h>
@@ -450,7 +453,11 @@ int km_dokey (int menu)
* loop now. Otherwise, continue to loop until reaching a total of
* $timeout seconds.
*/
+#ifdef USE_INOTIFY
+ if (tmp.ch != -2 || SigWinch || MonitorFilesChanged)
+#else
if (tmp.ch != -2 || SigWinch)
+#endif
goto gotkey;
i -= ImapKeepalive;
imap_keepalive ();
diff --git a/main.c b/main.c
index b189c860..d1c6c41d 100644
--- a/main.c
+++ b/main.c
@@ -47,6 +47,10 @@
#include "hcache.h"
#endif
+#ifdef USE_INOTIFY
+#include "monitor.h"
+#endif
+
#include <string.h>
#include <stdlib.h>
#include <locale.h>
@@ -498,6 +502,8 @@ static void show_version (void)
"-USE_HCACHE "
#endif
+ "\n"
+
#ifdef USE_SIDEBAR
"+USE_SIDEBAR "
#else
@@ -510,6 +516,12 @@ static void show_version (void)
"-USE_COMPRESSED "
#endif
+#ifdef USE_INOTIFY
+ "+USE_INOTIFY "
+#else
+ "-USE_INOTIFY "
+#endif
+
);
#ifdef ISPELL
diff --git a/monitor.c b/monitor.c
new file mode 100644
index 00000000..f5847816
--- /dev/null
+++ b/monitor.c
@@ -0,0 +1,389 @@
+/* * Copyright (C) 2018 Gero Treuner <g...@70t.de>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+ */
+
+#if HAVE_CONFIG_H
+# include "config.h"
+#endif
+
+#if HAVE_SYS_INOTIFY_H
+# include <sys/types.h>
+# include <sys/inotify.h>
+# include <unistd.h>
+# include <poll.h>
+#endif
+
+#include "mutt.h"
+#include "buffy.h"
+#include "monitor.h"
+#include "mx.h"
+
+#include <errno.h>
+#include <sys/stat.h>
+
+static size_t PollFdsCount = 0;
+static size_t PollFdsLen = 0;
+static struct pollfd *PollFds;
+
+typedef struct monitorinfo_t
+{
+ short magic;
+ short isdir;
+ const char *path;
+ dev_t st_dev;
+ ino_t st_ino;
+ MONITOR *monitor;
+ char _pathbuf[_POSIX_PATH_MAX]; /* access via path only (maybe not initialized) */
+}
+MONITORINFO;
+
+#define INOTIFY_MASK_DIR (IN_MOVED_TO | IN_ATTRIB | IN_CLOSE_WRITE | IN_ISDIR)
+#define INOTIFY_MASK_FILE IN_CLOSE_WRITE
+
+static void mutt_poll_fd_add(int fd, short events)
+{
+ int i = 0;
+ for (i = 0; i < PollFdsCount && PollFds[i].fd != fd; ++i);
+
+ if (i == PollFdsCount)
+ {
+ if (PollFdsCount == PollFdsLen)
+ {
+ PollFdsLen += 2;
+ safe_realloc (&PollFds, PollFdsLen * sizeof(struct pollfd));
+ }
+ ++PollFdsCount;
+ PollFds[i].fd = fd;
+ PollFds[i].events = events;
+ }
+ else
+ PollFds[i].events |= events;
+}
+
+static int mutt_poll_fd_remove(int fd)
+{
+ int i = 0, d;
+ for (i = 0; i < PollFdsCount && PollFds[i].fd != fd; ++i);
+ if (i == PollFdsCount)
+ return -1;
+ d = PollFdsCount - i - 1;
+ if (d)
+ memmove (&PollFds[i], &PollFds[i + 1], d * sizeof(struct pollfd));
+ --PollFdsCount;
+ return 0;
+}
+
+static int monitor_init ()
+{
+ if (INotifyFd == -1)
+ {
+ INotifyFd = inotify_init1(IN_NONBLOCK | IN_CLOEXEC);
+ if (INotifyFd == -1)
+ {
+ dprint (2, (debugfile, "monitor: inotify_init1 failed, errno=%d %s\n", errno, strerror(errno)));
+ return -1;
+ }
+ mutt_poll_fd_add(0, POLLIN);
+ mutt_poll_fd_add(INotifyFd, POLLIN);
+ }
+ return 0;
+}
+
+static void monitor_check_free ()
+{
+ if (!Monitor && INotifyFd != -1)
+ {
+ mutt_poll_fd_remove(INotifyFd);
+ close (INotifyFd);
+ INotifyFd = -1;
+ MonitorFilesChanged = 0;
+ }
+}
+
+static MONITOR *monitor_create (MONITORINFO *info, int descriptor)
+{
+ MONITOR *monitor = (MONITOR *) safe_calloc (1, sizeof (MONITOR));
+ monitor->magic = info->magic;
+ monitor->st_dev = info->st_dev;
+ monitor->st_ino = info->st_ino;
+ monitor->descr = descriptor;
+ monitor->next = Monitor;
+ if (info->magic == MUTT_MH)
+ monitor->mh_backup_path = safe_strdup(info->path);
+
+ Monitor = monitor;
+
+ return monitor;
+}
+
+static void monitor_delete (MONITOR *monitor)
+{
+ MONITOR **ptr = &Monitor;
+
+ if (!monitor)
+ return;
+
+ FOREVER
+ {
+ if (!*ptr)
+ return;
+ if (*ptr == monitor)
+ break;
+ ptr = &(*ptr)->next;
+ }
+
+ FREE (&monitor->mh_backup_path);
+ monitor = monitor->next;
+ FREE (ptr);
+ *ptr = monitor;
+}
+
+static int monitor_handle_ignore (int descr)
+{
+ int new_descr = -1;
+ MONITOR *iter = Monitor;
+ struct stat sb;
+
+ while (iter && iter->descr != descr)
+ iter = iter->next;
+
+ if (iter)
+ {
+ if (iter->magic == MUTT_MH && stat (iter->mh_backup_path, &sb) == 0)
+ {
+ if ((new_descr = inotify_add_watch (INotifyFd, iter->mh_backup_path, INOTIFY_MASK_FILE)) == -1)
+ dprint (2, (debugfile, "monitor: inotify_add_watch failed for '%s', errno=%d %s\n", iter->mh_backup_path, errno, strerror(errno)));
+ else
+ {
+ dprint (3, (debugfile, "monitor: inotify_add_watch descriptor=%d for '%s'\n", descr, iter->mh_backup_path));
+ iter->st_dev = sb.st_dev;
+ iter->st_ino = sb.st_ino;
+ iter->descr = new_descr;
+ }
+ }
+ else
+ {
+ dprint (3, (debugfile, "monitor: cleanup watch (implicitely removed) - descriptor=%d\n", descr));
+ }
+
+ if (new_descr == -1)
+ {
+ monitor_delete (iter);
+ monitor_check_free ();
+ }
+ }
+
+ return new_descr;
+}
+
+#define EVENT_BUFLEN MAX(4096, sizeof(struct inotify_event) + NAME_MAX + 1)
+
+/* mutt_monitor_poll: Waits for I/O ready file descriptors or signals.
+ *
+ * return values:
+ * -3 unknown/unexpected events: poll timeout / fds not handled by us
+ * -2 monitor detected changes, no STDIN input
+ * -1 error (see errno)
+ * 0 (1) input ready from STDIN, or (2) monitoring inactive -> no poll()
+ * MonitorFilesChanged also reflects changes to monitored files.
+ *
+ * Only STDIN and INotify file handles currently expected/supported.
+ * More would ask for common infrastructur (sockets?).
+ */
+int mutt_monitor_poll ()
+{
+ int rc = 0, fds, i, inputReady;
+ char buf[EVENT_BUFLEN]
+ __attribute__ ((aligned(__alignof__(struct inotify_event))));
+
+ MonitorFilesChanged = 0;
+
+ if (INotifyFd != -1)
+ {
+ fds = poll (PollFds, PollFdsLen, -1);
+ /* exits with !0 before notifier get written - permanent interrupt? */
+
+ if (fds == -1)
+ {
+ rc = -1;
+ if (errno != EINTR)
+ {
+ dprint (2, (debugfile, "monitor: poll() failed, errno=%d %s\n", errno, strerror(errno)));
+ }
+ }
+ else
+ {
+ inputReady = 0;
+ for (i = 0; fds && i < PollFdsCount; ++i)
+ {
+ if (PollFds[i].revents)
+ {
+ --fds;
+ if (PollFds[i].fd == 0)
+ {
+ inputReady = 1;
+ }
+ else if (PollFds[i].fd == INotifyFd)
+ {
+ MonitorFilesChanged = 1;
+ dprint (3, (debugfile, "monitor: file change(s) detected\n"));
+ int len;
+ char *ptr = buf;
+ const struct inotify_event *event;
+
+ FOREVER
+ {
+ len = read (INotifyFd, buf, sizeof(buf));
+ if (len == -1)
+ {
+ if (errno != EAGAIN)
+ dprint (2, (debugfile, "monitor: read inotify events failed, errno=%d %s\n",
+ errno, strerror(errno)));
+ break;
+ }
+
+ while (ptr < buf + len)
+ {
+ event = (const struct inotify_event *) ptr;
+ dprint (5, (debugfile, "monitor: + detail: descriptor=%d mask=0x%x\n",
+ event->wd, event->mask));
+ if (event->mask & IN_IGNORED)
+ monitor_handle_ignore (event->wd);
+ ptr += sizeof(struct inotify_event) + event->len;
+ }
+ }
+ }
+ }
+ }
+ if (!inputReady)
+ rc = MonitorFilesChanged ? -2 : -3;
+ }
+ }
+
+ return rc;
+}
+
+#define RESOLVERES_OK_NOTEXISTING 0
+#define RESOLVERES_OK_EXISTING 1
+#define RESOLVERES_FAIL_NOMAILBOX -2
+#define RESOLVERES_FAIL_STAT -1
+
+/* monitor_resolve: resolve monitor entry match by BUFFY, or - if NULL - by Context.
+ *
+ * return values:
+ * >=0 mailbox is valid and locally accessible:
+ * 0: no monitor / 1: preexisting monitor
+ * -2 no mailbox (MONITORINFO: no fields set)
+ * -1 stat() failed (see errno; MONITORINFO fields: magic, isdir, path)
+ */
+static int monitor_resolve (MONITORINFO *info, BUFFY *buffy)
+{
+ MONITOR *iter;
+ char *fmt = NULL;
+ struct stat sb;
+
+ if (buffy)
+ {
+ info->magic = buffy->magic;
+ info->path = buffy->realpath;
+ }
+ else if (Context)
+ {
+ info->magic = Context->magic;
+ info->path = Context->realpath;
+ }
+ else
+ {
+ return -2;
+ }
+
+ if (info->magic == MUTT_MAILDIR)
+ {
+ info->isdir = 1;
+ fmt = "%s/new";
+ }
+ else
+ {
+ info->isdir = 0;
+ if (info->magic == MUTT_MH)
+ fmt = "%s/.mh_sequences";
+ }
+
+ if (fmt)
+ {
+ snprintf (info->_pathbuf, sizeof(info->_pathbuf),
+ info->magic == MUTT_MAILDIR ? "%s/new" : "%s/.mh_sequences", info->path);
+ info->path = info->_pathbuf;
+ }
+ if (stat (info->path, &sb) != 0)
+ return -1;
+
+ iter = Monitor;
+ while (iter && (iter->st_ino != sb.st_ino || iter->st_dev != sb.st_dev))
+ iter = iter->next;
+
+ info->st_dev = sb.st_dev;
+ info->st_ino = sb.st_ino;
+ info->monitor = iter;
+
+ return iter ? 1 : 0;
+}
+
+/* mutt_monitor_add: add file monitor from BUFFY, or - if NULL - from Context.
+ *
+ * return values:
+ * 0 success: new or already existing monitor
+ * -1 failed: no mailbox, inaccessible file, create monitor/watcher failed
+ */
+int mutt_monitor_add (BUFFY *buffy)
+{
+ MONITORINFO info;
+ uint32_t mask;
+ int descr;
+
+ descr = monitor_resolve (&info, buffy);
+ if (descr != RESOLVERES_OK_NOTEXISTING)
+ return descr == RESOLVERES_OK_EXISTING ? 0 : -1;
+
+ mask = info.isdir ? INOTIFY_MASK_DIR : INOTIFY_MASK_FILE;
+ if ((INotifyFd == -1 && monitor_init () == -1)
+ || (descr = inotify_add_watch (INotifyFd, info.path, mask)) == -1)
+ {
+ dprint (2, (debugfile, "monitor: inotify_add_watch failed for '%s', errno=%d %s\n", info.path, errno, strerror(errno)));
+ return -1;
+ }
+
+ dprint (3, (debugfile, "monitor: inotify_add_watch descriptor=%d for '%s'\n", descr, info.path));
+ monitor_create (&info, descr);
+ return 0;
+}
+
+/* mutt_monitor_remove: remove file monitor from BUFFY, or - if NULL - from Context.
+ */
+void mutt_monitor_remove (BUFFY *buffy)
+{
+ MONITORINFO info;
+
+ if (monitor_resolve (&info, buffy) != RESOLVERES_OK_EXISTING
+ || (!buffy && mutt_find_mailbox (Context->realpath)))
+ return;
+
+ inotify_rm_watch(info.monitor->descr, INotifyFd);
+ dprint (3, (debugfile, "monitor: inotify_rm_watch for '%s' descriptor=%d\n", info.path, info.monitor->descr));
+
+ monitor_delete (info.monitor);
+ monitor_check_free ();
+}
diff --git a/monitor.h b/monitor.h
new file mode 100644
index 00000000..47478852
--- /dev/null
+++ b/monitor.h
@@ -0,0 +1,43 @@
+/*
+ * Copyright (C) 2018 Gero Treuer <g...@70t.de>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+ */
+
+#ifndef MONITOR_H
+#define MONITOR_H
+
+typedef struct monitor_t
+{
+ struct monitor_t *next;
+ char *mh_backup_path;
+ dev_t st_dev;
+ ino_t st_ino;
+ short magic;
+ int descr;
+}
+MONITOR;
+
+WHERE int MonitorFilesChanged INITVAL (0);
+WHERE int INotifyFd INITVAL (-1);
+WHERE MONITOR *Monitor INITVAL (NULL);
+
+#ifdef _BUFFY_H
+int mutt_monitor_add (BUFFY *b);
+void mutt_monitor_remove (BUFFY *b);
+#endif
+int mutt_monitor_poll ();
+
+#endif /* MONITOR_H */
