Paul Frields' slides about his notebook's email setup show quite some similarities with my own one. The major differences are:

• I moved from offlineimap to mbsync, mostly for performance reasons.
• Instead of having a full-fledged MTA running on my notebook, I prefer a small send-only MTA for the sake of small footprint and ease of configuration.

Here's how the setup is done in particular:

When it comes to using the same email account on multiple machines, IMAP is a great benefit. It allows for every box to see every incoming mail and housekeeping tasks such as deleting old mails or reorganizing them into subfolders will automatically be visible from the other machines, as well. By storing sent mail in an IMAP folder the same applies to outgoing mail, too.

The only downside with typical IMAP usage is that the task of email synchronization and caching has to be done by the email client (MUA). This naturally involves delays during client usage, and often clients are not really usable without direct connectivity to the IMAP server.

My personal MUA of choice is mutt, which suffers these issues, too. Especially since handling mail in local maildirs is strikingly fast, the added delay of having to sync with IMAP feels much higher than with other email clients.

In order to overcome this, an asynchronous approach serves well: A dedicated instance covers the task of bidirectional synchronization between remote IMAP and local maildirs, while mutt operates exclusively on the latter.

Using mutt's SMTP support involves surprisingly similar issues: Without direct connectivity, no mail sending is possible at all (of course not, but this can be directly experienced by mutt first hanging for a while and then giving up with an error message). But even if connectivity is given, any delay and/or bandwidth limitation on the line to the MTA is directly appreciable.

Here, an asynchronous approach is chosen as well: A dedicated instance takes care of queueing and ultimately delivering the email to be sent and mutt returns control to the user as soon as it has dispatched the job.

There are a number of tools to achieve this, but the only two which proved usable to me were offlineimap and mbsync. Of those the latter is superior, not least due to the fact that it is written in C.

mbsync is configured on a per user basis in $HOME/.mbsyncrc. In order to understand it's semantics, a number of concepts have to be known by the user:

• A store is basically anything that is able to keep emails, like e.g. an mbox, a maildir or even an IMAP account. mbsync defines specific store types for these (except mbox, but who uses that anyway?) which hold type-dependent information.

• A channel defines a path of synchronisation between two stores, optionally limited to a specific scope within both.

With these two in mind, let's start by defining two stores. The first one defines the local maildir:

MaildirStore local
Path ~/
Inbox ~/Maildir/
SubFolders Maildir++

The way this is defined is a bit unintuitive: Instead of setting Path to the actual maildir path, it is set to it's parent and Inbox is set to point to the top-most maildir. I don't really remember why, but this was important to get my setup working. The second store defines the remote IMAP:

IMAPStore remote
Host mail.nwl.cc
User myself
SSLType IMAPS

Obviously, this store defines the remote host running the IMAP service, the user name to use for authentication and that IMAPS should be used. If not using SASL authentication, one might want to additionally define the password using the Pass option.

With both endpoints being defined, channels can be declared. The following defines two channels between those peers: One for the inbox only, the other for all subfolders. This way synchronisation of the INBOX can be triggered separately from the subfolders, which becomes useful later on:

Channel inbox
Master :remote:INBOX/
Slave :local:Maildir/

Channel folders
Master :remote:
Slave :local:Maildir/
Patterns * !INBOX !Chats !Contacts !Emailed\ Contacts !Junk

A few things are noteworthy about the used semantics:

• Each channel defines which of the peers is master and which is slave. This is important when defining how synchronisation should take place. Apart from that, these two keywords are interchangeable.

• The IMAP server exposes the inbox as a subfolder on it's own, therefore the first channel specifies this explicitly.

• The second channel makes use of the Patterns keyword for two purposes:
  1. The asterisk (*) sign tells mbsync to recurse into subfolders.
  2. The exclamation mark (!) prefixed folder names are excluded from synchronisation. In this case the remote is a Zimbra instance which seems to export (uninteresting) non-email data via IMAP, too.

Finally, the above store and channel definitions are prepended by a few global statements:

Expunge Both
Create Both
Remove Both

SyncState *

The Expunge, Create and Remove statements control the synchronization process. Instead of Both one may also pass one of Master, Slave or None. Using the latter for Expunge is recommended measure against unwanted loss of data during testing.

SyncState defines where the synchronization meta data should be kept. The asterisk (*) makes mbsync save it's meta data in the slave's store itself.

To manually trigger synchronization of a defined channel, a direct call to mbsync suffices:

mbsync <channel_name>

To integrate this nicely into the rest of my setup, I have written a shell script which provides a few more goodies:

• Run indefinitely.
• Before every round of synchronization, try to ping the defined IMAP host (see Host directive above) or go to sleep for 10sec then try again.
• Synchronize the inbox every 10sec, the IMAP folders just once every 100sec.
• After every folder sync run, fetch a list of existing maildir subfolders into $HOME/.mutt/mbsyncloop.mailboxes in a format suitable for including into $HOME/.muttrc.

mbsyncloop

#!/bin/bash
mailboxes="${HOME}/.mutt/mbsyncloop.mailboxes"
 
do_mbsync() { # (channel)
	echo -e "synchronising $1 ..."
	mbsync $1 && echo -e "... done" || echo -e "... failed"
}
 
get_mailboxes() { # (maildir)
	echo -n "mailboxes \"+\""
	find "$1" -type d -name cur | while read path; do
		mbox="$(basename $(dirname $path))"
		[[ $mbox == "."* ]] || continue
		mbox="+$mbox"
		echo -n " \"$mbox\""
	done
	echo ""
}
 
update_mailboxes() {
	mboxes="$(get_mailboxes "$1")"
	[[ -f $mailboxes ]] || {
		echo "initially creating $mailboxes"
		echo -n $mboxes > "$mailboxes"
		return
	}
	old_mboxes="$(<$mailboxes)"
	[[ "$mboxes" == "$old_mboxes" ]] || {
		echo "updating $mailboxes"
		echo -n $mboxes > "$mailboxes"
	}
}
 
imaphost="$(awk '/^Host /{print $2}' ~/.mbsyncrc)"
 
# sync INBOX every 10 seconds,
# sync subfolders every minute
cnt=0
while true; do
	ping -c 1 $imaphost >/dev/null 2>&1 || {
		echo "Host $imaphost not responding"
		sleep 10
		continue
	}
	do_mbsync inbox
	[[ $((cnt % 10)) -eq 0 ]] && {
		cnt=0
		do_mbsync folders
		update_mailboxes "${HOME}/.maildir"
	}
	((cnt++))
	sleep 10
done

My notebook does not receive email from external hosts. For local delivery to maildirs, procmail serves well as simple alternative. The only thing left for an MTA on my notebook to do is to get my user's email out to the right MX using appropriate credentials. And all that in background, without disturbing my workflow in case something (Wifi, uplink, VPN, MX, etc.) does not work as it should.

Back then when I was searching for a relay-only MTA with queueing support, I didn't find any which felt comfortable. Therefore I sticked to esmtp I was using already and wrote a little shell-wrapper implementing the desired enhancement. This shell wrapper has been accepted into the official code repository back in 2007, but sadly the whole project was orphaned in 2011. Though, Gentoo luckily still ships it. Probably I should fork the project and revive it by implementing queueing support in C. Anyway, here's how to set things up using the latest release 1.2:

The configuration file is quite straightforward. The only thing worth mentioning in beforehand is esmtp's support for identities: These allow to choose a different SMTP upstream based on sending address. Here's the config I use:

mda = '/usr/bin/formail -a "Date: `date -R`" | /usr/bin/procmail -d %T'
 
hostname = smtp.corp.example.com
force reverse_path = myself@example.com
 
identity = myself@nwl.cc
	preconnect = "ssh -f -L 25025:localhost:25 nwl.cc 'sleep 5'"
	hostname = localhost:25025
	force reverse_path = myself@nwl.cc

The first line (mda) defines the method for local delivery, i.e. for recipients not containing an @-sign. Piping through formail adds the useful Date header, which is otherwise taken care of by the external MTA.

The default delivery method is via smtp.corp.example.com which requires no authentication (only reachable via VPN). Whatever From address I specify, esmtp sanitizes the reverse path for me.

A second identity defines my private mail setup. It establishes an SSH tunnel prior to connecting, which is convenient for me as it relieves me from having to specify login credentials here and SSH Agent is running anyway.

Now for queueing support: The whole magic is implemented by my shell wrapper, esmtp-wrapper. I have it installed in $HOME/bin with appropriate symlinks (it changes it's operation mode depending the name by which it was called):

% find bin -lname esmtp-wrapper
bin/mailq
bin/deliver
bin/sendmail

The commands to what their name suggests: mailq lists the local mail queue, deliver triggers mail delivery and sendmail just pretends to be sendmail itself. Queueing is implemented as follows:

Upon invocation, sendmail writes the parameters it was called with along with the mail body passed via stdin into a temporary directory below $HOME/.esmtp_queue and then calls itself as deliver in background.

Upon invocation, deliver first checks if it has been called by sendmail and in that case sleeps 5sec before doing anything (this is meant to catch mass mail delivery like e.g. git-send-email typically does). Then it continues to browse through any directories below $HOME/.esmtp_queue and calls esmtp exactly like sendmail has been called before. If that succeeds, the temporary directory is deleted, otherwise it's kept for a later try.

If initial mail delivery fails, it has to be retried later. Since esmtp does not run as a daemon, there is no automatic way to achieve this. Instead, a simple cron job serves well:

*/10 * * * * /bin/ping -c1 smtp.corp.example.com >/dev/null 2>&1 && $HOME/bin/deliver

This checks connectivity every 10min and calls deliver if given.

The last piece of the puzzle is mutt. It's own setup is trivial, as it actually does not know about mbsync at all. And for esmtp, all there is to specify is the path to sendmail:

set sendmail=~/bin/sendmail

What is more interesting is how to conveniently run mutt and mbsyncloop together. For that purpose, I have a dedicated .screenrc:

source .screenrc
screen -t SYNC	0 mbsyncloop
screen -t MUTT	-h 0 1 mutt_then_killall_mbsyncloop
bind c screen mutt
bind ^c screen mutt

When screen is invoked passing this config via it's -c parameter, it will automatically create two windows:

• Window 0 runs mbsyncloop,
• Window 1 has no scrollback buffer (-h 0) and calls a wrapper to mutt with an admittedly quite verbose name. Scrollback is not needed since mutt is ncurses-based.

Finally, it remaps the c command key which is normally used to create a new screen window to do just the same but automatically run another instance of mutt. This makes it very convenient to quickly fire up a secondary mutt to search mails or whatever in case the first one should be occupied (e.g. with composing an email).

Finally, the last puzzle piece's stud is that oddly named wrapper. Luckily, it does just what it's name (subtly) suggests:

mutt_then_killall_mbsyncloop

#!/bin/bash
 
mutt; killall mbsyncloop

The idea here is that once the initial instance of mutt ends, it also kills the running mbsyncloop instance and (in case no other windows (read: mutt instances) are running, the screen session ends.

• esmtp project page at sourceforge
• isync, the project containing mbsync