How I do email (at home)

I thought I might write something up on how I’ve been doing email both at home and at work. I very much on purpose keep the two completely separate, and have slightly different use cases for both of them.

For work, I do not want mail on my phone. For personal mail, it turns out I do want this on my phone, which is currently an Android phone. Since my work and personal email is very separate, the volume of mail is really, really different. Personal mail is maybe a couple of dozen a day at most. Work is… orders of magnitude more.

Considering I generally prefer free software to non-free software, K9 Mail is the way I go on my phone. I have it set up to point at the IMAP and SMTP servers of my mail provider (FastMail). I also have a google account, and the gmail app works fine for the few bits of mail that go there instead of my regular account.

For my mail accounts, I do an INBOX ZERO like approach (in reality, I’m pretty much nowhere near zero, but today I learned I’m a lot closer than many colleagues). This means I read / respond / do / ignore mail and then move it to an ARCHIVE folder. K9 and Gmail both have the ability to do this easily, so it works well.

Additionally though, I don’t want to care about limits on storage (i.e. expire mail from the server after X days), nor do I want to rely on “the cloud” to be the only copy of things. I also don’t want to have to upload any of past mail I may be keeping around. I also generally prefer to use notmuch as a mail client on a computer.

For those not familiar with notmuch, it does tags on mail in Maildir, is extremely fast and can actually cope with a quantity of mail. It also has this “archive”/INBOX ZERO workflow which I like.

In order to get mail from FastMail and Gmail onto a machine, I use offlineimap. An important thing to do is to set “status_backend = sqlite” for each Account. It turns out I first hacked on sqlite for offlineimap status a bit over ten years ago – time flies. For each Account I also set presynchook = ~/Maildir/maildir-notmuch-presync (below) and a postsynchook = notmuch new. The presynchook is run before we sync, and its job is to move files around based on the tags in notmuch and the postsynchook lets notmuch catch any new mail that’s been fetched.

My maildir-notmuch-presync hook script is:

#!/bin/bash
notmuch search --output=files not tag:inbox and folder:fastmail/INBOX|xargs -I'{}' mv '{}' "$HOME/Maildir/INBOX/fastmail/Archive/cur/"

notmuch search --output=files folder:fastmail/INBOX and tag:spam |xargs -I'{}' mv '{}' "$HOME/Maildir/INBOX/fastmail/Spam/cur/"
ARCHIVE_DIR=$HOME/Maildir/INBOX/`date +"%Y%m"`/cur/
mkdir -p $ARCHIVE_DIR
notmuch search --output=files folder:fastmail/Archive and date:..90d and not tag:flagged | xargs -I'{}' mv '{}' "$ARCHIVE_DIR"

# Gmail
notmuch search --output=files not tag:inbox and folder:gmail/INBOX|grep 'INBOX/gmail/INBOX/' | xargs -I'{}' rm '{}'
notmuch search --output=files folder:gmail/INBOX and tag:spam |xargs -I'{}' mv '{}' "$HOME/Maildir/INBOX/gmail/[Gmail].Spam/cur/"

So This keeps 90 days of mail on the fastmail server, and archives older mail off into month based archive dirs. This is simply to keep directory sizes not too large, you could put everything in one directory… but at some point that gets a bit silly.

I don’t think this is all the most optimal setup I could have, but it does let me read and answer mail on my phone and desktop (as well as use a web client if I want to). There is a bit of needless copying of messages by offlineimap under certain circumstances, but I don’t get enough personal mail for it to be a problem.

Without notmuch, I would simply delete your email

I have been using notmuch (http://notmuchmail.org/) as my email client for quite a while now. It’s fast. I don’t mean that everything happens instantly (some actions do take a bit longer than ideally they would), but with the quantity of mail I (and others) throw at it? Beats everything else I’ve ever tried.

I keep seeing people complain about not being able to keep up with various email loads and I am convinced it is because their mail client sucks.

I hear people go on about mutt…. well, I stopped using mutt when it would take two minutes to open some of my mail folders.

I hear some people talk about Evolution…. well, I stopped using Evolution when I realized that when it was rebuilding its index, I couldn’t use my mail client for at least twenty minutes.

Gmail…. well, maybe. Except that I don’t want all my mail to be sitting on google servers, I want to be able to work disconnected and the amount of time it would take to upload my existing mail makes it a non-starter (especially from the arse end of the internet – Australia). I also do not want email on my phone.

My current problem with notmuch? It just uses Maildir…. and this isn’t the most efficient for mail that never changes, some kind of archive format that is compressed would be great. Indeed, I started looking into this ages ago, but just haven’t had the spare cycles to complete it (and getting SSDs everywhere has not helped with the motivation).

Coming back from vacation, my mailbox had about 4,700 messages sitting in it. I’ve been able to get through just about all of them without blindly deleting mail. This is largely due to the great UI of notmuch for being able to quickly look at threads and then mark as read, quickly progressing to the next message. I can tag mail for action, I can very quickly search for email on an urgent topic (and find it) and generally get on with the business of getting things done rather than using an email program.