Category Archives: mu4e

Fixing duplicate UID errors when using mbsync and mu4e

This is a particularly niche post but hopefully will save someone the time I spent trying to solve this problem.

I have migrated from offlineimap to mbsync (part of isync) for synchronising my local email with Gmail. mbsync is much faster and I’ve been happy with the switch but I kept encountering error messages from mbsync about duplicate UIDs. The form of the message was Maildir error: duplicate UID 12 (or some other number).

After a lot of digging around I found the solution was in the mbsync and mu4e manuals (of course). The problem is caused because mu4e does not by default rename files when moving them to a new directory and this then causes problems for mbsync. This is easily fixed by setting

(setq mu4e-change-filenames-when-moving t)

Tweaking email contact completion in mu4e

The excellent emacs email client mu4e has very good completion of email contacts, which are ranked in a smart way. I have made a few tweaks to this to better suit my needs which I’ll describe below. In a nutshell, they let me

  • add favourites to the start of the completion list
  • offer a list of contacts as soon as I compose an email
  • insert a contact anywhere

For my favourite contacts, I create a text file with contacts in the form

First Last <>

I then create a list of contacts by appending the sorted list of mu4e contacts to this list of favourites and removing duplicates. This forms the final contact list that is offered for completion.

The favourite list used to be more useful before the sorting of contacts in mu4e was improved in the last release. It still has one nice use though, which is that I can specify the name I want to appear for each contact, which otherwise mu4e takes from the message header. This is useful since my email templates make use of this name to address the email, but sometimes these are unhelpful. For example “B Maughan <>” in the default mu4e contact list would cause my template to start “Hi B,” but if I put “Ben Maughan <>” in my favourites file, this is the one that will end up in the TO field and so I will get a nice “Hi Ben,” in my template.

I wrap this in a function using the ivy completion library, based on an example by Jon Kitchin that used helm, and bind it to S-TAB so that when I hit SHIFT and TAB in the TO field of an email I get this tweaked version of the address completion.

;;need this for hash access
(require 'subr-x)

;;my favourite contacts - these will be put at front of list
(setq bjm/contact-file "/full/path/to/fave-contacts.txt")

(defun bjm/read-contact-list ()
  "Return a list of email addresses"
    (insert-file-contents bjm/contact-file)
    (split-string (buffer-string) "\n" t)))

;;ivy contact completion
;;based on
(defun bjm/ivy-select-and-insert-contact (&optional start)
  ;;make sure mu4e contacts list is updated - I was having
  ;;intermittent problems that this was empty but couldn't see why
  (let ((mail-abbrev-mode-regexp mu4e~compose-address-fields-regexp)
        (eoh ;; end-of-headers
           (goto-char (point-min))
           (search-forward-regexp mail-header-separator nil t)))
        ;;append full sorted contacts list to favourites and delete duplicates
         (delq nil (delete-dups (append (bjm/read-contact-list) (mu4e~sort-contacts-for-completion (hash-table-keys mu4e~contacts)))))))
    (when (and eoh (> eoh (point)) (mail-abbrev-in-expansion-header-p))
      (let* ((end (point))
              (or start
                    (re-search-backward "\\(\\`\\|[\n:,]\\)[ \t]*")
                    (goto-char (match-end 0))
              (ivy-read "Contact: "
                        :re-builder #'ivy--regex
                        :sort nil
                        :initial-input (buffer-substring-no-properties start end))))
        (unless (equal contact "")
          (kill-region start end)
          (insert contact))))))

;;bind it
(define-key mu4e-compose-mode-map (kbd "<S-tab>") 'bjm/ivy-select-and-insert-contact)

Next I add this function to the hook that runs when I compose an email in mu4e, which launches me straight into address completion:

;;launch automatically
(add-hook 'mu4e-compose-mode-hook 'bjm/ivy-select-and-insert-contact)

Finally, here is a function that lets you insert a contact from your list anywhere in your email (not just the address fields). It will also work in any other buffer once you have started mu4e for the first time to initialise the contacts list.

;;ivy contacts for use anywhere
;;based on
(defun bjm/ivy-select-and-insert-contact-anywhere ()
  (let (contacts-list contact)
    ;;append full sorted contacts list to favourites and delete duplicates
    (setq contacts-list
          (delq nil (delete-dups (append (bjm/read-contact-list) (mu4e~sort-contacts-for-completion (hash-table-keys mu4e~contacts))))))
    (setq contact
          (ivy-read "Contact: "
                    :re-builder #'ivy--regex
                    :sort nil))
        (unless (equal contact "")
          (insert contact))))

Email attachment reminders in mu4e

Continuing my series of posts on using mu4e for emails in emacs, I wanted to duplicate the functionality of other email clients that warn you if you appear to have forgotten an attachment when you send an email.

My solution is decidedly lo-fi, but it does the trick. It is based on this post (coincidentally by the author of mu4e), which shows how to create highlighting for arbitrary keywords in emacs. This is a versatile trick and with the simple code below, any strings in a mu4e composition buffer that match “attach”, “pdf” or “file” get a nice red on yellow highlight.

;; attachment reminder based on
(set-face-attribute 'font-lock-warning-face nil :foreground "red" :weight 'bold :background "yellow")
(add-hook 'mu4e-compose-mode-hook
          (defun bjm/mu4e-highlight-attachment ()
            "Flag attachment keywords"
            (font-lock-add-keywords nil
                                    '(("\\(attach\\|pdf\\|file\\)" 1 font-lock-warning-face t)))))

It’s not pretty but it works for me! I’d prefer something that checked my email to see if it actually had an attachment and then warned me when I tried to send if there was no attachement there. I’ve not had time to work on that though, but if you know of anything similar, let me know!


There were some helpful suggestions in the comments for improved methods that prompt the user if they try to send a mail without an attachment. mbork has a nice clear example on their website.

Email templates in mu4e with yasnippet

This is the second in a series of posts on using mu4e for email in emacs. When I moved from thunderbird to mu4e, there were some thunderbird extensions that I missed. One was quicktext which I used to create simple email templates for common sorts of emails. This is easily replicated in emacs using yasnippet.

As a simple example, I made a snippet to expand the key all into

Hi all,



The snippet is

# -*- mode: snippet -*-
# name: hi all
# key: all
# --
Hi all,



This snippet is saved in the message-mode subdirectory of my snippets directory, since message-mode is the major mode for email composition in mu4e.

We can be a bit smarter than this with a snippet that takes the name of the email recipient and adds that to the template. The following is close to binchen’s instructions with a few small modifications. There are two parts to this: a snippet that expands out as normal, and a lisp function that it calls when it expands to extract the recipient’s name.

First, the lisp function

;; function to return first name of email recipients
;; used by yasnippet
;; inspired by
(defun bjm/mu4e-get-names-for-yasnippet ()
  "Return comma separated string of names for an email"
  (let ((email-name "") str email-string email-list email-name2 tmpname)
      (goto-char (point-min))
      ;; first line in email could be some hidden line containing NO to field
      (setq str (buffer-substring-no-properties (point-min) (point-max))))
    ;; take name from TO field - match series of names
    (when (string-match "^To: \"?\\(.+\\)" str)
      (setq email-string (match-string 1 str)))
    ;;split to list by comma
    (setq email-list (split-string email-string " *, *"))
    ;;loop over emails
    (dolist (tmpstr email-list)
      ;;get first word of email string
      (setq tmpname (car (split-string tmpstr " ")))
      ;;remove whitespace or ""
      (setq tmpname (replace-regexp-in-string "[ \"]" "" tmpname))
      ;;join to string
      (setq email-name
            (concat email-name ", " tmpname)))
    ;;remove initial comma
    (setq email-name (replace-regexp-in-string "^, " "" email-name))

    ;;see if we want to use the name in the FROM field
    ;;get name in FROM field if available, but only if there is only
    ;;one name in TO field
    (if (< (length email-list) 2)
        (when (string-match "^\\([^ ,\n]+\\).+writes:$" str)
          (progn (setq email-name2 (match-string 1 str))
                 ;;prefer name in FROM field if TO field has "@"
                 (when (string-match "@" email-name)
                   (setq email-name email-name2))

This function takes the first name of the email recipient from the “To:” field of the message. It also looks for a name in the “Joe Bloggs writes:” text that mu4e generates when you reply to an email. This is populated from the “From:” field of the email being replied to and sometimes gives a better match for the name. The function compares the two name strings and prefers the “To:” name unless it contains an “@” in which case it chooses the “From:” name (this is the addition I made to binchen‘s version). You could make this more sophisticated but it works pretty well for me as is.

Now we need a snippet to expand:

# -*- mode: snippet -*-
# name: dear name
# key: dear
# --
Dear ${1:`(bjm/mu4e-get-names-for-yasnippet)`},


Best wishes,

Note how our function is straightforwardly called by the snippet to give “Dear NAME”.

Hopefully this gives you some ideas of how to make useful email templates with yasnippet. In future posts I’ll talk about how I have added attachment reminders and delayed sending features to mu4e.

Update 11/3/2016

I updated the code to give a comma separated list of names in the case of more than one recipient.

Master your inbox with mu4e and org-mode

In the following I will put forward my philosophy on handling emails and then show how this is realised in emacs using mu4e and org-mode.

I couple of years ago I read an article by the economist Tim Harford which hugely influenced the way I handle my emails. The ideas in the article are not unique but they really struck a chord with me. My email philosophy can be distilled down to one key concept:

your inbox is not a todo list

Like many people I used to keep emails in my inbox as a way of reminding me of something I needed to do, but the fact is that an inbox is a rubbish todo list.

I also had folders for putting emails in and I would occasionally have a painful cleanout of my towering inbox, agonising over which folder to put an email in, or whether I should create a new folder for it. No more! As long as your email programme has a good search, then it is quicker to search than to use a filing system.

Now when I check my emails, I do one of the following

  • delete if it is rubbish
  • read and delete if it is not something I’ll need to revisit
  • read and archive if it is something I might need to look up again
  • reply and archive if it is something that will take less than a couple of minutes to reply to and I have the time
  • add to todo list and archive if it is something that requires an action or just needs a longer reply than I have time to write

To use this system effectively, all you really need is: (i) an email client with a good search function so you can archive all mail in the same folder and not worry about filing it neatly, and (ii) a good system for adding tasks from your emails to a todo list.

The mu4e email client in emacs, combined with org-mode for todo organisation is the perfect way to do both of these things. There is very good documentation on how to set up mu4e on the project web page, including configuring it to work well with gmail, so I won’t go over that here. What I will say is that mu4e is built on mu, a powerful email indexer so it has all of your search needs covered.

Apart from searching, mu4e integrates very well with org-mode to make it seamless to generate todo items from emails. To set this up, add the following to your emacs config file

;;store org-mode links to messages
(require 'org-mu4e)
;;store link to message if in header view, not to header query
(setq org-mu4e-link-query-in-headers-mode nil)

Now update your org-mode capture template to something like this

(setq org-capture-templates
      '(("t" "todo" entry (file+headline "~/" "Tasks")
         "* TODO [#A] %?\nSCHEDULED: %(org-insert-time-stamp (org-read-date nil t \"+0d\"))\n%a\n")))

This looks like the version we had before, but the extra %a adds a link to the file you are visiting when you invoke the capture template.

The beauty of this is that hitting C-c c t now generates a todo item that contains a link to the email you are currently viewing. So you have zero friction in creating a todo item to e.g. reply to an email by a certain deadline, and you can happily archive that email knowing that clicking the link in the todo item will take you directly back to it.

I moved from thunderbird to mu4e a couple of months ago and really haven’t looked back. The things I missed at first were some of the extensions I was using to: create email templates; remind me about attachments; and add a delay to outgoing email so that I could have an “undo send” functionality. Happily I’ve found solutions to all of these in mu4e and I’ll be covering them in future posts.