A workflow to quickly add photos to org-mode notes

I was at a conference this week and a colleague was making notes using Evernote on her laptop and taking photos of key slides on her phone which then appeared in her notes. Of course I was making my notes in org-mode but I was envious of this behaviour so decided to emulate it.

With the function below, I can take a photo on my phone and upload to google drive (I use Photo & Picture Resizer, but you could use anything you like to get the pictures onto your computer). Then with a single command in Emacs, I am prompted with a list of photos in the folder to which they are uploaded, with the most recent first. The selected image is then:

  1. Moved the same directory as my org-mode notes file
  2. Renamed based on the heading of the current section in my notes, with a numeric suffix if there is already a photo with that name
  3. Linked in the notes and then the image is displayed

Here is a demonstration:

insert-slide-image.gif

Here is the code:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; add image from conference phone upload                                 ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; use case is taking a photo of a slide in a conference and uploading
;; it to google drive or dropbox or whatever to get it on your
;; computer. You then want to embed it in an org-mode document by
;; moving it to the same folder and renaming according to the current
;; section of the org file, avoiding name clashes

;; required libraries
(require 'dash)
(require 'swiper)
(require 's)

;; start directory
(defvar bjm/conference-image-dir (expand-file-name "/path/to/image/upload/dir"))

(defun bjm/insert-conference-image ()
  "Insert image from conference directory, rename and add link in current file.

The file is taken from a start directory set by `bjm/conference-image-dir' and moved to the current directory, renamed and embedded at the point as an org-mode link. The user is presented with a list of files in the start directory, from which to select the file to move, sorted by most recent first."
  (interactive)
  (let (file-list target-dir file-list-sorted start-file start-file-full file-ext end-file end-file-base end-file-full file-number)
    ;; clean directories from list but keep times
    (setq file-list
          (-remove (lambda (x) (nth 1 x))
                   (directory-files-and-attributes bjm/conference-image-dir)))

    ;; get target directory
    (setq target-dir (file-name-directory (buffer-file-name)))

    ;; sort list by most recent
  ;; http://stackoverflow.com/questions/26514437/emacs-sort-list-of-directories-files-by-modification-date
  (setq file-list-sorted
        (mapcar #'car
                (sort file-list
                      #'(lambda (x y) (time-less-p (nth 6 y) (nth 6 x))))))

  ;; use ivy to select start-file
  (setq start-file (ivy-read
                    (concat "Move selected file to " target-dir ":")
                    file-list-sorted
                    :re-builder #'ivy--regex
                    :sort nil
                    :initial-input nil))

  ;; add full path to start file and end-file
  (setq start-file-full
        (expand-file-name start-file bjm/conference-image-dir))
  ;; generate target file name from current org section
  ;; (setq file-ext (file-name-extension start-file t))

  ;; my phone app doesn't add an extension to the image so I do it
  ;; here. If you want to keep the existing extension then use the
  ;; line above
  (setq file-ext ".jpg")
  ;; get section heading and clean it up
  (setq end-file-base (s-downcase (s-dashed-words (nth 4 (org-heading-components)))))
  ;; shorten to first 40 chars to avoid long file names
  (setq end-file-base (s-left 40 end-file-base))
  ;; number to append to ensure unique name
  (setq file-number 1)
  (setq end-file (concat
                  end-file-base
                  (format "-%s" file-number)
                  file-ext))

  ;; increment number at end of name if file exists
  (while (file-exists-p end-file)
    ;; increment
    (setq file-number (+ file-number 1))
    (setq end-file (concat
                    end-file-base
                    (format "-%s" file-number)
                    file-ext))
    )

  ;; final file name including path
  (setq end-file-full
        (expand-file-name end-file target-dir))
  ;; rename file
  (rename-file start-file-full end-file-full)
  (message "moved %s to %s" start-file-full end-file)
  ;; insert link
  (insert (org-make-link-string (format "file:%s" end-file)))
  ;; display image
  (org-display-inline-images t t)))

Multiple choices in yasnippets

I’ve written before about using yasnippet as a powerful text expansion and template tool in Emacs. In its basic form you type a keyword and press TAB to expand that into a template with active fields and then use TAB to move between the fields adding your content. I recently discovered that you can programme a snippet to contain multiple choices for fields.

For example, the following snippet uses the keyword test and inserts a template with the cursor initially in the first field and prompts the user to choose from the options “high”, “average” and “below average”. Selecting your choice and then hitting TAB takes you to the next field with options of “easy” and “hard”.

# -*- mode: snippet -*-
# name: test
# key: test
# --
Importance: ${1:$$(yas-choose-value '("high" "average" "below average"))}
Difficulty: ${2:$$(yas-choose-value '("easy" "hard"))}
$0

Insert internal org-mode links the ivy way

Org-mode is a fantastic way to organise information in simple text files. I often use internal links to other sections in a document for navigation, but I’ve found that there is not a great mechanism to quickly insert an internal link. I wanted an interface that would provide me a list of headlines in the document that I could use to select a link target, so I put together the functions below.

These simple functions leverage the fantastic ivy and worf packages, so install them first. Then put the code below into your emacs config file. Then, invoking M-x bjm/worf-insert-internal-link provides an ivy completion interface for the list of headlines in the document. Once you select the headline you want as the link target, the link is inserted for you.

;; use ivy to insert a link to a heading in the current document
;; based on `worf-goto`
(defun bjm/worf-insert-internal-link ()
  "Use ivy to insert a link to a heading in the current `org-mode' document. Code is based on `worf-goto'."
  (interactive)
  (let ((cands (worf--goto-candidates)))
    (ivy-read "Heading: " cands
              :action 'bjm/worf-insert-internal-link-action)))


(defun bjm/worf-insert-internal-link-action (x)
  "Insert link for `bjm/worf-insert-internal-link'"
  ;; go to heading
  (save-excursion
    (goto-char (cdr x))
    ;; store link
    (call-interactively 'org-store-link)
    )
  ;; return to original point and insert link
  (org-insert-last-stored-link 1)
  ;; org-insert-last-stored-link adds a newline so delete this
  (delete-backward-char 1)
  )

To eww or not to eww

Recent Emacs versions include a decent text based web browser called eww. There is a good example configuration here. I found that I like using eww for some things, but at other times I want to use a full GUI web browser (I like Firefox).

I set eww to be my default browser in Emacs with

(setq browse-url-browser-function 'eww-browse-url)

but then I create some wrapper functions for times when I want to use my system default browser (i.e. Firefox).

In mu4e emails, hitting g will now open a URL in eww, but with the following lines I can use G to open in Firefox:

;; open link in firefox rather than eww
(defun bjm/mu4e-view-go-to-url-gui ()
  "Wrapper for mu4e-view-go-to-url to use gui browser instead of eww"
  (interactive)
  (let ((browse-url-browser-function 'browse-url-default-browser))
    (mu4e-view-go-to-url)))
;; bind it
(define-key mu4e-view-mode-map (kbd "G") 'bjm/mu4e-view-go-to-url-gui)

In elfeed, hitting b opens an article in eww if that is set to be the default Emacs browser. With the following lines I can hit B to open an article in Firefox. Note that I had to resort to calling the Mac open command to open the URL with the system default browser. On linux systems the xdg-open command should do the same thing.

;; browse article in gui browser instead of eww
(defun bjm/elfeed-show-visit-gui ()
  "Wrapper for elfeed-show-visit to use gui browser instead of eww"
  (interactive)
  (let ((browse-url-generic-program "/usr/bin/open"))
    (elfeed-show-visit t)))

(define-key elfeed-show-mode-map (kbd "B") 'bjm/elfeed-show-visit-gui)

A shorter shortcut to capture todo tasks

Org mode is wonderful for managing todo lists. Usually to add a task, I would run org-capture using the default C-c c and then select my todo template by hitting t. I do this so often that I wanted a shorter shortcut, so I defined a function to call org-capture with the specific template I wanted and bound this to a simple shortcut (I used C-9 after removing numeric prefixes):

;; function to capture a todo
(defun bjm/org-capture-todo ()
  (interactive)
  "Capture a TODO item"
  (org-capture nil "t"))

;; bind
(define-key global-map (kbd "C-9") 'bjm/org-capture-todo)

Now C-9 takes me straight to my todo capture template.

An unobtrusive email monitor for mu4e on the Mac

This is not exactly an Emacs post, but some users of the mighty mu4e might find it useful. I don’t like big alerts to new emails that distract me from what I am doing, but I also don’t want to keep switching to my mu4e buffer to see if new emails have arrived. My solution is to have a small notification item in my Mac status bar to tell me how many emails I have.

It looks like this (the bit underlined in red):

email-counter2.png

The text @ 1/3/0/0 tells me that I have, respectively,

  • 1 unread email in my inbox.
  • 3 total emails in my inbox (i.e. 3 too many!).
  • 0 emails in my drafts folder. This is useful to keep an eye on in case I save a draft and then forget about it.
  • 0 emails in my outbox. This is my postfix email outbox and is useful to keep an eye on in case emails sent when I’m offline don’t get automatically sent when I reconnect.

To set this up I use shellwrangler which is a small app that embeds the output of a script in the status bar. This then calls the following perl script which generates the text. Shellwrangler updates at a specified interval so that is all there is to it.

#!/usr/bin/perl -w

###########################################################################
#
# Count number of emails in inbox, drafts and outbox for mail indexed
# with mu and sent with sendmail or equivalent
#
# by Ben Maughan
# http://www.pragmaticemacs.com
#
# Feel free to distribute, modify, whatever
#
###########################################################################

## total mails in inbox
chomp(my @tot=`/usr/local/bin/mu find maildir:/work/INBOX`);
## unread mails in inbox
chomp(my @unread=`/usr/local/bin/mu find maildir:/work/INBOX AND flag:unread`);
## drafts
chomp(my @drafts=`/usr/local/bin/mu find maildir:/work/[Gmail].Drafts`);

## number in outbox
chomp(my @mailq=`/usr/bin/mailq`);
my @outbox = grep /^[0-9A-F]{12}/, @mailq;

printf "@ %d/%d/%d/%d",$#unread+1,$#tot+1,$#drafts+1,$#outbox+1;

exit;

Reformatting Tabular Data

Sometimes in my research I need to extract tabular data from a pdf paper. I can copy and paste the table into an Emacs buffer but the data is generally not formatted in a usable way. Luckily Emacs has a wealth of tools to reformat this sort of data.

Here is an animated gif illustrating some tools I use to do this (of course there are lots of other ways to do the same thing).

reformat-table.gif

In the animation I use the following tools

  • C-x h to select the whole buffer.
  • C-c | to run org-table-create-or-convert-from-region to convert the region to an org table. This doesn’t get me all the way to where I want to be, but I find it helpful to see the data clearly.
  • M-S-<left> to delete some unwanted columns
  • mc/mark-next-like-this from multiple cursors to give me a cursor on each line (I bind this to M-.)
  • M-f to move forward by word
  • shrink-whitespace to remove whitespace (I bind this to M-SPACE)
  • C-c - to run org-table-insert-hline to add a nice horizontal line to my table

This restructures the data in the way I need, and I can now use org-table-export to export to other useful formats.

Save window layouts with ivy-view

The wonderful ivy library provides a command ivy-view which allows you to quickly bookmark the current arrangement of windows in your Emacs frame. The nice thing is that once you do this, the bookmarked arrangement then appears in your ivy-powered buffer switching list so changing back to the arrangement you had is as easy as switching buffers. This make a lightweight alternative to other mathods for managing window layouts.

To use this, just run ivy-push-view to store the current view, and optionally give it a name (a useful default we be offered). This will then be offered when you switch buffer using ivy-switch-buffer (which you are using automatically if you use ivy-mode). To make these ivy-views appear in your buffer list, you might need to set the option

(setq ivy-use-virtual-buffers t)

in you emacs config file.

Ivy author abo-abo gives an example on his blog post – take a look if this sounds useful.

Using email address lists in mu4e

To set up lists of multiple email multiple recipients with mu4e, you can make use of mail aliases. To do this, add lists to a file with the following format, with the word “alias” followed by the name you want for the list, followed by a space-separated list of email addresses:

alias jsmiths "John Q. Smith <none@example.com>" "Jane Q. Smith <email@address.com>"

You can either save this file as ~/.mailrc or put it anywhere and tell Emacs where to find it using e.g.

;;mail aliases for lists (address lists)
(setq mail-personal-alias-file (expand-file-name "/homeb/bjm/docs/mail-aliases"))

Now, when composing an email you can type the alias (“jsmiths” in our example) and hit space, and it will expand to the list of email addresses.

If you are using my improved address completion, then you can add those aliases to your bjm/contact-file to have them appear in your completion lists.

Make Emacs a bit quieter

The minibuffer is the area at the bottom of your Emacs frame where you interact with Emacs prompts, or type commands. It also doubles up as the echo area, where Emacs tells you useful things. However sometimes when Emacs is prompting me for an answer in the minibuffer, the prompt is hidden by a message so I can’t see what I’m being asked any more.

When this happens, I found it is almost always a message about a buffer being reverted or a file being auto-saved. These messages can be prevented from appearing in the echo area as follows.

The revert messages are easy to silence with the following setting:

;; turn off auto revert messages
(setq auto-revert-verbose nil)

For the auto-save messages, it is a bit trickier. Some solutions suggest advising do-auto-save which is the function that does the actual job of auto-saving (e.g. this stackexchange question). However, this doesn’t work for Emacs 24.4 and newer since the call to do-auto-save bypasses the advice, for technical reasons I don’t fully understand!

A work around is to switch off the built-in auto-save, and replace it with our own silent version:

;; custom autosave to suppress messages
;;
;; For some reason `do-auto-save' doesn't work if called manually
;; after switching off the default autosave altogether. Instead set
;; to a long timeout so it is not called.
(setq auto-save-timeout 99999)

;; Set up my timer
(defvar bjm/auto-save-timer nil
  "Timer to run `bjm/auto-save-silent'")

;; Auto-save every 5 seconds of idle time
(defvar bjm/auto-save-interval 5
  "How often in seconds of idle time to auto-save with `bjm/auto-save-silent'")

;; Function to auto save files silently
(defun bjm/auto-save-silent ()
  "Auto-save all buffers silently"
  (interactive)
  (do-auto-save t))

;; Start new timer
(setq bjm/auto-save-timer
      (run-with-idle-timer 0 bjm/auto-save-interval 'bjm/auto-save-silent))

Note that I found some strange behaviour that do-auto-save does not work if the built-in auto-save is switched off altogether using (setq auto-save-default nil), so instead I had to set it for a long timeout to prevent it from running.