Category Archives: files

Open files with the system default application

I’ve mentioned crux before; it’s a package providing a set of general-purpose useful commands. One that I use all the time is crux-open-with, which opens the file currently being visited (or the file at the point in a dired buffer) using the system default application for that filetype. It works on Mac or Linux, by using the open or xdg-open commands respectively.

I bind the command to C-c o, using the following code (which also binds the previously mentioned crux-move-beginning-of-line)

(use-package crux
  :bind (("C-c o" . crux-open-with)
         ("C-a" . crux-move-beginning-of-line)))

Find and open files from anywhere with helm-for-files

Helm is an extremely powerful package that provides a search interface that narrows the matches as you type. What makes it powerful is the range of back-ends that you can access through helm. I don’t use it for a lot of things (I slightly prefer ivy in general), but one of my favourites is helm-for-files. This gives you a search interface where you start typing the name of a file and it narrows a list of files from multiple useful sources: currently opened files; recent files; bookmarked files; files in the current directory; and files anywhere on your system using your system’s locate command. Basically, it find any file anywhere on your system, showing you the most likely matches first. Hitting RET to select a file opens it in Emacs as usual.

Here is how I install and configure helm for this purpose

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; helm                                                                   ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(use-package helm
  :ensure t
  :init
  (progn
    (require 'helm-config)
    ;; limit max number of matches displayed for speed
    (setq helm-candidate-number-limit 100)
    ;; ignore boring files like .o and .a
    (setq helm-ff-skip-boring-files t)
    ;; replace locate with spotlight on Mac
    (setq helm-locate-command "mdfind -name %s %s"))
  :bind (("C-x f" . helm-for-files)))

Since I use a Mac, I tell helm to use mdfind which is the command line interface to the spotlight indexer in place of the normal locate command. Finally, I bind the command to C-x f to serve as a memorable alternative for C-x C-f (the default interface for opening files).

Insert file name

Here is a simple function from the emacs wiki to insert the name of a file into the current buffer. The convenient thing is that it uses the normal find-file prompt with whatever your completion setting is, so it works very easily. I bind it to C-c b i and as the documentation says, by default it inserts a relative path but called with a prefix (C-u C-c b i) it inserts the full path.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; insert file name at point                                              ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; https://www.emacswiki.org/emacs/InsertFileName
(defun bjm/insert-file-name (filename &optional args)
  "Insert name of file FILENAME into buffer after point.

  Prefixed with \\[universal-argument], expand the file name to
  its fully canocalized path.  See `expand-file-name'.

  Prefixed with \\[negative-argument], use relative path to file
  name from current directory, `default-directory'.  See
  `file-relative-name'.

  The default with no prefix is to insert the file name exactly as
  it appears in the minibuffer prompt."
  ;; Based on insert-file in Emacs -- ashawley 20080926
  (interactive "*fInsert file name: \nP")
  (cond ((eq '- args)
         (insert (expand-file-name filename)))
        ((not (null args))
         (insert filename))
        (t
         (insert (file-relative-name filename)))))

;; bind it
(global-set-key (kbd "C-c b i") 'bjm/insert-file-name)

Auto Save and Backup Every Save

Emacs has two useful ways of protecting you from data loss. The first is auto save, which saves a copy of a file every so often while you are editing it. If some catastrophe caused you to close Emacs or shut down your machine without saving the file then you can use M-x recover-file to recover the file from its auto save. By default, the auto save files are saved in the same directory as the original file, and are given a name of the form #file#. This is fine for me, but you can configure this.

I like to make auto saves often, so I make the following tweaks to my emacs config file:

;; auto save often
;; save every 20 characters typed (this is the minimum)
(setq auto-save-interval 20)

When you save a file, the auto save file is deleted.

The other way Emacs protects you is to make backups of your files. By default the backup file is made in the same directory as the original with a name like file~. The way the backup works is that Emacs makes a copy of a file the first time you save it in an Emacs session. It only makes that one backup though, so this is not very useful if you keep your session running for a long time and want to recover an earlier version of a file.

The following code sets some general backup options and then configures Emacs to make a backup of a file every time you save it. The code builds on bits from here and here, and the comments should be quite self-explanatory.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; backup settings                                                        ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; https://www.emacswiki.org/emacs/BackupFiles
(setq
 backup-by-copying t     ; don't clobber symlinks
 kept-new-versions 10    ; keep 10 latest versions
 kept-old-versions 0     ; don't bother with old versions
 delete-old-versions t   ; don't ask about deleting old versions
 version-control t       ; number backups
 vc-make-backup-files t) ; backup version controlled files

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; backup every save                                                      ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; http://stackoverflow.com/questions/151945/how-do-i-control-how-emacs-makes-backup-files
;; https://www.emacswiki.org/emacs/backup-each-save.el
(defvar bjm/backup-file-size-limit (* 5 1024 1024)
  "Maximum size of a file (in bytes) that should be copied at each savepoint.

If a file is greater than this size, don't make a backup of it.
Default is 5 MB")

(defvar bjm/backup-location (expand-file-name "~/emacs-backups")
  "Base directory for backup files.")

(defvar bjm/backup-trash-dir (expand-file-name "~/.Trash")
  "Directory for unwanted backups.")

(defvar bjm/backup-exclude-regexp "\\[Gmail\\]"
  "Don't back up files matching this regexp.

Files whose full name matches this regexp are backed up to `bjm/backup-trash-dir'. Set to nil to disable this.")

;; Default and per-save backups go here:
;; N.B. backtick and comma allow evaluation of expression
;; when forming list
(setq backup-directory-alist
      `(("" . ,(expand-file-name "per-save" bjm/backup-location))))

;; add trash dir if needed
(if bjm/backup-exclude-regexp
    (add-to-list 'backup-directory-alist `(,bjm/backup-exclude-regexp . ,bjm/backup-trash-dir)))

(defun bjm/backup-every-save ()
  "Backup files every time they are saved.

Files are backed up to `bjm/backup-location' in subdirectories \"per-session\" once per Emacs session, and \"per-save\" every time a file is saved.

Files whose names match the REGEXP in `bjm/backup-exclude-regexp' are copied to `bjm/backup-trash-dir' instead of the normal backup directory.

Files larger than `bjm/backup-file-size-limit' are not backed up."

  ;; Make a special "per session" backup at the first save of each
  ;; emacs session.
  (when (not buffer-backed-up)
    ;;
    ;; Override the default parameters for per-session backups.
    ;;
    (let ((backup-directory-alist
           `(("." . ,(expand-file-name "per-session" bjm/backup-location))))
          (kept-new-versions 3))
      ;;
      ;; add trash dir if needed
      ;;
      (if bjm/backup-exclude-regexp
          (add-to-list
           'backup-directory-alist
           `(,bjm/backup-exclude-regexp . ,bjm/backup-trash-dir)))
      ;;
      ;; is file too large?
      ;;
      (if (<= (buffer-size) bjm/backup-file-size-limit)
          (progn
            (message "Made per session backup of %s" (buffer-name))
            (backup-buffer))
        (message "WARNING: File %s too large to backup - increase value of bjm/backup-file-size-limit" (buffer-name)))))
  ;;
  ;; Make a "per save" backup on each save.  The first save results in
  ;; both a per-session and a per-save backup, to keep the numbering
  ;; of per-save backups consistent.
  ;;
  (let ((buffer-backed-up nil))
    ;;
    ;; is file too large?
    ;;
    (if (<= (buffer-size) bjm/backup-file-size-limit)
        (progn
          (message "Made per save backup of %s" (buffer-name))
          (backup-buffer))
      (message "WARNING: File %s too large to backup - increase value of bjm/backup-file-size-limit" (buffer-name)))))

;; add to save hook
(add-hook 'before-save-hook 'bjm/backup-every-save)

Files are backed up to a customisable directory in subdirectories per-session once per Emacs session, and per-save every time a file is saved. You can specify a maximum file size for files that will be backed up, and a regular expression to specify files that will not be backed up. I use the latter to avoid making backups of my emails by using the regular expression \\[Gmail\\] which matches emails I compose with mu4e. You can set this to nil if you want to disable this.

Edit files inside zip or tar archives

This is as easy as opening a .zip file in emacs. You will be shown a dired style file browser (in Zip-Archive mode) in which you can open files and edit and save them as normal, without having to extract them from the archive. You can also do a few other things to the files in the archive – use C-h m to see the help for the current major mode.

Similarly, opening a .tar (or even a compressed .tar.gz) archive will open the contents in Tar mode. This is very similar to the above, except that once you edit a file, you have to go back to the Tar mode buffer and save that to update the original archive file (this is automatic in Zip-Archive mode).

Super spotlight search with counsel

Inspired by abo-abo‘s post on using his excellent tools counsel, ivy and swiper to search the contents of files indexed by the recoll search tool, I tried to make something similar for spotlight on the Mac. My attempt is below.

Using counsel gives us incremental updates of the spotlight search results (accessed using its command line interface mdfind). When a match is selected, it is opened in emacs, and (unless it is a pdf) a swiper search is launched on the search string.

This works really nicely. The only problem I’ve had is that I wanted to sort the results to prioritise .org and .tex files, but my sort function is not being used correctly in ivy, but I can’t tell why. It gets passed the counsel prompt for more characters, but not the set of mdfind matches for sorting. My lisp skills have been exhausted at this point, but maybe someone else can see what I’ve done wrong! UPDATE: this problem was fixed with an update to ivy and counsel, and the sorting command below now works.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; counsel-spotlight                                                      ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Incrementally search the Mac spotlight database and open matching
;; files with a swiper search on the query text.
;; Based on http://oremacs.com/2015/07/27/counsel-recoll/

(require 'counsel)

;; Function to be called by counsel-spotlight
;; The onlyin option limits results to my home directory
;; and directories below that
;; mdfind is the command-line interface to spotlight
(defun counsel-mdfind-function (string &rest _unused)
  "Issue mdfind for STRING."
  (if (< (length string) 4)
      (counsel-more-chars 4)
    (counsel--async-command
     (format "mdfind -onlyin ~/ '%s'" string))
    nil))

;; Main function
(defun counsel-spotlight (&optional initial-input)
  "Search for a string in the mdfind database.
You'll be given a list of files that match.
Selecting a file will launch `swiper' for that file.
INITIAL-INPUT can be given as the initial minibuffer input."
  (interactive)
  (ivy-read "spotlight: " 'counsel-mdfind-function
            :initial-input initial-input
            :dynamic-collection t
            :sort t
            :action (lambda (x)
                      (when (string-match "\\(\/.*\\)\\'" x)
                        (let ((file-name (match-string 1 x)))
                          (find-file file-name)
                          (unless (string-match "pdf$" x)
                            (swiper ivy-text)))))))

;; Define my sort function
(defun bjm-counsel-mdfind-sort-function (x y)
  "Compare two files X and Y. Prioritise org then tex."
  (if (string-match "org$" x)
      t
    (if (string-match "tex$" x)
        (if (string-match "org$" y)
            nil
          t)
      nil)))

;; Add to list of ivy sorting functions
(add-to-list 'ivy-sort-functions-alist
             '(counsel-mdfind-function . bjm-counsel-mdfind-sort-function))

In an ideal world, I’d like to be able to interactively narrow the matches from mdfind with a second counsel on the filenames. The selected file would then open with a swiper search for the first search term. For example, I would like to

  1. Call M-x counsel-spotlight and enter caustic (or enough characters to give me useful results) to get a list of names of files which contain the text caustic somewhere inside them.
  2. Hit some keybinding and get a new counsel prompt and enter chandra to incrementally filter the list of filenames to those containing the string chandra in the filename.
  3. Select the file I want and hit RET to have emacs open that with a swiper search of my original query caustic.

Given the state of my lisp skills, this might take me a while, but it is nice to dream!

Update: A cheat to sort/filter results

Given that my perl skills are much better than my lisp skills, I cheated and wrote a perl wrapper for mdfind that filters and sorts the results for me. The code is below – feel free to use and modify and share. To use this, save the code to a file called bjm-mdfind in your $PATH, make sure it has executable permissions set, and modify the lisp code above to use it:

;; Function to be called by counsel-spotlight
;;
;; mdfind is the command-line interface to spotlight
;;
;; The onlyin option limits results to my home directory
;; and directories below that
;;
;; N.B. below this is replaced with a custom perl wrapper to sort
;; and filter the mdfind results
(defun counsel-mdfind-function (string &rest _unused)
  "Issue mdfind for STRING."
  (if (< (length string) 4)
      (counsel-more-chars 4)
    (counsel--async-command
     (format "bjm-mdfind '%s'" string))
     ;;(format "mdfind -onlyin ~/ '%s'" string))
    nil))

Here is the perl code:

#!/usr/bin/perl -w

###############################################################################
# bjm-mdfind
#
# wrapper for mdfind to filter and sort results
# written by Ben Maughan http://pragmaticemacs.com/
#
# $Id: bjm-mdfind,v 1.2 2015/08/06 11:21:37 bjm Exp $
###############################################################################

use strict;
use File::Basename;
use Getopt::Long;

my $scriptname = basename($0); # Strip away the leading directory names
my $runtime = localtime;

##########################################################################
# customise these options                                                #
##########################################################################
# limit search to this dir (recursively)
my $dir="~/";
# preferred order of file extensions
# others will appear later
my @order=(".org",".tex",".el",".txt",".dat",".pdf");
# exclude files matching these strings
my @exclude=("Library/Caches","Application Support");

###############################################################################
# Handle command line arguments

my $help;
my $man;
my $v=0; #set default verbosity
my $version = defined ((split / /, q/$Revision: 1.2 $/)[1]) ? (split / /, q/$Revision: 1.2 $/)[1] : 0;
my @args=@ARGV;
my $nargs=1; #number of required command line args

&Getopt::Long::Configure( 'bundling' );
GetOptions(
           'help|h' => \$help,
           'man|m'  => \$man,
           'verbose|v=i' => \$v,
) or die "ERROR: Invalid command line option $!";

if ($help||$man||$#ARGV<$nargs-1) { #print help
  # Load Pod::Usage only if needed.
  require "Pod/Usage.pm";
  import Pod::Usage;
  pod2usage(VERBOSE => 1) if $help;
  pod2usage(VERBOSE => 2) if $man;
  pod2usage(VERBOSE => 0, -message => "ERROR: not enough arguments use --help or --man for more help") if $#ARGV<0;
  pod2usage(VERBOSE => 0, -message => "ERROR: not enough arguments - your input was:\n    $scriptname @args") if $#ARGV<$nargs-1;
}

#check input
my $string=$ARGV[0];

###############################################################################
# Main part of program

if ($v > 0) {
  print <<EOF

------------------------$scriptname version $version------------------

Invocation was:
$scriptname @args

Runtime $runtime

EOF
}

chomp(my @out=`mdfind -onlyin $dir $string`);

## sort and filter
##print "$order[2]\n\n";

##filter
my $exc=join "|", @exclude;
@out = grep !/$exc/, @out;

##sort
my @out2;
foreach my $ext (@order) {
    my @tmp = grep /$ext$/, @out;
    print "###$ext\n###@tmp\n\n" if $v>1;
    push @out2, @tmp;
}
##everything else
my $inv=join "\$|", @order;
print "###$inv\n" if $v>1;
my @rest = grep !/$inv$/, @out;
print "###@rest\n" if $v>1;
push @out2, @rest;

##join
my $out=join "\n", @out2;

##print
print "$out\n";

#report successful completion
print "$scriptname completed successfully\n\n" if $v>0;

###############################################################################
# POD documentation

=head1 NAME

bjm-mdfind

=head1 SYNOPSIS

B<bjm-mdfind> [options] string

=head1 DESCRIPTION

wrapper for mdfind to filter and sort results

  string        - search query

=head1 OPTIONS

=over 4

=item B<-h, --help>

Prints out a brief help message.

=item B<-m, --man>

Prints out detailed help.

=item B<-v, --verbosity>

Control the amount of output B<(Default = 1)>

=back

=head1 VERSION

This is $RCSfile: bjm-mdfind,v $ $Revision: 1.2 $

=head1 AUTHOR

Ben Maughan <benmaughan@gmail.com>

=cut

###############################################################################
# $Log: bjm-mdfind,v $
# Revision 1.2  2015/08/06 11:21:37  bjm
# Summary: added URL
#
# Revision 1.1  2015/08/06 11:17:37  bjm
# Initial revision
#

Visualise and copy differences between files

To compare the contents of two text files, use M-x ediff-files and open the two files you want to compare. Emacs will then open one above the other in your main emacs window (N.B. a window is called a frame in emacs terminology), and also open a smaller window which says Type ? for help.

Press | to put the two files side by side in your main emacs frame, and then stretch your window nice and wide so you can see both files side by side clearly. Now click back on the small emacs window with the help text.

In your main window, you should see blocks of text that differ between the two buffers highlighted. Press n to move to the next difference and p to move to the previous difference. Press a to copy the text from the left-hand file to the right-hand file, or press b to copy text from the right-hand file to the left-hand file.

Press q to exit ediff and then save your files if you have modified them.

To make the experience a little smoother I add the following to my emacs config file:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; ediff                                                                  ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(require 'ediff)
;; don't start another frame
;; this is done by default in preluse
(setq ediff-window-setup-function 'ediff-setup-windows-plain)
;; put windows side by side
(setq ediff-split-window-function (quote split-window-horizontally))
;;revert windows on exit - needs winner mode
(winner-mode)
(add-hook 'ediff-after-quit-hook-internal 'winner-undo)

Ediff can do more than this, but this is my main use case – see the manual for more.

Dired: redirect symbolic links

We have previously looked at using dired for managing files and switching dired into writeable mode for renaming multiple files. Here is a related tip to redirect symbolic links in dired.

Suppose you moved some data from one location to another and ended up with a bunch of broken symbolic links, you can easily edit those links to point to the new path. In the example below, I have links pointing to files in /old/path and I want the to point to files in /new/dir instead, so I do the following

  • Use C-x d to enter dired and choose the directory with the files in
  • Use C-x C-q to turn dired into editing mode. You can then edit the file names by hand or
  • Use multiple cursors to edit the links in one go (you could also use e.g. M-% to do a query-replace). This changes the path of the links.
  • Use C-c C-c so apply the changes, or C-c C-k to cancel

This is illustrated in the following animation.

dired-links.gif

This is just great – it feels like magic every time!

Searching multiple files with rgrep

Use M-x rgrep to search for a string in multiple files. You will be prompted for a string, the files to search and the directory to start the search. So, for example, to search for the word “emacs” in all .txt files in my documents directory, I might use M-x rgrep and then answer emacs, *.txt, and ~/docs/.

rgrep will open a new buffer containing the results. The nice thing is that clicking on one of the matches will open that file in emacs and take you right to the matching line.

Note that rgrep is recursive and so will also search all directories below the one specified. For more control, try M-x grep and M-x grep-find.