John Borwick

Emacs for email: HTML and replies

I've been using Emacs to read and send email for a couple of years now, specifically notmuch-emacs. I've improved my system in fits and starts. Here are a couple of things that have really helped me.

Sending HTML email with org-mime

Whenever I'm about to send an email, I "htmlize" it with C-c M-o: 

(use-package org-mime
  :config (setq org-mime-export-options '(:preserve-breaks t))
          (add-hook 'message-mode-hook
          (lambda ()
            (local-set-key "\C-c\M-o" 'org-mime-htmlize))))
(add-hook 'message-send-hook 'org-mime-confirm-when-no-multipart)

(Although I don't contribute much, I'm happy to say that I contributed org-mime-confirm-when-no-multipart.)

This ensures that people see HTML rather than org-mode content, assuming they prefer HTML.

Using org-mode to compose emails

If I'm sending a complicated email and I want to get help from org-mode formatting, I can smash C-c o: 

(use-package polymode
  :config
    (define-hostmode poly-mml-hostmode :mode 'notmuch-message-mode)
    (define-innermode jb-poly-org-innermode
      :mode 'org-mode
      :head-matcher "^--text follows this line--$"
      :tail-matcher "^THISNEVEREXISTS$"
      :head-mode 'host
      :tail-mode 'org-mode)
    (define-polymode poly-org-mode
      :hostmode 'poly-mml-hostmode
      :innermodes '(jb-poly-org-innermode))
    (add-hook 'mml-mode-hook
      (lambda () (local-set-key (kbd "C-c o") #'poly-org-mode)))
    )

This changes everything in the buffer after --text follows this line-- to be org-mode. The only complication is I need to jump to the header (e.g. with M-<) to use mail commands.

Reducing cruft in my email replies

Today I finally decided to address some of my email reply cruft—things like lots of blank lines in my replies, or multiple copies of my signature. I've been thinking about this for probably 15 years ever since I tried to build a JavaScript processor for Thunderbird. Part of my success is figuring out that I don't want to build this processor in Emacs! In my case, I want to use Python.

There's two parts:

Part #1: Parser program

This program is a work in progress, but conceptually it takes stdin, messes with it, and whatever's in stdout is what the reply will be. stdin is designed to be the body of the reply message that notmuch generates, sent via Emacs. 

#!/bin/python
"""
This takes an Emacs email buffer (starting with the body!) and
makes the reply look better:
1. removes my signature
2. removes multiple blank reply lines
3. removes [cidimage:]
This is designed to be used through shell command on region.
"""
import logging
import re
import sys

# logger = logging.basicConfig(level=logging.DEBUG)


SIG_BEGIN_RE = re.compile('^>+\s–\s*$')
SIG_END_RE = re.compile('^>+\s*From')
REPLY_END_RE = re.compile('^$')
REPLY_START_RE = re.compile('^>')
BLANK_REPLY_LINE = re.compile('^>+\s*$')
CID_IMAGE_LINE = re.compile(
    '^>+\s*'                    # Reply line
    '\[cid[^\]]+\]'             # [cid.*]
    '\s*$',                     # that's it
    re.VERBOSE)

def is_signature(text_to_check):
    text_to_parse = ''
    last_line = ''
    for line in text_to_check.split('\n'):
        if not line:
            continue
        line += '\n'
        if not BLANK_REPLY_LINE.match(line):
            last_line = line
            text_to_parse += line

    logging.debug("Checking '%s' for sig", text_to_parse)
    if text_to_check.count('\n') > 5:
        return False

    return True


def strip_message(msg_lines):
    """
    Initial goal is to strip multiple copies of my signature
    """
    msg = ''
    temp_buf = ''
    state = 'NOT_YET_REPLY'
    last_line_blank = False
    for line in msg_lines:
        if state == 'END':
            msg += line
            continue

        if state == 'NOT_YET_REPLY':
            msg += line
            if REPLY_START_RE.match(line):
                state = 'IN_REPLY'
            continue

        # supercedes other reply states:
        if REPLY_END_RE.match(line):
            logging.debug("Reply end")
            state = 'END'
            msg += line
            continue

        if BLANK_REPLY_LINE.match(line):
            if last_line_blank is True:
                continue
            last_line_blank = True
        elif last_line_blank is True:
            last_line_blank = False

        if CID_IMAGE_LINE.match(line):
            continue

        if state == 'IN_REPLY':
            if SIG_BEGIN_RE.match(line):
                logging.debug("Found sig")
                state = 'IN_SIG'
                temp_buf = line
            else:
                msg += line
        elif state == 'IN_SIG':
            if SIG_END_RE.match(line):
                logging.debug("Signature ended")
                state = 'IN_REPLY'

                # check whether temp_buf is a signature
                if not is_signature(temp_buf):
                    msg += temp_buf
                msg += line
                temp_buf = ''
            else:
                temp_buf += line

    return msg


if __name__ == '__main__':
    raw_message = sys.stdin.readlines()
    stripped_message = strip_message(raw_message)
    print(stripped_message, end='')

Part 2: Using the parser

Now I need lisp code to automatically run the reply parser: 

(defun replace-body-with-reply-parser (&rest _)
   (interactive)
   (if (boundp 'reply-parser-program)
       (progn
        (message-goto-body)
        (call-process-region
          (point); START
          (point-max); END
          "python"
          t t t
         (expand-file-name reply-parser-program))
         (message-goto-body))))

 (advice-add 'notmuch-mua-new-reply :after
             #'replace-body-with-reply-parser)

 (setq reply-parser-program "~/bin/reply_parser.py")

Any time I'm trying to write lisp it's going to be a struggle, but I think this is a mostly OK way to do this. The above

  1. creates a function replace-body-with-reply-parser, and then
  2. it changes the function notmuch-mua-new-reply to call this function, using Emacs "advice"

Advice lets you change how functions work.

If reply-parser-program is unset, nothing happens; if it is set, then the reply's body is sent to the program and is replaced by stdout.

My overall approach to making these improvements is to make small improvements when I'm blocked or significantly challenged. By just continuing to use these tools for years and years, I eventually get to a system that works really well for me! :-)

Updated: