Gnus is a mail user agent (MUA) and newsreader written in Emacs Lisp. It is famous for being very configurable, and for being one of the few MUAs that come with a decent editor to compose your messages (by virtue of actually running inside Emacs). I have switched between Gnus and Thunderbird as my primary MUA several times over the years and recently switched to Gnus again. This time, however, I made an additional resolution: should I encounter any problems or rough edges I would attempt to debug and fix them (instead of grudgingly living with them and eventually switching MUAs again).
I already had some superficial experience with Emacs Lisp, and this seemed like a great opportunity to do something more complicated with it. Not surprisingly, I quickly discovered things I disliked. To my delight, I also managed to fix several issues and got the changes incorporated upstream.
However, the predominant feeling that I had when hacking Gnus was one of disappointment. I always wondered how you could actually write something as complicated as a MUA in a language designed to facilitate text editing. After looking at the source, my answer to that question is "badly". There are no clever techniques or powerful language features I wasn't aware of, instead everything looks more hacky and fragile than I ever thought possible for a project with the size, history and reputation of Gnus.
Turning an editor into a MUA
I think I was always assuming that Gnus was essentially implemented like any other MUA - only using Emacs buffers to render the user interface. This turns out not to be the case. Instead, Gnus attempts to cast most of the tasks that a MUA has to perform into something that looks like text editing, and then uses Emacs text editing functions to accomplish the task. This may not seem like such a terrible idea at first, but here are some examples of what this means in practice:
There are no complex data structures to pass information from one function to another. Instead, information is passed in a dedicated Emacs buffer. Sometimes the buffer is allocated dynamically, but sometimes even with a fixed name (giving the term "global variable" a whole new meaning). I would expect that e.g. a function that fetches headers from an IMAP server would return some sort of array of structs but instead it actually writes them into buffer in RFC822 format. The caller then reads this buffer and parses the context.
Basically any sort of parsing is done in an ad-hoc fashion using lots of regular expressions (potentially inside a while loop). This includes both the parsing of the buffers that are used for communication between Gnus functions, and the parsing of standardized protocols that are used with remote servers. Take, for example, the parsing of the IMAP FETCH response. The format is specified in RFC 3501, section 7.4.2. Instead of writing a proper parser that would e.g. ensure that parentheses are properly nested and would return some structured object, the way Gnus determines the properties of a message is to first move to the beginning of the line, do a regular expression search for the name of the property (e.g. RFC822.SIZE followed by some numbers), and then take the first match. After this, the cursor (remember, everything happens in a buffer) is moved to the beginning of the line and the procedure starts again for the next property. There is absolutely zero awareness of nesting levels or a distinction between keys and values.
This sort of parsing appears extremely fragile. I'm pretty sure that something like an email with content type text/RFC822.SIZE 47 would confuse the hell out of Gnus. It doesn't even take a malicious sender: one of the bugs that I fixed was the implicit assumption that in the FETCH response of an IMAP server the UID always comes before the size - because the "parser" forgot to go back to the beginning of the line after it was looking for the UID.
There are no good means to signal errors. Basically, if there is any sort of problem (e.g. the SMTP server refuses to accept a message) there are only two choices: write a message into the minibuffer (that the user may or may not see, and that's overwritten as soon as any other Emacs components wants to say something), or signal a lisp error. Unfortunately Lisp errors are (or are used this way in Gnus) completely unstructured, so they are often caught and suppressed indistinctly by higher level code.
There are various examples for this: did you specify an invalid expiry target? Messages will just silently not expire. Did you specify the wrong credentials for your IMAP server? You will only get a brief message in the minibuffer. Is the server refusing to delete or store an IMAP message? You may or may get a brief message in the minibuffer, but Gnus will otherwise appear to have successfully executed the command.
There is very little error detection, and lots of chances for errors to occur and propagate. Since data is passed in form of buffers, functions can receive and return completely unstructured, arbitrary values. In some cases, the expected format of the buffer is documented in the comments. In other cases, it is completely undocumented and has to be deduced by looking at all the participating functions. But only very rarely is the structure of the data in actually reflected in the datatype of a function's arguments.
Even worse, most functions also do not make any attempt to validate the buffers they work with. If they happen to contain garbage, the function will happily work with it. And since parsing of the buffer is done with some ad-hoc regular expression, chances are that the function will even be able "process" the buffer and obtain a result that it passes on (but which is just garbage).
The Gnus code is full of arcane workarounds. For example, "select methods" (which is Gnus name for backends that implement different protocols) can share common code. Normally, this would be implemented by making backends into classes that can inherit from each other. Gnus, however, was written before Emacs Lisp had support for object-oriented programming and thus comes with its own runtime emulation of classes. In other words, there is code that dynamically renames your variables and functions to make it appear they're part of multiple backends (look at nnoo.el for some code that will blow your head).
UPDATE: there has just been a decision to move Gnus development into the GNU Emacs Git repository and scrap support for anything but the most-recent GNU Emacs version. Nice!.
Concluding Thoughts
Having seen this, my optimism of making Gnus into my one and only tool for mail handling is rather diminished. For me, one of Gnus' most appealing features is that it is written in a high-level, interpreted language and can therefore very easily be modified and extended. However, now that I have actually worked with the code, I don't think it's all that easy and pleasant in practice. Emacs Lisp as a language is nice to work with. However, the contortions that have been done to turn Emacs into a MUA seem to make everything so fragile and complex that even small changes become rather unpleasant to implement (at least for me).
Unfortunately, at this point Gnus nevertheless seems to be the least worst option when it comes to mail handling. In my opinion, web-based interfaces are a non-starter for anything but casual use and Thunderbird has a terrible editor (just try to rearrange quotes when composing in HTML, or to copy & paste a fragment of source code without getting automatic line breaks when composing in text mode). There is Evolution, Geary, and Claws, but the last time I looked at them none of them did a better job at Email handling than Thunderbird. I guess what I really want is something like Gnus but written in a language that's more suitable to this task than Emacs Lisp - maybe Python. Does anyone have any suggestions?