A simple ‘letn’ macro

The ‘let’ special form is one of the most frequently used constructs in Common Lisp. This allows true lexical binding of variables (and is quite often used like local variables in imperative languages). The general form of this binding is as follows:

(let ((binding)*)

For instance, we might have a function which reads in two numbers and finds their sum:

(defun add-two-numbers ()
      (let ((x (read))
               (y (read)))
           (+ x y)))

The example above is trivial enough and is quite readable. However, let bindings can become increasingly complex and unreadable in most general cases. Having had a taste of the mind-bendingly powerful macro system (plain macros, not Reader Macros, which are an entirely different concept), I decided to patch together a simple macro called ‘letn’ that would allow me to write let-bindings in a much more simplistic (with fewer parentheses) form. Of course, this is merely an exercise in testing out macros rather than a serious extension to the language (even though I have found it useful beyond my own expectations). To wit, instead of a form such as:

(let ((a 1)
         (b 2)
         (c 3)
          (e 100)
          (f “hello”))
     (let ((g “world”)
              (h “again”))
           (format t “~d~%” (+ a b c e))
           (format t “~a~%” (concatenate ‘string f g h))
           (format t “The value of ~a is, of course, ~a~%” ‘d d)

We might write:

(letn a 1 b 2 c 3 d e 100 f “hello”
    (letn g “world” h “again”
             (format t “~d~%” (+ a b c e))
             (format t “~a~%” (concatenate ‘string f g h))
             (format t “The value of ~a is, of course, ~a~%” ‘d d)

Notice how the second form is much more readable and less-dependent on form structure and indentation to convey its meaning. As the level of nesting increases, this becomes even more apparent, and thanks to the power of Common Lisp’s macro system, this nesting can be handle to arbitrary levels without any modifications or special handling on our behalf.

Now let’s write this macro!

To begin with, let us write out the logical form of the macro itself and then fill out the missing functions/macros as we go along.

Here’s what the macro looks like at this stage:

(defmacro letn (&body body)
    (multiple-value-bind (arg-forms body-forms) (parse-body body)
        `(let (,@(loop for arg in arg-forms
                              collect `(,(car arg) ,(cdr arg))))

Explanation: The assumption is that all variable bindings occur in the beginning – either as pairs of variables and values (such as, a 1) or as a variable followed by no value (such as, b, in which case it is assigned nil by default). ‘multiple-value-bind’ allows us to bind multiple return values to variables of our own. In this case, I want to assign the list of all variable bindings to the variable ‘arg-forms’ and then the remaining forms (which may be arbitrarily nested binding and body forms) to the ‘body-forms’ variable. These lists are returned by the ‘parse-body’ function which is filled out next:

(defun parse-body (body)
	   (let ((args '())
		 (forms '()))
	     (labels ((f (lst)
			(if (listp (car lst))
			    (setf forms lst)
			      (push (car lst) args)
			      (f (cdr lst))))))
	       (f body))
	     (values (param-pairs (reverse args)) forms)))

Explanation: The parse-body function simply takes the list of values passed in (body), and then sets about recursing until the entire list has been processed. While recursing, any non-list values are assumed to be variable bindings and are pushed on to the ‘args’ variable, and upon encountering the first list form, the remaining list forms (including the current one) are bound to the ‘forms’ variable. Finally, since ‘push’ pushes values at the front, we need to reverse this list, invoke ‘param-pairs (which will create dotted pairs of variable bindings), and then return this processed list as well as the unprocessed ‘forms’ variable back to the macro (which then uses ‘multiple-value-bind’ to extract out the components as explained before). Finally, the param-pairs function looks as shown below:

(defun param-pairs (args)
	   (labels ((f (lst rev)
		      (if (null lst)
			  (reverse rev)
			  (if (symbolp (cadr lst))
			      (f (cdr lst) (cons (cons (car lst) nil) rev))
			      (f (cddr lst) (cons (cons (car lst) (cadr lst)) rev))))))
	     (f args '())))

Explanation: This is a relatively simply function that simply takes the input list (which might be of the form, say, (a 1 b 2 c 3 d e 100), and then constructs dotted pairs of the form, ((a . 1) (b . 2) (c . 3) (d . nil) (e . 100)). Again, since ‘cons’ adds elements to the front, when the entire list has been processed, the concatenated list needs to be reversed and returned.

And finally, we are done! To see how our example runs, we can simply execute it:

CL-USER> (letn a 1 b 2 c 3 d e 100 f "hello"
	      (letn g "world" h "again"
		    (format t "~d~%" (+ a b c e))
		    (format t "~a~%" (concatenate 'string f g h))
		    (format t "The value of ~a is, of course, ~a~%" 'd d)
The value of D is, of course, NIL

As can be seen, the new macro itself can be nested. It might seem a bit weird since we are mixing macros (which are expanded at macro-expansion time as part of the compilation phase) and functions (which are evaluated at runtime), but this is completely normal since we are taking care to ensure that evaluated and non-evaluated forms mix together correctly. The functions generate values which are then quasi-quoted and unquoted appropriately. And, of course, the nesting is handled by the wonderful splicing-comma (,@(body-forms) in the ‘letn’ macro). To see what this example actually expands out to, we can used macroexpand (or macroexpand-1):

CL-USER> (setf *print-pretty* t)
CL-USER> (macroexpand-1
	  '(letn a 1 b 2 c 3 e 100 f "hello"
	    (letn g "world" h "again"
		  (format t "~d~%" (+ a b c e))
		  (format t "~a~%" (concatenate 'string f g h))
		  (format t "The value of ~a is, of course, ~a~%" 'd d)
(LET ((A 1) (B 2) (C 3) (E 100) (F "hello"))
        (FORMAT T "~d~%" (+ A B C E))
        (FORMAT T "The value of ~a is, of course, ~a~%" 'D D)

Of course, the inner ‘letn’ will be expanded according to the same rules as well.

Future improvements: This is a trivial macro because there are a lot of assumptions about the use of the new ‘letn’ construct. The biggest problem is that it cannot handle invocations of the form (amongst others):

(letn a (read) b (read)
      (+ a b))

The reason is that this macro assumes that the first occurrence of a list entity marks the beginning of the forms that constitute the macro body, and as such, this simple example would expand out to:

CL-USER> (macroexpand-1
	  '(letn a (read) b (read)
	    (+ a b)))
(LET ((A NIL)) (READ) B (READ) (+ A B))

This is clearly incorrect and results in ‘b’ becoming an unbound variable as well. The problem now is how to distinguish between variable bindings and genuine body forms. I am still working on improving my understanding of how macros work and whether there are mechanisms which can help in this situation, or if I need to handle them myself. I will, of course, post my improvements as I go along. Stay tuned!


Path to Common Lisp

I have always been extremely intrigued by Common Lisp ever since I can remember. The sheer simplicity of the basic concepts of this language (which is essentially Lambda Calculus in disguise) is what drew me on to it, and for a long time, it remained just that – an enigma that attracted me every so often and yet left me exasperated with the paucity of good materials and an active community. However, of late, I have been giving it a real serious go. The immediate impetus has to be this excellent collection of interviews by Vsevolod Dyomkin (available here – Lisp Hackers. It is a veritable treasure trove of information about the latest generation of Lisp hackers. The common theme that is to be found though is that Common Lisp itself is not used as much as it is worked upon.

My current undertaking of Lisp is a rather determined one and I daresay that I have made good progress. Another advantage of this is that Scheme and Clojure (which I had tried before but didn’t like that much) are but a small leap from here. My main interest is however in mastering Lisp concepts well enough so that I can use it for my own personal projects to begin with, and then see where that leads me (I really do envy Zach Beane in this regard – the man hacks on Lisp full time at Clozure Associates!).

In this introductory post (after a long long hiatus from blogging), I would like to begin with describing my own path to Common Lisp (and I believe this should help act as a rough guide for any beginners embarking upon the Lisp journey!):

  • Start off with Land of Lisp – arguably the most user-friendly (and yet powerful) introduction to Common Lisp today. It is extremely well-structured and fun to work with. In my opinion, most of the “games” in later chapters can be safely skipped to begin with. They can taken up after a solid understanding of Lisp fundamentals is in place.
  • Next up is Practical Common Lisp. Never mind what the reviews say, this is definitely not a great first book for absolute beginners in Common Lisp. The book makes a lot more sense after finishing Land of Lisp. In my experience, some of the chapters are well-written, quite a few are slipshod and follow a bizarre series of crammed and cryptic notes (especially the chapter on CLOS), but the practical chapters are absolutely necessary to get the hands-on experience that makes one a well-rounded programmer. Overall, a great book.
  • Now one should be in a position to tackle the venerable (if opinionated) Paul Graham’s anthology – ANSI Common Lisp and On Lisp (in that order). This is where I am as well on my own journey in Common Lisp.
  • One must-read book is Let Over Lambda by Doug Hoyte. This is a good follow-up book to Paul Graham’s classics (and even on its own, a great way to expand your perspective – the first few chapters made me really “get” Closures and true Lexical Scoping at long last).
  • Tons of hands-on (this should be orthogonal to this list in any case. Practical sessions are what really teach one…well, anything really). Some few resources to get started off would be – exercism.io and 99 Problems in Lisp. The latter is really useful for practising Functional Programming in Common Lisp.
  • Just a quick note on development environments. I personally use emacs + SLIME + SBCL. SBCL is a very efficient implementation of Common Lisp. CLisp works fine too, but it’s rather very slow (understandably so since it’s not compiled to machine code like SBCL is). Some other flavours are – Clozure CL, LispWorks, and AllegroCL. The latter two are commercial distros, but they do offer personal editions which work just fine for most purposes. The fun bit is that SLIME can connect to any of these flavours in any case, so your development environment can remain consistent irrespective of which Lisp flavour you choose to work with.

    I suppose that should about do it for an introductory post on Common Lisp! Happy Hacking!