Demoing a simple module in Java 9

In the spirit of the upcoming Java 9 release, I decided to port my D version of the “Guessing game” into Java, but with a twist. I thought that it would be a nice touch to develop the game into a nice module, and turn the process of developing it into an approachable starter tutorial for creating, using, and most importantly, understanding the basics of Java 9 modules.

Background

One of the biggest changes in Java 9 is the introduction of support for modules into the language itself. The rationale and reasoning behind it is very well documented on the OpenJDK site under JEP261.

However, if you find that heavy reading, a wonderfully succinct and clear Quick Start
is also provided for developers who want to simply get on with it. This post is based on that tutorial.

Before we get started with the actual game, I just wanted to ruminate a bit on why I think it’s a welcome idea that modules are being introduced into the language itself. A lot of people conflate packages in Java with modularity. Well, in a sense that does make sense. However, I like to view it as more of a namespacing mechanism than anything else.

Modularity, by definition, would imply a stronger separation of concerns – classes and interfaces wrapped in packages wrapped within modules. Java 9 modules achieve precisely that – they control which packages (and thereby which classes) are visible outside the module itself. Whereas previously we could just toss any old JAR file into the classpath and load classes at will, now we can only do so if the module they reside in explicitly expose them. Very nice!

The game

The game is dead simple – the program generates a secret number in the range [1, 100], and it’s the player’s job to guess the number.

The setup

Okay, so let’s start off by creating a directory for our game module.

Macushla:modules z0ltan$ mkdir guessing-game
Macushla:modules z0ltan$ cd guessing-game

Now, let’s create a directory called mods where we will store the generated module, and also a directory called src (following convention, not strictly enforced) which will hold the module source code:

Macushla:guessing-game z0ltan$ mkdir mods
Macushla:guessing-game z0ltan$ mkdir src
Macushla:guessing-game z0ltan$ tree .
.
├── mods
└── src

2 directories, 0 files

The naming convention for modules is identical to that of packages (com.foo.bar), so pay close attention to the next few steps. Inside the src directory, let us create a directory named com.guessing (note: it is a single directory, not nested
directories). Also, inside the com.guessing directory, let us create a file called module-info.java:

Macushla:guessing-game z0ltan$ mkdir src/com.guessing
Macushla:guessing-game z0ltan$ touch src/com.guessing/module-info.java
Macushla:guessing-game z0ltan$ tree .
.
├── mods
└── src
    └── com.guessing
        └── module-info.java

3 directories, 1 file

So here’s how it works – the directory com.guessing represents the module that we are creating for the game, and it is signified by the module declaration that we will store inside module-info.java. The casing and hyphenation might throw you off a bit, but that’s how Java knows that you have declared a module, so make sure to get the name right!

Right, so let’s go on ahead and write up the module declaration inside module-info.java. Open up the file in your favourite text editor, and then enter the following metadata inside it:

 module com.guessing {
    exports com.my.guessing.game;
 }

Let’s take a moment to understand this bit – the module keyword is how you introduce a module declaration. So we’re declaring here that com.guessing is a module, and that we are exposing the package structure, com.my.guessing.game
from this module. What does this mean? Well, it means that any other module that wishes to use our module can only access the classes that are in this package structure. This is how modules implement real modularity where only the code that needs to be visible to the outside world is made so.

Since we have declared that we are exposing the package com.my.guessing.game, it follows that we should have that package structure in our src directory as well. Let’s go ahead and
create it:

Macushla:guessing-game z0ltan$ mkdir -p src/com.guessing/com/my/guessing/game

And let’s create out single source file named TheGame.java inside the same location:

Macushla:guessing-game z0ltan$ touch src/com.guessing/com/my/guessing/game/TheGame.java

So here’s how the project tree looks at this stage:

Macushla:guessing-game z0ltan$ tree .
.
├── mods
└── src
    └── com.guessing
        ├── com
        │   └── my
        │       └── guessing
        │           └── game
        │               └── TheGame.java
        └── module-info.java
        

Excellent! So, let’s fill out the code for the game now, inside TheGame.java

package com.my.guessing.game;

import java.util.Random;
import java.util.Scanner;

public class TheGame {
	private static Random rand;

	public TheGame() {
		rand = new Random();
	}

	public static void main(String[] args) {
		TheGame game = new TheGame();
	
		game.play();
	}

	private void play() {
		try{
			Scanner in = new Scanner(System.in);		

			System.out.println("Welcome to the Guessing Game!");

			// secret number in the range [1, 100]
			int mySecretNumber = rand.nextInt(100) + 1;
			int attempts = 0;

			while (true) {
				attempts++;

				System.out.println("Enter your guess (between 1 and 100 inclusive)");
				int guess = in.nextInt();

				if (guess < mySecretNumber) {
					System.out.println("Too small!");
				} else if (guess == mySecretNumber) {
					System.out.format("You win! You took %d guesses!\n", attempts);
				} else {
					System.out.println("Too big!");
				}
			}
		} catch (Exception ex) {
			System.err.println("Oh no! Something went terribly wrong. Aborting game!");
			System.exit(1);
		}
	}
}

Nice! So now all we have left to do is to compile the game and run it. Ah, but there remains one more step before we can actually do that. Remember the mods directory we had created in the first step? That directory is meant to hold all the modules that we would generate for our project (there can be as many modules as you like in your project, with various modules dependent on various other modules). In our game, we have only one module,
com.guessing. To hold the contents of this module, we need to create the corresponding module directory in the mods directory, like so:

Macushla:guessing-game z0ltan$ mkdir mods/com.guessing
Macushla:guessing-game z0ltan$ tree .
.
├── mods
│   └── com.guessing
└── src
    └── com.guessing
        ├── com
        │   └── my
        │       └── guessing
        │           └── game
        │               └── TheGame.java
        └── module-info.java

8 directories, 2 files

Excellent! Let’s go ahead and compile the game:

Macushla:guessing-game z0ltan$ javac --module-path=mods -d mods/com.guessing \
  src/com.guessing/module-info.java src/com.guessing/com/my/guessing/game/TheGame.java 
Macushla:guessing-game z0ltan$ tree .
.
├── mods
│   └── com.guessing
│       ├── com
│       │   └── my
│       │       └── guessing
│       │           └── game
│       │               └── TheGame.class
│       └── module-info.class
└── src
    └── com.guessing
        ├── com
        │   └── my
        │       └── guessing
        │           └── game
        │               └── TheGame.java
        └── module-info.java

12 directories, 4 files

Okay, the compilation command requires a bit of explanation – the --module-path option is used to indicate the root directory (or directories) where the module (or modules) is to be stored. In our case, that is the mods directory. The -d flag indicates where the generated class files are to be stored. In our case, this is the mods/com.guessing
directory. Finally, we link in all the source files that need to be compiled, and this includes the module-info.java metadata file as well. Note that all this should be automatically done once IDE support is widely available for Java 9 (IntelliJ apparently has some support for Java 9 already,
and Eclipse also has an extra plugin that supports Java 9).

Also observe the contents of the mods directory. It’s an exact replica of the src directory albeit containing the generated classes.

As a final step, let’s run the game! This is how you would run it from the command-line:

Macushla:guessing-game z0ltan$ java --module-path mods -m com.guessing/com.my.guessing.game.TheGame
Welcome to the Guessing Game!
Enter your guess (between 1 and 100 inclusive)
50
Too small!
Enter your guess (between 1 and 100 inclusive)
75
Too big!
Enter your guess (between 1 and 100 inclusive)
63
Too big!
Enter your guess (between 1 and 100 inclusive)
56
You win! You took 4 guesses!

Again, a bit of commentary on the run command – we again use the --module-path option to tell Java where our module resides, and then we pass the actual fully-qualified name of the class to the -m flag, which tells Java which module we are interested in running. Note that there are various short forms (for instance, we can use
-p> instead of --module-path in this case, refer to the documentation for all the flags and options).

And we’re done!

Extras

One additional interesting feature in Java 9 is that we can bundle our game into a nice little app. To do that, we need to understand that Java 9 actually comes with all its functionality divided into modules. For instance, all core functionality resides in the java.base module. Also note that this module is automatically assumed to be used by all custom modules. For instance, the module declaration that we used for this game could very well have been written explicitly as:

module com.guessing {
   requires java.base;
   exports com.guessing.my.game;
}

where the requires module is the opposite of exports. Beware though that having the requires declaration is not enough to be able to use classes from that module. It only makes the exported packages in that module visible in the require-ing module. That means that in our code, we should still have explicit importS for those public packages. For instance, if we want to find out which modules are present
by default, we can view them so:

Macushla:guessing-game z0ltan$ java --list-modules
java.activation@9-ea
java.annotations.common@9-ea
java.base@9-ea
java.compact1@9-ea
java.compact2@9-ea
java.compact3@9-ea
java.compiler@9-ea
java.corba@9-ea
java.datatransfer@9-ea
java.desktop@9-ea
java.httpclient@9-ea
java.instrument@9-ea
java.jnlp@9-ea
java.logging@9-ea
java.management@9-ea
java.naming@9-ea
java.prefs@9-ea
java.rmi@9-ea
java.scripting@9-ea
java.se@9-ea
java.se.ee@9-ea
java.security.jgss@9-ea
java.security.sasl@9-ea
java.smartcardio@9-ea
java.sql@9-ea
java.sql.rowset@9-ea
... (content elided due to space contraints)

All the modules reside in the $JAVA_HOME/jmods directory, but you might be surprised if you check the contents – they won’t correspond to the output we saw above. The reason for this is the modules that come bundled with the JDK are in “jmod” form, which is a compiled format. The jmod tool comes bundled with the JDK, and can be used to create a jmod binary for our custom modules as well. The terminology for the module format that we have used here is “exploded module”. This JEP contains all the gruesomely interesting details.

Right, so where are we going with this? Ah yes, we wanted to bundle our game into a custom app. Let’s do that (but all this jmod discussion will come in handy later on)!

First off, let’s set our JAVA_HOME environment variable for convenience. Note, this section describes how to set it for macOS users only. For windows users, consult your documentation!

On macOS, one problem is that finding the paths of various installed tools can be a real PITA. Thankfully, we have a bundled tool called java_home to help us out:

Macushla:guessing-game z0ltan$ /usr/libexec/java_home -V
Matching Java Virtual Machines (3):
    9, x86_64:	"Java SE 9-ea"	/Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home
    1.8.0_112, x86_64:	"Java SE 8"	/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home
    1.8.0_45, x86_64:	"Java SE 8"	/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home

/Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home

Okay, so on my machine, I have three different versions of the JDK installed. Clearly, I need to use the Java 9 version, so let’s set the environment variable to use that:

Macushla:guessing-game z0ltan$ export JAVA_HOME="$(/usr/libexec/java_home -v 9)"
Macushla:guessing-game z0ltan$ echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home

Excellent. Now, let’s go ahead and create our app. For this, we use another new tool that comes bundled with the JDK – jlink. As the name suggests, it is a linker, and we use this linker to link our module into Java, and create a custom version of the JVM that contains only our dependencies (in addition to the mandatory java.base module):

Macushla:guessing-game z0ltan$ jlink --module-path $JAVA_HOME/jmods:mlib --add-modules mods/com.guessing \
> --output my_guessing_app
Error: Module com.guessing not found

Module not found? How is it possible? We specified the fully-qualified module name, mods/com.guessing, didn’t we? Well, the reason jlink failed is that it accepts modules in only two formats – either JMOD (as discussed above), or as a JAR file containing our “exploded” module. So before we can bundle our app, there remains just one more step – bundling our app as a JAR or as JMOD. Let’s go for the jAR file
option:


Macushla:guessing-game z0ltan$ jar --create --file=guessing_game@0.0.1.jar \
  --module-version=0.0.1 --main-class=com.my.guessing.game.TheGame -C mods/com.guessing .

Note how we have to specify the module version explicitly. Java 9 modules do not (as yet) support automatic versioning, nor is versioning checked or enforced (as far as I can tell).

And now we can check that the JAR file contains a valid module by passing the --print-module-descriptor
flag to the jar command:

Macushla:guessing-game z0ltan$ jar --print-module-descriptor guessing_game\@0.0.1.jar 
^CMacushla:guessing-game z0ltan$ jar --print-module-descriptor --file guessing_game\@0.0.1.jar 

com.guessing@0.0.1
  requires mandated java.base
  exports com.my.guessing.game
  main-class com.my.guessing.game.TheGame

So it looks like all is in order. Let’s try creating the app now!

Macushla:guessing-game z0ltan$ jlink --module-path $JAVA_HOME/jmods:mlib --add-modules mods/com.guessing/ \
> --output my_guessing_game
Error: Module mods/com.guessing/ not found
Macushla:guessing-game z0ltan$ jlink --module-path $JAVA_HOME/jmods:mlib --add-modules com.guessing/ \
   --output my_guessing_game
Error: Module com.guessing/ not found

What gives? Note that the official convention is to create a directory to contain the JAR or JMOD file, called mlib. The modules that come bundled with the JDK all follow the same convention as well.

So let’s create a directory called mlib in our project root directory:

Macushla:guessing-game z0ltan$ jar --create --file=mlib/guessing_game@0.0.1.jar \
  --module-version=0.0.1 --main-class=com.my.guessing.game.TheGame \
   -C mods/com.guessing .

So the final directory structure looks like so:

Macushla:guessing-game z0ltan$ tree .
.
├── mlib
│   └── guessing_game@0.0.1.jar
├── mods
│   └── com.guessing
│       ├── com
│       │   └── my
│       │       └── guessing
│       │           └── game
│       │               └── TheGame.class
│       └── module-info.class
└── src
    └── com.guessing
        ├── com
        │   └── my
        │       └── guessing
        │           └── game
        │               └── TheGame.java
        └── module-info.java

13 directories, 5 files

The final step is to run the same command that we had tried before:

Macushla:guessing-game z0ltan$ jlink --module-path $JAVA_HOME/jmods:mlib --add-modules com.guessing --output my_guessing_game
Macushla:guessing-game z0ltan$ ls
mlib             mods             my_guessing_game src

Excellent! So we have a directory called my_guessing_game created. Let’s check its
contents:

Macushla:guessing-game z0ltan$ tree my_guessing_game/
my_guessing_game/
├── bin
│   ├── com.guessing
│   ├── java
│   └── keytool
├── conf
│   ├── net.properties
│   └── security
│       ├── java.policy
│       ├── java.security
│       └── policy
│           ├── README.txt
│           ├── limited
│           │   ├── default_US_export.policy
│           │   ├── default_local.policy
│           │   └── exempt_local.policy
│           └── unlimited
│               ├── default_US_export.policy
│               └── default_local.policy
├── lib
│   ├── jli
│   │   └── libjli.dylib
│   ├── jspawnhelper
│   ├── jvm.cfg
│   ├── libjava.dylib
│   ├── libjimage.dylib
│   ├── libjsig.dylib
│   ├── libnet.dylib
│   ├── libnio.dylib
│   ├── libosxsecurity.dylib
│   ├── libverify.dylib
│   ├── libzip.dylib
│   ├── modules
│   ├── security
│   │   ├── blacklist
│   │   ├── blacklisted.certs
│   │   ├── cacerts
│   │   ├── default.policy
│   │   └── trusted.libraries
│   ├── server
│   │   ├── Xusage.txt
│   │   ├── libjsig.dylib
│   │   └── libjvm.dylib
│   └── tzdb.dat
└── release

10 directories, 34 files

Whoa! That’s a lot of files and directories. The good part is that it is now a full-fledged, self-contained Java app containing our game!

Let’s take it for a spin!

Macushla:guessing-game z0ltan$ cd my_guessing_game/
Macushla:my_guessing_game z0ltan$ ls
bin     conf    lib     release
Macushla:my_guessing_game z0ltan$ bin/
com.guessing  java          keytool       
Macushla:my_guessing_game z0ltan$ bin/com.guessing 
Welcome to the Guessing Game!
Enter your guess (between 1 and 100 inclusive)
50
Too big!
Enter your guess (between 1 and 100 inclusive)
25
Too small!
Enter your guess (between 1 and 100 inclusive)
38
Too small!
Enter your guess (between 1 and 100 inclusive)
44
Too small!
Enter your guess (between 1 and 100 inclusive)
47
Too big!
Enter your guess (between 1 and 100 inclusive)
46
Too big!
Enter your guess (between 1 and 100 inclusive)
45
You win! You took 7 guesses!

Noiiiice! However, before you get too excited, note that this is not AOT (Ahead of time compilation)! bin/com.guessing is simply a script file that uses the bundled JVM instance to run the app. We can verify this by checking the contents of com.guessing:

Macushla:bin z0ltan$ cat com.guessing 
#!/bin/sh
JLINK_VM_OPTIONS=
DIR=`dirname $0`
$DIR/java $JLINK_VM_OPTIONS -m com.guessing/com.my.guessing.game.TheGame $@

Still, pretty nice, innit? We have come a long way from your grandfather’s Java, my friends.

One last interesting observation before we collapse from mental overload! Recall the command we had used to check the modules bundled in the default JDK, java --list-modules? Well, let’s try out the same command on the bundled JVM instance in the bin
directory of our app:

Macushla:my_guessing_game z0ltan$ bin/java --list-modules
com.guessing@0.0.1
java.base@9-ea

Heh! As we expected, since our project depends only on two modules, com.guessing, which is our game, and on the mandatory module, java.base, we see that this stripped down JVM only has these two modules, which leads to a very small overall app size as well (if we had dependencies on other modules, then those would be transitively included
as well):

 Macushla:my_guessing_game z0ltan$ cd ..
Macushla:guessing-game z0ltan$ ls
mlib             mods             my_guessing_game src
Macushla:guessing-game z0ltan$ du -m my_guessing_game/
1	my_guessing_game//bin
1	my_guessing_game//conf/security/policy/limited
1	my_guessing_game//conf/security/policy/unlimited
1	my_guessing_game//conf/security/policy
1	my_guessing_game//conf/security
1	my_guessing_game//conf
1	my_guessing_game//lib/jli
1	my_guessing_game//lib/security
13	my_guessing_game//lib/server
31	my_guessing_game//lib
31	my_guessing_game/
Macushla:guessing-game z0ltan$

A mere 31MB with our bundled JVM! Ain’t that nice? Time for a nice beer now!

Demoing a simple module in Java 9

A simple guessing game in D

This short post is inspired by the recent spate of languages introducing themselves through a small game program. As such, here is a version of the guessing game in D.

The objective of the game is very simple – there is a secret number randomly generated per game (in the range [1, 100] for simplicity), and the user has to guess the number to exit the game.

I also wanted to get a bit more practice with the de facto package cum dependency manager for D, dub.

Let’s start off by creating the project:

Macushla:MiniProjects z0ltan$ dub init guessing_game
Package recipe format (sdl/json) [json]: json
Name [guessing_game]: 
Description [A minimal D application.]: A simple guessing game in D
Author name [Timmy Jose]: 
License [proprietary]: MIT
Copyright string [Copyright © 2017, Timmy Jose]: 
Add dependency (leave empty to skip) []: 
Successfully created an empty project in '/Users/z0ltan/Rabota/ProgrammingLanguages/D/MiniProjects/guessing_game'.
Package successfully created in guessing_game

By default, dub also creates a .gitignore file in the root directory of the new project/package.

Here’s how the layout of the default configuration looks like:

Macushla:MiniProjects z0ltan$ cd guessing_game/
Macushla:guessing_game z0ltan$ tree .
.
├── dub.json
└── source
    ├── app.d

1 directory, 2 files

The dub.json file contains the basic configuration metadata for the project, and is for consumption by dub. Its initial contents are the data entered during the creation of the project.

A sample file, app.d is created inside the source directory. These can all be customised, of course.

Okay, so let’s create a new file to contain our guessing game code as a module.

Macushla:MiniProjects z0ltan$ touch source/guessing_game.d

All right, time to code up the game. Here are the contents of source/guessing_game.d

module guessing_game;

import std.stdio;

public void playGame() 
{
	import std.random: uniform;

	immutable uint secretNumber = uniform(1, 101);
	uint guesses = 0;

	writeln("\nLet's play a guessing game. Your task is to guess the secret number (between 1 and 100, inclusive)...");

	import std.conv: ConvException;

	while (true) {
		try {
			uint guess = getNumber("Enter your guess: ");

			++guesses;
			if (guess == secretNumber) {
				writefln("You win! You took %s guesses.", guesses);
				break;
			} else if (guess < secretNumber) {
				writeln("Too small!");
			} else {
				writeln("Too big!");
			}
		} catch (ConvException ex) {
			writeln("Did not get valid input. Try again...");
		}
	}
}


uint getNumber(string prompt) 
{
	writeln(prompt);

	import std.conv: to;
	import std.string: chomp;
		
	return readln().chomp.to!uint;
}

As you can see, this program is trivially readable for anyone who’s worked with C, C++, or indeed Java. The interesting bits to note would be the module system, as well as selective imports of module’s functions inside local lexical scopes.
Thankfully, D comes with a built-in module, std.random in its standard library (Phobos), and we can use the uniform function to generate our random number.

Another interesting bit to note is the following snippet of code:

return readln().chomp.to!uint;

The to! is a template function that is extremely powerful. It can be used to convert almost any type to any other type. In case the conversion fails though, this template function will throw a std.conv.ConvException exception. Note that D does not have checked exceptions (in Java-speak), only unchecked exceptions.

Also note the exception handling mechanism, which is extremely similar to that of Java.

Finally, observe the declaration of playGame as public. The helper function getNumber in source/guessing_game.d, by contrast is private, and not accessible outside the module in which it is defined.

Okay, so this is the guessing game logic. Now let’s take a look at our game’s entry point, source/app.d

import guessing_game;

void main()
{
	guessing_game.playGame();	
}

All we do is import the guessing_game module and then invoke playGame. Note that we could also have written this module like so:

import guessing_game: playGame;

void main() 
{
      playGame();
}     

Let’s try out the game!

Macushla:guessing_game z0ltan$ dub run
Performing "debug" build using dmd for x86_64.
guessing_game ~master: building configuration "application"...
Linking...
Running ./guessing_game 

Let's play a guessing game. Your task is to guess the secret number (between 1 and 100, inclusive)...
Enter your guess: 
50
Too small!
Enter your guess: 
hello
Did not get valid input. Try again...
Enter your guess: 
-1
Did not get valid input. Try again...
Enter your guess: 
75
Too small!
Enter your guess: 
88
Too big!
Enter your guess: 
81
Too big!
Enter your guess: 
78
You win! You took 5 guesses.
A simple guessing game in D

A simple letn macro in Racket

A long time back I had written a small blog about a ‘letn’ macro in Common Lisp (check it out here). Of late, I have been venturing deeper and deeper into Racket, and I am starting to like the language, and more importantly, the ecosystem more and more, especially with the express purpose of implementing languages and expanding my understanding of programming language theory.

This is why I decided to give the letn macro a go in Racket this time. Of course, this may by no means be the best way to implement it, but it was an enjoyable exercise all the same. The basic idea is to transform a form like:

   (letn [a 1 b 2 c 3]
       (+ a b c))

into the corresponding syntactic form:

   (let ([a 1]
         [b 2]
         [c 3])
      (+ a b c))

The only difference between this version and the Common Lisp version, functionally speaking, is that in this version, I expect the input to be well-formed pairs of variables and values. This makes more sense now since we cannot possibly substitute a sane value for a variable with no associated value (null? 0? void?), and expect a sane result.

Anyway, so here’s how it looks:

;;; a letn macro in Racket.

(module letn racket
  (provide letn)


;;; (letn [a 1 b 2 c 3 d 4 e 5 f 6] (+ a b c d e f)) ->
;;;
;;; (let ([a 1]
;;;       [b 2]
;;;       [c 3]
;;;       [d 4]
;;;       [e 5]
;;;       [f 6])
;;;    (+ a b c d e f))


(require (for-syntax racket/list)) ;; for empty

(begin-for-syntax
  (define (process-args-into-pairs lst)
    (letrec ([f (lambda (pairs lst)
                  (if (null? lst)
                      (reverse pairs)
                      (f (cons (list (car lst) (cadr lst)) pairs) (cddr lst))))])
      (f empty lst))))


(define-syntax letn
  (lambda (stx)
    (syntax-case stx ()
      [(_ (params ...) body0 body ...)
       (let ([pairs (process-args-into-pairs (syntax->datum #'(params ...)))])
         (with-syntax ([arg-pairs (datum->syntax stx pairs)])
           #'(let arg-pairs
                 body0 body ...)))]))))

Some notes

: Okay, so begin by defining the macro in a module of its own. Then we come to this interesting snippet of code”

    (require (for-syntax racket/list)) ;; for empty

What this code means is that we wish to use the racket/list module during compilation-time (since macro-expansion is part of the compilation phase). The reason for this is that we use empty, which denotes an empty list, in our program. Of course it would be easier to simply use the literal form, '() and eschew requiring racket/list altogether, but this is simply to demonstrate how we can require modules whose functions and symbols we would need at compile time.

Next up, we have the following code block:

(begin-for-syntax
  (define (process-args-into-pairs lst)
    (letrec ([f (lambda (pairs lst)
                  (if (null? lst)
                      (reverse pairs)
                      (f (cons (list (car lst) (cadr lst)) pairs) (cddr lst))))])
      (f empty lst))))

The begin-for-syntax starts off a new block where we can define functions that we need during compile time itself. In this case, we need a helper function called process-args-into-pairs which simply takes an input of the form

'(a 1 b 2 c 3 d 4 e 5)

and transforms those into a list of lists:

 '((a 1) (b 2) (c 3) (d 4) (e 5))

This is precisely what the actual macro needs during its expansion. Note that since we only have one helper function in this case, we could have used the simpler version, define-for-syntax to define our helper function like so:

    (define-for-syntax (process-args-into-pairs lst)
        (letrec ([f (lambda (pairs lst)
                      (if (null? lst)
                          (reverse pairs)
                          (f (cons (list (car lst) (cadr lst)) pairs) 
                             (cddr lst))))])
              (f empty lst))))

The process-args-into-pairs helper function itself is extremely straightforward – we simply accumulate lists of pairs of objects from the input list, and then return them to the caller. This code works on the understanding (as mentioned before) that we expect the input to consist of well-formed pairs).

Now let’s get down to the meat of the business, the letn macro itself. Here is the code:

(define-syntax letn
  (lambda (stx)
    (syntax-case stx ()
      [(_ (params ...) body0 body ...)
       (let ([pairs (process-args-into-pairs (syntax->datum #'(params ...)))])
         (with-syntax ([arg-pairs (datum->syntax stx pairs)])
           #'(let arg-pairs
                 body0 body ...)))]))))

There are various ways of defining macros in Racket – define-syntax-rule, define-syntax with syntax-rules,, define-syntax with syntax-case, define-syntax with custom transformer functions, syntax-parse,etc.

In this case, I have decided to use syntax-case since it suits the requirements quite nicely – built-in pattern matching is quite nifty!

So here’s how it works – we pattern-match on the supplied syntax (object), and we expect the pattern to be of the form: (letn [*] *). Note that in Racket, square brackets are essentially equivalent to (and are converted to, internally) parentheses. As a side-note, when entering code, however, mixing square brackets and parentheses for the same form is an error.

In the template (the right-hand-side of this case), we bind pairs to the output of the process-args-into-pairs helper function. Remember that all this happens during compile-time itself. Also note that we need to pass a plain list to the helper function. This is the reason why we need to convert the syntax object into a proper datum (done using (syntax->datum #'(params ...)).

Next, we need to construct the actual form that the invocation of the macro will expand into. In Common Lisp, we do all that using quasi-quoting and unquoting (with splicing if needed). However, in Racket, we have a different set of forms that deal with this business. The helper function returns a list of lists of variable-value pairs, and this list needs to be inserted into the let form that we use for the actual body of the template. This is why we need to convert pairs into a syntax object (since Racket works with syntax objects directly almost throughout). This is why we have
(datum->syntax stx pairs).

Finally, we now return the syntax from the template. Note the reader macro, #' which is shorthand for syntax.

Well, that’s about it! In macro, defining recursive macros is pretty easy using ellipses (...). This is used throughout for both the parameters and the body forms of the macro invocation. This also conveniently ensures that we can nest arbitrary forms inside out stonking new letn macro.

All right, let’s take it for a quick spin:

Here is the test code:

#lang racket

(require "letn.rkt")

(define (test-case-1)
  (letn [a 1 b 2]
        (displayln (+ a b))))

(define (test-case-2)
  (letn (a "hello" b "world")
        (displayln (string-append a ", " b))))


(define (test-case-3)
  (letn (a 1 b 2 c 3)
        (displayln "Adding three numbers")
        (displayln (+ a b c))))


(define (test-case-4)
  (letn (a 1 b 2 c 3)
        (letn (d 4 e 5)
              (displayln "Adding nested variables")
              (displayln (+ a b c d e)))))

(define (test-case-5)
  (letn (a 1 b 2 c 3)
        (let ([d 4]
              [e 5])
          (letn (f 6 g 7)
                (displayln "Nested letnS and letS")
                (displayln (+ a b c d e f g))))))

(define (run-all-tests)
  (test-case-1)
  (newline)
  (test-case-2)
  (newline)
  (test-case-3)
  (newline)
  (test-case-4)
  (newline)
  (test-case-5))  

And here is the output of a test run:

letn-test.rkt> (run-all-tests)
3

hello, world

Adding three numbers
6

Adding nested variables
15

Nested letnS and letS
28

Excellent! This was quite a satisfying little exercise. Now time to up the ante and start off with real languages!

A simple letn macro in Racket