In my last post I had mentioned that I would like to start on the Functional Programming series by creating a basic functional library that would start off by implementing the (Untyped) Lambda Calculus from scratch in various languages. However, before we jump down right into that, I thought I’d post a few short tutorials on how to use concurrency and parallelism in Common Lisp. Information on this topic is quite difficult to come by on the Internet, and it would serve as a good refresher course for me as well. Please note that I am myself but a beginner in the advanced features of Common Lisp, so bear with me if I do make some silly goof-ups.
I’ll start off with a short tutorial on the required setup. After that, I will show how we can build SBCL with threading support on Mac OS X, Then I will tackle basic concurrency support in Common Lisp, and finally, I will conclude this short series with a brief tutorial on parallelism support in Common Lisp.
One fundamental problem that arises when one talks of concurrency and parallelism in Common Lisp is that these topics are not covered by the ANSI Common Lisp standard. As such, each implementation is free to choose to provide support for these vital features as well they choose.
In this series, I will cover concurrency using the wonderful (if limited) Bordeaux library and also the native SBCL threading APIs. Parallelism will be covered using the lparallel library.
I’ll be using an Mac OS X (El Capitan) system for showing these examples (as well as the demos in the next two posts in this series). However, I’ve also tested out these steps on my Windows and Ubuntu Linux VMs, so you should be well covered whichever platform you might have.
Setup
So, let’s get started. However, we need to get some basic setup details take care of first. To give you an idea of what’s coming up, you will need the following set up in order to follow this series:
- A good Common Lisp implementation (SBCL would be my recommendation)
- Emacs with SLIME
- Quicklisp.
- Bordeaux threading library (installed using Quicklisp).
- lparallel threading library (installed using Quicklisp).
And that’s all there it to it! Let’s break it down step by step.
Which Common Lisp to install?
If you don’t have a Common Lisp implementation already installed, you can choose from a wide variety of options:
- Steel Bank Common Lisp (SBCL)
- Clozure Common Lisp (CCL)
- LispWorks (commercial, but free version available)
- Allegro Common Lisp (ACL) (commercial again, but free version available)
- CMUCL
- CLISP (interpreted, seems to be more or less dead now)
Not all of these implementations may be available on your platform, nor may all of them support threading on your platform. Take care to read the fine print!
I have worked with SBCL, LispWorks and CCL. For this series, I’ll be using my favourite Common Lisp implementation – SBCL throughout with small snippets of how things might work in other major implementations.
In case you want to use SBCL with Mac OS X, please check out my next blog post on how to bootstrap SBCL using itself (we need to build SBCL from source to enable thread support).
Emacs and SLIME
Of course, I assume that you are already familiar with and/or have configured Emacs along with SLIME. There is really no other way to actually work with Common Lisp in my opinion.
In case you are an absolute beginner and would like to try things out before committing to the arduous task of manually configuring Emacs and SLIME, I’d recommend LispBox (https://common-lisp.net/project/lispbox/). It is slightly outdated, but still works well enough (jump to the “LispBox Setup” section if you take this route).
In case you would like to setup it yourself (a great learning experience in itself), the following would be my recommended steps:
- Install emacs for your platform. Even if your OS comes pre-installed with emacs, I’d still recommend updating to the latest one manually (some package managers are very brittle in my experience): https://www.gnu.org/software/emacs/download.html
- Install SLIME. This is a good starting point in case you are on a non-Windows machine. If you are on a Windows machine, this is what worked for me. Of course, it usually doesn’t work straight off the box. A little tweaking will be necessary.
This video is also a well-prepared guide on how to install Emacs, SLIME, and Quicklisp. It is targeted for Windows, but it should mostly work for other platforms as well. Highly recommended in case you don’t enjoy reading reams of text!
Installing Quicklisp
Right. Now that Emacs has been installed and SLIME configured, let’s install Quicklisp. Quicklisp is a wonderful library manager created by Zach Beane. It makes installing third-party libraries almost seamless. The best part is that it handles all the libraries’ dependencies itself so you don’t have to sign up for that brand new sanatorium!
Download Quicklisp:
timmyjose@ubuntu:~/Software$ curl -O https://beta.quicklisp.org/quicklisp.lisp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 57144 100 57144 0 0 199k 0 --:--:-- --:--:-- --:--:-- 200k
timmyjose@ubuntu:~/Software$ ls
lein #lein# nim-0.14.2 quicklisp.lisp racket-6.5.0.5 rust-1.9.0
sbcl-1.3.8-x86-64-linux sbcl-1.3.8-x86-64-linux-binary.tar scala-2.11.8
Load up Quicklisp in SBCL (or whichever your flavour might be) and trigger the installation:
timmyjose@ubuntu:~/Software$ sbcl --load quicklisp.lisp
This is SBCL 1.3.8, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
==== quicklisp quickstart 2015-01-28 loaded ====
To continue with installation, evaluate: (quicklisp-quickstart:install)
For installation options, evaluate: (quicklisp-quickstart:help)
* (quicklisp-quickstart:install)
; Fetching #<URL "http://beta.quicklisp.org/client/quicklisp.sexp">
; 0.82KB
==================================================
838 bytes in 0.00 seconds (818.36KB/sec)
; Fetching #<URL "http://beta.quicklisp.org/client/2016-02-22/quicklisp.tar">
; 240.00KB
==================================================
245,760 bytes in 0.16 seconds (1518.99KB/sec)
; Fetching #<URL "http://beta.quicklisp.org/client/2015-09-24/setup.lisp">
; 4.94KB
==================================================
5,054 bytes in 0.00 seconds (4935.55KB/sec)
; Fetching #<URL "http://beta.quicklisp.org/asdf/2.26/asdf.lisp">
; 194.07KB
==================================================
198,729 bytes in 0.06 seconds (3289.34KB/sec)
; Fetching #<URL "http://beta.quicklisp.org/dist/quicklisp.txt">
; 0.40KB
==================================================
408 bytes in 0.00 seconds (398.44KB/sec)
Installing dist "quicklisp" version "2016-08-25".
; Fetching #<URL "http://beta.quicklisp.org/dist/quicklisp/2016-08-25/releases.txt">
; 350.44KB
==================================================
358,854 bytes in 0.26 seconds (1347.86KB/sec)
; Fetching #<URL "http://beta.quicklisp.org/dist/quicklisp/2016-08-25/systems.txt">
; 269.74KB
==================================================
276,212 bytes in 0.09 seconds (2900.41KB/sec)
==== quicklisp installed ====
To load a system, use: (ql:quickload "system-name")
To find systems, use: (ql:system-apropos "term")
To load Quicklisp every time you start Lisp, use: (ql:add-to-init-file)
For more information, see http://www.quicklisp.org/beta/
NIL
To add the initialisation code for Quicklisp to the SBCL init file (so that Quicklisp is
automatically loaded whenever SBCL is launched):
* (ql:add-to-init-file)
I will append the following lines to #P"/home/timmyjose/.sbclrc":
;;; The following lines added by ql:add-to-init-file:
#-quicklisp
(let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp"
(user-homedir-pathname))))
(when (probe-file quicklisp-init)
(load quicklisp-init)))
Press Enter to continue.
#P"/home/timmyjose/.sbclrc"
And finally, to use Quicklisp with Emacs and SLIME:
* (ql:quickload "quicklisp-slime-helper")
To load "quicklisp-slime-helper":
Install 3 Quicklisp releases:
alexandria quicklisp-slime-helper slime
; Fetching #<URL "http://beta.quicklisp.org/archive/slime/2016-05-31/slime-v2.18.tgz">
; 1076.19KB
<elided messages>
slime-helper.el installed in "/home/timmyjose/quicklisp/slime-helper.el"
To use, add this to your ~/.emacs:
(load (expand-file-name "~/quicklisp/slime-helper.el"))
;; Replace "sbcl" with the path to your implementation
(setq inferior-lisp-program "sbcl")
("quicklisp-slime-helper")
And we’re done with Quicklisp!
Installing the Bordeaux Threading Library
Now that we’ve got Quicklisp installed and configured, life should be much easier. To install the Bordeaux library, simply follow the following steps:
CL-USER> (ql:quickload :bt-semaphore)
To load "bt-semaphore":
Load 1 ASDF system:
alexandria
Install 2 Quicklisp releases:
bordeaux-threads bt-semaphore
; Fetching #<URL “http://beta.quicklisp.org/archive/bordeaux-threads/2016-03-18/
bordeaux-threads-v0.8.5.tgz">
; 19.63KB
==================================================
20,105 bytes in 0.01 seconds (1308.92KB/sec)
; Fetching #<URL "http://beta.quicklisp.org/archive/bt-semaphore/2013-10-03/bt-semaphore-20131003-git.tgz">
; 4.09KB
==================================================
4,191 bytes in 0.00 seconds (1364.26KB/sec)
; Loading "bt-semaphore"
[package bordeaux-threads]........................
[package bt-semaphore].
(:BT-SEMAPHORE)
Installing the lparallel Parallelism library
Installing the lparallel library follows pretty much the same idiom:
CL-USER> (ql:system-apropos "lparallel")
#<SYSTEM lparallel / lparallel-20160825-git / quicklisp 2016-08-25>
#<SYSTEM lparallel-bench / lparallel-20160825-git / quicklisp 2016-08-25>
#<SYSTEM lparallel-test / lparallel-20160825-git / quicklisp 2016-08-25>
; No value
CL-USER> (ql:quickload "lparallel")
To load "lparallel":
Load 2 ASDF systems:
alexandria bordeaux-threads
Install 1 Quicklisp release:
lparallel
; Fetching #<URL “http://beta.quicklisp.org/archive/lparallel/2016-08-25/
lparallel-20160825-git.tgz">
; 76.71KB
==================================================
78,551 bytes in 0.93 seconds (82.93KB/sec)
; Loading "lparallel"
[package lparallel.util]..........................
[package lparallel.thread-util]...................
[package lparallel.raw-queue].....................
[package lparallel.cons-queue]....................
[package lparallel.vector-queue]..................
[package lparallel.queue].........................
[package lparallel.counter].......................
[package lparallel.spin-queue]....................
[package lparallel.kernel]........................
[package lparallel.kernel-util]...................
[package lparallel.promise].......................
[package lparallel.ptree].........................
[package lparallel.slet]..........................
[package lparallel.defpun]........................
[package lparallel.cognate].......................
[package lparallel]
("lparallel")
Connecting from SLIME to LispWorks or ACL
In case you’re using a commercial Common Lisp implementation like LispWorks or Allegro Common Lisp (ACL), you can still use Emacs for your development. You just need to fire up your CL implementation, start Emacs, and then connect your SLIME instance to the CL backend.
To connect to a LispWorks instance, start it up, and in the REPL, type in the following:
CL-USER 2 : 1 > (load #P"C:/Users/Timmy Jose/AppData/Roaming/.emacs.d/elpa/slime-2.18/swank-loader.lisp") ; Loading text file C:\Users\Timmy Jose\AppData\Roaming\.emacs.d\elpa\slime-2.18\swank-loader.lisp #P"C:/Users/Timmy Jose/AppData/Roaming/.emacs.d/elpa/slime-2.18/swank-loader.lisp” CL-USER 4 : 2 > (swank-loader:init) ;;; Compiling file C:\Users\Timmy Jose\AppData\Roaming\.emacs.d\elpa\slime-2.18\packages.lisp ... ;;; Safety = 3, Speed = 1, Space = 1, Float = 1, Interruptible = 1 ;;; Compilation speed = 1, Debug = 2, Fixnum safety = 3 ;;; Source level debugging is on ;;; Source file recording is on ;;; Cross referencing is on <EXTRA MESSAGES ELIDED> CL-USER 5 : 2 >(swank:create-server :port 9999) ;; Swank started at port: 9999. 9999
And from Emacs, invoke the following command (follow the prompts and enter the same port number as that used in the previous command):
M-x slime-connect
And now we can see that we are indeed connected to a LispWorks instance:
; SLIME 2016-04-19 CL-USER> (lisp-implementation-type) "LispWorks Personal Edition” CL-USER> (lisp-implementation-version) "6.1.1"
Follow the same process for an ACL instance as well (after M-x slime-connect from Emacs):
CL-USER> (lisp-implementation-type)
"International Allegro CL Free Express Edition”
CL-USER> (lisp-implementation-version)
"10.0 [Windows] (Dec 2, 2015 16:05)"
("lisp_build 80")
(Note that SLIME can only be connected to a single backend at a time).
Next Steps
Well, I hope things went smoothly, and now you have a stonking hot Common Lisp environment to conquer the world with! On a more serious note, the hardest part should be installing and configuring Emacs itself. However, I would still recommend putting in the effort because the payoff is immense.
Next up — how to compile SBCL from source with threading support on a Mac OS X system.
Talking with a Lisp 