All of the websites that I maintain are being served with Common Lisp, utilizing the Caveman2 web framework.
I’ve wanted to improve developer ergonomics for several common tasks, and consolidate to using the lisp repl for as much automation as possible. Namely, I want to speed up the process of:
- Loading a Lisp package.
- Stepping into a Lisp package.
- Starting a local web server.
- Launch any project specific development facilities.
Using skyefreeman.com as an
example (where you’re currently reading this), I have a makefile that
consolidates a number of tasks. One example is running
make develop starts the browsersync and tailwindcss filewatchers:
# Starts each facility for developing skyefreeman-web. develop: browser-sync start --proxy "localhost:8080" --files "**/*" \ & npx tailwindcss -i ./static/css/main.css -o ./static/css/generated/tailwind.css --watch \
Development of this website currently requires a number of setup commands before I start work:
- Load the package with Quicklisp.
CL-USER> (ql:quickload :skyefreeman-web)
- Step into the package.
in-package :skyefreeman-web)CL-USER> (
- Start the webserver
8080)SKYEFREEMAN-WEB> (start :port
- Start development facilities (from a separate shell)
I have a package that loads automatically when starting up my lisp called skyetools, which I use to collect generally applicable functions between all my projects. Using Roswell to launch and manage a lisp implementation, a file at ~/.roswell/init.lisp will get loaded alongside each startup. I generally try and treat “starting the lisp repl” the same as “starting the computer”.
Everything in the
:skyetools package will be available
from the repl by default (
skyetools is used as a
Step one and two from above can be consolidated into a single
defmacro into-package (keyword) ("Load and move into the package defined by KEYWORD." progn `(keyword) (ql:quickload ,in-package ,keyword))) (
This loads and moves into the package in one fell swoop. We use a macro here to ensure the keyword maintains self-evaluation.
Next, step three and four of “starting the webserver” and “running development facilities” can also be consolidated:
First off, we write a function called
defparameter *make-develop-process* nil) ( defun make-develop () ("Run the 'make develop' script, which starts the dev server." (uiop:chdir *application-root*)let ((process (uiop:launch-program "make develop" (:input :stream :output :stream))) setq *make-develop-process* process))) (
This wraps the
make develop command that used to be
called from an external shell, and launches it as an asynchronous
that gets returned is stored in the
parameter, so that we can inspect and manage it independantly.
make develop is callable from inside
:skyefreeman-web package, we can wrap it inside another
function to start the web server and call
defun start-dev () (if (and (not (running?)) (equalp *make-develop-process* nil)) (progn (8080) (start :port (make-develop))error "Development server is already running."))) (
This checks whether the server is running, and if not, starts everything up.
While this whole exercise results in two small wrappers, this
improvement greatly reduces the cognitive load of getting started on a
new feature or bugfix. We could even take it a step further and wrap
(into-package :skyefreeman-web) and
(skyefreeman-web:start-dev), making this a single command
for switching work contexts immediately - from anywhere in my
For posterity, here’s a number of other functions for stopping the dev server, and managing spawned processes that may be handy
defun process-kill (process) ( (uiop:close-streams process) (uiop:terminate-process process)) defun process-output (process) ("Print the output from the given process." let ((stream (uiop:process-info-output process))) (loop while (listen stream) do (princ (read-line stream)) (terpri)))) ( defparameter *make-develop-process* nil) (defun make-develop () ("Run the 'make develop' script, which starts the dev server." (uiop:chdir *application-root*)let ((process (uiop:launch-program "make develop" (:input :stream :output :stream))) setq *make-develop-process* process))) ( defun start-dev () (if (and (not (running?)) (equalp *make-develop-process* nil)) (progn (8080) (start :port (make-develop))error "Development server is already running."))) ( defun stop-dev () ( (stop) (process-kill *make-develop-process*)setq *make-develop-process* nil) ( ;; This closes browsersync "lsof -t -i tcp:3000 | xargs kill")) (uiop:run-program
I’ve found that spending the time to incorporate external tools into lisp has a compounding effect. Anything can become lisp code, whether it’s a non-lisp programming language, or any format of data. A shell command can easily be bridge and utilized from within a lisp program, as if it was a part of the language.
Scripts, tools, code generation, the web server and everything else. It’s all part of the same lisp program, and writing tools for building your program faster can all live in the same place.
Writing a set of functions for generating new files, html templates, server deployment, or running tests. These can all just be lisp functions that live within the same package/program.
The experience of writing lisp is a constant upward ascent, where every line of code you write has the ability to amplify future output. Your lisp program is the development environment.